feat(database): support for postgres (#3998)

* beginning with postgres statements * try pgx * use pgx * database * init works for postgres * arrays working * init for cockroach * init * start tests * tests * TESTS * ch * ch * chore: use go 1.18 * read stmts * fix typo * tests * connection string * add missing error handler * cleanup * start all apis * go mod tidy * old update * switch back to minute * on conflict * replace string slice with `database.StringArray` in db models * fix tests and start * update go version in dockerfile * setup go * clean up * remove notification migration * update * docs: add deploy guide for postgres * fix: revert sonyflake * use `database.StringArray` for daos * use `database.StringArray` every where * new tables * index naming, metadata primary key, project grant role key type * docs(postgres): change to beta * chore: correct compose * fix(defaults): add empty postgres config * refactor: remove unused code * docs: add postgres to self hosted * fix broken link * so? * change title * add mdx to link * fix stmt * update goreleaser in test-code * docs: improve postgres example * update more projections * fix: add beta log for postgres * revert index name change * prerelease * fix: add sequence to v1 "reduce paniced" * log if nil * add logging * fix: log output * fix(import): check if org exists and user * refactor: imports * fix(user): ignore malformed events * refactor: method naming * fix: test * refactor: correct errors.Is call * ci: don't build dev binaries on main * fix(go releaser): update version to 1.11.0 * fix(user): projection should not break * fix(user): handle error properly * docs: correct config example * Update .releaserc.js * Update .releaserc.js Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Elio Bischof <eliobischof@gmail.com>
2025-08-12 01:37:31 +00:00 · 2022-08-31 09:52:43 +02:00
parent d6c9815945
commit 77b4fc5487
189 changed files with 3401 additions and 2956 deletions
--- a/docs/docs/concepts/architecture/software.md
+++ b/docs/docs/concepts/architecture/software.md
@@ -144,4 +144,4 @@ The storage layer of ZITADEL is responsible for multiple things. For example:
 - Backup and restore operation for disaster recovery purpose

 ZITADEL currently supports CockroachDB as first choice of storage due to its perfect match for ZITADELs needs.
-Postgresql support is work in progress and should be available soon as well.
+Postgresql support is currently in beta.
--- a/docs/docs/concepts/architecture/solution.md
+++ b/docs/docs/concepts/architecture/solution.md
@@ -10,7 +10,7 @@ Since the storage layer takes the heavy lifting of making sure that data in sync
 Depending on your projects needs our general recommendation is to run ZITADEL and ZITADELs storage layer across multiple availability zones in the same region or if you need higher guarantees run the storage layer across multiple regions.
 Consult the [CockroachDB documentation](https://www.cockroachlabs.com/docs/) for more details or use the [CockroachCloud Service](https://www.cockroachlabs.com/docs/cockroachcloud/create-an-account.html)

-> Soon ZITADEL will also support Postgres as database.
+> Postgres support of ZITADEL is currently in beta.

 ## Scalability

--- a/docs/docs/guides/deploy/docker-compose.yaml
+++ b/docs/docs/guides/deploy/docker-compose.yaml
@@ -8,19 +8,19 @@ services:
    image: 'ghcr.io/zitadel/zitadel:stable'
    command: 'start-from-init --masterkey "MasterkeyNeedsToHave32Characters" --tlsMode disabled'
    environment:
-      - 'ZITADEL_DATABASE_COCKROACH_HOST=db'
+      - 'ZITADEL_DATABASE_COCKROACH_HOST=crdb'
      - 'ZITADEL_EXTERNALSECURE=false'
    depends_on:
-      db:
+      crdb:
        condition: 'service_healthy'
    ports:
      - '8080:8080'

-  db:
+  crdb:
    restart: 'always'
    networks:
      - 'zitadel'
-    image: 'cockroachdb/cockroach:v22.1.0'
+    image: 'cockroachdb/cockroach:v22.1.3'
    command: 'start-single-node --insecure'
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health?ready=1"]
--- a/docs/docs/guides/deploy/overview.mdx
+++ b/docs/docs/guides/deploy/overview.mdx
@@ -15,7 +15,7 @@ By default, it runs a highly available ZITADEL instance along with a secure and
 ## Prerequisits

 - ZITADEL does not need many resources, 1 CPU and 512MB memory are more than enough. (With more CPU, the password hashing might be faster)
- A cockroachDB or [🚧 Postgresql coming soon](https://github.com/zitadel/zitadel/pull/3998) as only needed storage
+- A cockroachDB or Postgresql (currently in beta) as only needed storage
 - If you want to front ZITADEL with a reverse proxy, web application firewall or content delivery network, make sure to support [HTTP/2](../manage/self-hosted/http2)


--- a/docs/docs/guides/manage/self-hosted/database/_cockroachdb.mdx
+++ b/docs/docs/guides/manage/self-hosted/database/_cockroachdb.mdx
@@ -0,0 +1,32 @@
+## Cockroach
+
+The default database of ZITADEL is [CockroachDB](https://www.cockroachlabs.com). The SQL database provides a bunch of features like horizontal scalability, data reginality and many more.
+
+The default configuration of the database looks like this:
+```yaml
+Database:
+  cockroach:
+    Host: localhost
+    Port: 26257
+    Database: zitadel
+    MaxOpenConns: 20
+    MaxConnLifetime: 30m
+    MaxConnIdleTime: 30m
+    Options: ""
+    User:
+      Username: zitadel
+      Password: ""
+      SSL:
+        Mode: disable
+        RootCert: ""
+        Cert: ""
+        Key: ""
+    Admin:
+      Username: root
+      Password: ""
+      SSL:
+        Mode: disable
+        RootCert: ""
+        Cert: ""
+        Key: ""
+```
--- a/docs/docs/guides/manage/self-hosted/database/_postgres.mdx
+++ b/docs/docs/guides/manage/self-hosted/database/_postgres.mdx
@@ -0,0 +1,36 @@
+## Postgres
+
+:::caution
+Postgres extension is currently in beta.
+:::
+
+If you want to use a Postgres database instead of CockroachDB you can [overwrite the default configuration](../configure/configure.mdx).
+
+Postgres can be configured as follows:
+```yaml
+Database:
+  postgres:
+    Host: localhost
+    Port: 5432
+    Database: zitadel
+    MaxOpenConns: 25
+    MaxConnLifetime: 1h
+    MaxConnIdleTime: 5m
+    Options:
+    User:
+      Username: zitadel
+      Password: zitadel
+      SSL:
+        Mode: disable
+        RootCert:
+        Cert:
+        Key:
+    Admin:
+      Username: postgres
+      Password: postgres
+      SSL:
+        Mode: disable
+        RootCert:
+        Cert:
+        Key:
+```
--- a/docs/docs/guides/manage/self-hosted/database/database.mdx
+++ b/docs/docs/guides/manage/self-hosted/database/database.mdx
@@ -0,0 +1,28 @@
+---
+title: Database
+---
+
+import Tabs from "@theme/Tabs";
+import TabItem from "@theme/TabItem";
+import Cockroach from './_cockroachdb.mdx'
+import Postgres from './_postgres.mdx'
+
+# Database Configuration
+
+<Tabs
+    groupId="database-vendor"
+    default="cockroach"
+    values={[
+        {'label': 'Cockroach', 'value': 'crdb'},
+        {'label': 'Postgres', 'value': 'pg'},
+    ]}
+>
+    <TabItem value="crdb">
+        <Cockroach/>
+        <More/>
+    </TabItem>
+    <TabItem value="pg">
+        <Postgres/>
+        <More/>
+    </TabItem>
+</Tabs>
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -86,6 +86,7 @@ module.exports = {
            "guides/manage/self-hosted/custom-domain",
            "guides/manage/self-hosted/http2",
            "guides/manage/self-hosted/tls_modes",
+            "guides/manage/self-hosted/database/database",
          ]
        },
        {
@@ -95,7 +96,6 @@ module.exports = {
            "guides/manage/console/organizations",
            "guides/manage/console/projects",
            "guides/manage/console/applications",
-
          ]
        },
        {