TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2024-12-04 23:45:07 +00:00
Code Issues Packages Projects Releases Wiki Activity
main
zitadel/cmd/initialise/sql
History
Silvan dab5d9e756
refactor(eventstore): move push logic to sql (#8816) 
# Which Problems Are Solved

If many events are written to the same aggregate id it can happen that
zitadel [starts to retry the push
transaction](48ffc902cc/internal/eventstore/eventstore.go (L101))
because [the locking
behaviour](48ffc902cc/internal/eventstore/v3/sequence.go (L25))
during push does compute the wrong sequence because newly committed
events are not visible to the transaction. These events impact the
current sequence.

In cases with high command traffic on a single aggregate id this can
have severe impact on general performance of zitadel. Because many
connections of the `eventstore pusher` database pool are blocked by each
other.

# How the Problems Are Solved

To improve the performance this locking mechanism was removed and the
business logic of push is moved to sql functions which reduce network
traffic and can be analyzed by the database before the actual push. For
clients of the eventstore framework nothing changed.

# Additional Changes

- after a connection is established prefetches the newly added database
types
- `eventstore.BaseEvent` now returns the correct revision of the event

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/8931

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com>
Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com>
Co-authored-by: Yxnt <Yxnt@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Zach H <zhirschtritt@gmail.com>
2024-12-04 13:51:40 +00:00
..
cockroach refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
postgres refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
README.md fix: enable env vars in setup steps (and deprecate admin subcommand) (#3871) 2022-06-27 10:32:34 +00:00

README.md

SQL initialisation

The sql-files in this folder initialize the ZITADEL database and user. These objects need to exist before ZITADEL is able to set and start up.

files

  • 01_user.sql: create the user zitadel uses to connect to the database
  • 02_database.sql: create the database for zitadel
  • 03_grant_user.sql: grants the user created before to have full access to its database. The user needs full access to the database because zitadel makes ddl/dml on runtime
  • 04_eventstore.sql: creates the schema needed for eventsourcing
  • 05_projections.sql: creates the schema needed to read the data
  • 06_system.sql: creates the schema needed for ZITADEL itself
  • 07_encryption_keys_table.sql: creates the table for encryption keys (for event data)
  • files 08_enable_hash_sharded_indexes.sql and 09_events_table.sql must run in the same session
    • 08_enable_hash_sharded_indexes.sql enables the hash sharded index feature for this session
    • 09_events_table.sql creates the table for eventsourcing