zitadel/internal
Silvan dab5d9e756
refactor(eventstore): move push logic to sql (#8816)
# Which Problems Are Solved

If many events are written to the same aggregate id it can happen that
zitadel [starts to retry the push
transaction](48ffc902cc/internal/eventstore/eventstore.go (L101))
because [the locking
behaviour](48ffc902cc/internal/eventstore/v3/sequence.go (L25))
during push does compute the wrong sequence because newly committed
events are not visible to the transaction. These events impact the
current sequence.

In cases with high command traffic on a single aggregate id this can
have severe impact on general performance of zitadel. Because many
connections of the `eventstore pusher` database pool are blocked by each
other.

# How the Problems Are Solved

To improve the performance this locking mechanism was removed and the
business logic of push is moved to sql functions which reduce network
traffic and can be analyzed by the database before the actual push. For
clients of the eventstore framework nothing changed.

# Additional Changes

- after a connection is established prefetches the newly added database
types
- `eventstore.BaseEvent` now returns the correct revision of the event

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/8931

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com>
Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com>
Co-authored-by: Yxnt <Yxnt@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Zach H <zhirschtritt@gmail.com>
2024-12-04 13:51:40 +00:00
..
actions fix: correctly check denied domains and ips for actions (#8810) 2024-10-22 16:16:44 +02:00
activity feat: trusted (instance) domains (#8369) 2024-07-31 18:00:38 +03:00
admin/repository/eventsourcing feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
api refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
auth/repository feat(OIDC): add back channel logout (#8837) 2024-10-31 15:57:17 +01:00
auth_request/repository fix(login): correctly reload policies on auth request (#7839) 2024-04-24 08:44:55 +00:00
authz docs: standardize multi-factor spelling and related string updates (#8752) 2024-10-22 14:59:16 +00:00
cache feat(cache): organization (#8903) 2024-11-21 08:05:03 +02:00
command fix: project existing check removed from project grant remove (#9004) 2024-12-03 14:38:25 +00:00
config feat(v3alpha): read actions (#8357) 2024-08-12 22:32:01 +02:00
crypto perf(query): remove transactions for queries (#8614) 2024-11-04 10:06:14 +01:00
database refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
domain feat: action v2 signing (#8779) 2024-11-28 10:06:52 +00:00
eventstore refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
execution feat: action v2 signing (#8779) 2024-11-28 10:06:52 +00:00
feature feat(OIDC): add back channel logout (#8837) 2024-10-31 15:57:17 +01:00
form refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
i18n fix(middleware): init translation messages (#7778) 2024-04-16 12:08:18 +00:00
iam refactor: cleanup unused code (#7130) 2024-01-02 14:26:31 +00:00
id feat(cmd): Added machine ID mode to zitadel start up logs (#8251) 2024-07-16 09:53:57 +00:00
idp fix(saml): improve error handling (#8928) 2024-12-03 10:38:28 +00:00
integration feat(cache): organization (#8903) 2024-11-21 08:05:03 +02:00
logstore perf: project quotas and usages (#6441) 2023-09-15 16:58:45 +02:00
migration fix(setup): improve search query to use index (#8898) 2024-11-13 07:50:23 +00:00
net perf: project quotas and usages (#6441) 2023-09-15 16:58:45 +02:00
notification feat(i18n): add korean language support (#8879) 2024-12-02 13:11:31 +00:00
org refactor: cleanup unused code (#7130) 2024-01-02 14:26:31 +00:00
project refactor: cleanup unused code (#7130) 2024-01-02 14:26:31 +00:00
protoc merge main into next 2023-10-19 12:34:00 +02:00
qrcode
query feat: action v2 signing (#8779) 2024-11-28 10:06:52 +00:00
renderer fix(login): (re)allow HTML in custom login texts (#7575) 2024-03-15 16:29:10 +01:00
repository feat: action v2 signing (#8779) 2024-11-28 10:06:52 +00:00
static feat(i18n): add korean language support (#8879) 2024-12-02 13:11:31 +00:00
statik chore: initial version of a devcontainer (#6352) 2023-08-15 10:49:05 +02:00
telemetry feat: enable application performance profiling (#8442) 2024-08-16 13:26:53 +00:00
test refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
user feat(OIDC): add back channel logout (#8837) 2024-10-31 15:57:17 +01:00
v2 fix(eventstore): revert precise decimal (#8527) (#8679) 2024-09-24 18:43:29 +02:00
view/repository perf(query): remove transactions for queries (#8614) 2024-11-04 10:06:14 +01:00
webauthn fix: add domain as attribute to list user auth methods (#8718) 2024-10-10 16:50:53 +00:00
zerrors fix(crypto): reject decrypted strings with non-UTF8 characters. (#8374) 2024-08-02 08:38:37 +00:00