zitadel/internal/api/grpc
Silvan dab5d9e756
refactor(eventstore): move push logic to sql (#8816)
# Which Problems Are Solved

If many events are written to the same aggregate id it can happen that
zitadel [starts to retry the push
transaction](48ffc902cc/internal/eventstore/eventstore.go (L101))
because [the locking
behaviour](48ffc902cc/internal/eventstore/v3/sequence.go (L25))
during push does compute the wrong sequence because newly committed
events are not visible to the transaction. These events impact the
current sequence.

In cases with high command traffic on a single aggregate id this can
have severe impact on general performance of zitadel. Because many
connections of the `eventstore pusher` database pool are blocked by each
other.

# How the Problems Are Solved

To improve the performance this locking mechanism was removed and the
business logic of push is moved to sql functions which reduce network
traffic and can be analyzed by the database before the actual push. For
clients of the eventstore framework nothing changed.

# Additional Changes

- after a connection is established prefetches the newly added database
types
- `eventstore.BaseEvent` now returns the correct revision of the event

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/8931

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com>
Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com>
Co-authored-by: Yxnt <Yxnt@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Zach H <zhirschtritt@gmail.com>
2024-12-04 13:51:40 +00:00
..
action feat(v3alpha): write actions (#8225) 2024-07-31 14:42:12 +02:00
admin fix(saml): improve error handling (#8928) 2024-12-03 10:38:28 +00:00
auth fix: user grants deactivation (#8634) 2024-09-17 12:18:29 +00:00
authn chore(v2): move to new org (#3499) 2022-04-26 23:01:45 +00:00
change refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
client/middleware refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
event refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
feature feat(OIDC): add back channel logout (#8837) 2024-10-31 15:57:17 +01:00
gerrors fix: exclude db connection error details (#7785) 2024-04-23 08:35:25 +00:00
idp chore: improve integration tests (#8727) 2024-10-17 21:20:57 +00:00
instance feat: trusted (instance) domains (#8369) 2024-07-31 18:00:38 +03:00
management fix(saml): improve error handling (#8928) 2024-12-03 10:38:28 +00:00
member refactor: cleanup unused code (#7130) 2024-01-02 14:26:31 +00:00
metadata fix(api): correct mapping of metadata queries (#7609) 2024-03-21 14:56:58 +00:00
object feat: api v2beta to api v2 (#8283) 2024-07-26 22:39:55 +02:00
oidc chore: add await for project to oidc integration tests (#8809) 2024-10-23 07:36:50 +00:00
org chore: remove parallel running in integration tests (#8904) 2024-11-27 15:32:13 +01:00
policy feat(cnsl): docs link can be customized and custom button is available (#7840) 2024-05-13 16:01:50 +02:00
project feat(OIDC): add back channel logout (#8837) 2024-10-31 15:57:17 +01:00
resources refactor(eventstore): move push logic to sql (#8816) 2024-12-04 13:51:40 +00:00
server feat: action v2 signing (#8779) 2024-11-28 10:06:52 +00:00
session chore: improve integration tests (#8727) 2024-10-17 21:20:57 +00:00
settings chore: correct require usage to assert for eventual consistency (#8795) 2024-10-21 19:15:02 +00:00
system chore: remove parallel running in integration tests (#8904) 2024-11-27 15:32:13 +01:00
text fix: automatically link user without prompt (#8487) 2024-08-28 05:33:20 +00:00
user fix: consistent permission check on user v2 (#8807) 2024-12-03 10:14:04 +00:00
config.go remove pointers on configs 2020-03-27 13:57:16 +01:00
fields.go chore(tests): use a coverage server binary (#8407) 2024-09-06 14:47:57 +02:00
header_test.go remove negated integration tags 2023-04-26 19:55:13 +03:00
header.go chore(v2): move to new org (#3499) 2022-04-26 23:01:45 +00:00
probes_test.go add server reflection to Probes list 2023-05-07 16:47:43 +02:00
probes.go add server reflection to Probes list 2023-05-07 16:47:43 +02:00