Commit Graph

6 Commits

Author SHA1 Message Date
Tim Möhlmann
2727fa719d perf(actionsv2): execution target router (#10564)
# Which Problems Are Solved

The event execution system currently uses a projection handler that
subscribes to and processes all events for all instances. This creates a
high static cost because the system over-fetches event data, handling
many events that are not needed by most instances. This inefficiency is
also reflected in high "rows returned" metrics in the database.

# How the Problems Are Solved

Eliminate the use of a project handler. Instead, events for which
"execution targets" are defined, are directly pushed to the queue by the
eventstore. A Router is populated in the Instance object in the authz
middleware.

- By joining the execution targets to the instance, no additional
queries are needed anymore.
- As part of the instance object, execution targets are now cached as
well.
- Events are queued within the same transaction, giving transactional
guarantees on delivery.
- Uses the "insert many fast` variant of River. Multiple jobs are queued
in a single round-trip to the database.
- Fix compatibility with PostgreSQL 15

# Additional Changes

- The signing key was stored as plain-text in the river job payload in
the DB. This violated our [Secrets
Storage](https://zitadel.com/docs/concepts/architecture/secrets#secrets-storage)
principle. This change removed the field and only uses the encrypted
version of the signing key.
- Fixed the target ordering from descending to ascending.
- Some minor linter warnings on the use of `io.WriteString()`.

# Additional Context

- Introduced in https://github.com/zitadel/zitadel/pull/9249
- Closes https://github.com/zitadel/zitadel/issues/10553
- Closes https://github.com/zitadel/zitadel/issues/9832
- Closes https://github.com/zitadel/zitadel/issues/10372
- Closes https://github.com/zitadel/zitadel/issues/10492

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
(cherry picked from commit a9ebc06c77)
2025-09-01 08:16:52 +02:00
Fabi
d8e42744b4 fix: move v2 pkgs (#1331)
* fix: move eventstore pkgs

* fix: move eventstore pkgs

* fix: remove v2 view

* fix: remove v2 view
2021-02-23 15:13:04 +01:00
Fabi
28bfe72930 feat: uniqueness (#1190)
* feat: uniqueness on events

* fix: some tests

* fix: add unique username

* fix: nice error message

* fix: add unique test

* fix: add unique test

* fix: add unique constraint to events

* fix: correct unique constraint on user events

* fix: remove user constraint

* fix: add unique constraints

* fix: add unique constraints

* fix: add unique constraints

* fix: unnique constraints without interface

* fix: unique idp config

* fix: unique constraint comments

* fix: unique constraints in one table

* fix: unique constraints error

* fix: fix unique constraint on create user

* fix: fix unique constraint on create project

* fix: fix unique constraint on idp configs
2021-01-21 10:49:38 +01:00
Silvan
dd5e4acd24 fix(event handling): use internal pubsub for view update (#1118)
* start sub

* start implement subsciptions

* start subscription

* implementation for member done

* admin done

* fix: tests

* extend handlers

* prepary notification

* no errors in adminapi

* changed current sequence in all packages

* ignore mocks

* works

* subscriptions as singleton

* tests

* refactor: rename function scope var
2020-12-18 16:47:45 +01:00
Silvan
41e1a7cc7b fix(spooler): correct workers (#508)
* one concurrent task

* disable spooler

* fix: improve concurrency in spooler

* fix: dont block lock

* fix: break if lock failed

* fix: check if handler is working

* fix: worker id

* fix: test

* fix: use limit for spoolers configured in startup.yaml

* fix test

* fix: factory

* fix(key): only reduce if not expired

* fix(searchQueryFactory): check for string-slice in aggregateID

* fix(migrations): combine migrations

* fix: allow saving multiple objects in one request

* fix(eventstore): logging

* fix(eventstore): rethink insert i locks table

* fix: ignore failed tests for the moment

* fix: tuubel

* fix: for tests in io

* fix: ignore tests for io

* fix: rename concurrent tasks to workers

* fix: incomment tests and remove some tests

* fix: refert changes for io

* refactor(eventstore): combine types of sql in one file

* refactor(eventstore): logs, TODO's, tests

* fix(eventstore): sql package

* test(eventstore): add tests for search query factory

* chore: logs

* fix(spooler): optimize lock query
chore(migrations): rename locks.object_type to view_name
chore(migrations): refactor migrations

* test: incomment tests

* fix: rename PrepareSaves to PrepareBulkSave

* chore: go dependencies

* fix(migrations): add id in events table

* refactor(lock): less magic numbers

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2020-07-28 09:42:21 +02:00
Silvan
fbeab4c582 Eventstore (#22)
* feat: eventstore repository

* fix: remove gorm

* version

* feat: pkg

* feat: eventstore without eventstore-lib

* rename files

* gnueg

* fix: add object

* fix: global model

* feat(eventstore): sdk

* fix(eventstore): search query

* fix(eventstore): rename app to eventstore

* delete empty test

* remove unused func

* merge master

* fix(eventstore): tests

* fix(models): delete unused struct

* feat(eventstore): implemented push events

* feat(eventstore): overwrite context data

* feat(types): SQL-config

* feat(eventstore): options to overwrite editor

* Update internal/eventstore/models/field.go

Co-Authored-By: livio-a <livio.a@gmail.com>

* fix(eventstore): code quality

* fix(eventstore): rename modifier* to editor*

* fix(eventstore): delete editor_org

Co-authored-by: Fabiennne <fabienne.gerschwiler@gmail.com>
Co-authored-by: livio-a <livio.a@gmail.com>
2020-04-06 06:42:21 +02:00