Commit Graph

9 Commits

Author SHA1 Message Date
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Silvan
22af4dcd97
feat(storage): read only transactions (#6417)
feat(storage): read only transactions for queries (#6415)

* fix: tests

* bastle wie en grosse

* fix(database): scan as callback

* fix tests

* fix merge failures

* remove as of system time

* refactor: remove unused test

* refacotr: remove unused lines
2023-08-22 14:49:02 +02:00
Tim Möhlmann
c0e45b63d8
fix: reset the call timestamp after a bulk trigger (#6080)
* reproduce #5808

Add an integration test that imports and gets N amount of human users.
- With N set to 1-10 the operation seems to succeed always
- With N set to 100 the operation seems to fail between 1 and 7 times.

* fix merge issue

* fix: reset the call timestamp after a bulk trigger

With the use of `AS OF SYSTEM TIME` in queries,
there was a change for the query package not
finding the latest projection verson after
a bulk trigger.
If events where processed in the bulk trigger,
the resulting row timestamp would be after the call
start timestamp.
This sometimes resulted in consistency issues when
Set and Get API methods are called in short succession.
For example a Import and Get user could sometimes result in a Not Found
error.

Although the issue was reported for the Management API user import,
it is likely this bug contributed to the flaky integration and e2e tests.

Fixes #5808

* trigger bulk action in GetSession

* don't use the new context in handler schedule

* disable reproduction test

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-07 08:15:05 +00:00
Silvan
e38abdcdf3
perf: query data AS OF SYSTEM TIME (#5231)
Queries the data in the storage layser at the timestamp when the call hit the API layer
2023-02-27 22:36:43 +01:00
Silvan
28760ab4b3
fix(query): add tracing for each method (#4777)
* fix(query): add tracing for each method
2022-12-01 09:18:53 +01:00
Silvan
f3e6f3b23b
feat: remove org (#4148)
* feat(command): remove org

* refactor: imports, unused code, error handling

* reduce org removed in action

* add org deletion to projections

* add org removal to projections

* add org removal to projections

* org removed projection

* lint import

* projections

* fix: table names in tests

* fix: table names in tests

* logging

* add org state

* fix(domain): add Owner removed to object details

* feat(ListQuery): add with owner removed

* fix(org-delete): add bool to functions to select with owner removed

* fix(org-delete): add bools to user grants with events to determine if dependencies lost owner

* fix(org-delete): add unit tests for owner removed and org removed events

* fix(org-delete): add handling of org remove for grants and members

* fix(org-delete): correction of unit tests for owner removed

* fix(org-delete): update projections, unit tests and get functions

* fix(org-delete): add change date to authnkeys and owner removed to org metadata

* fix(org-delete): include owner removed for login names

* fix(org-delete): some column fixes in projections and build for queries with owner removed

* indexes

* fix(org-delete): include review changes

* fix(org-delete): change user projection name after merge

* fix(org-delete): include review changes for project grant where no project owner is necessary

* fix(org-delete): include auth and adminapi tables with owner removed information

* fix(org-delete): cleanup username and orgdomain uniqueconstraints when org is removed

* fix(org-delete): add permissions for org.remove

* remove unnecessary unique constraints

* fix column order in primary keys

* fix(org-delete): include review changes

* fix(org-delete): add owner removed indexes and chang setup step to create tables

* fix(org-delete): move PK order of instance_id and change added user_grant from review

* fix(org-delete): no params for prepareUserQuery

* change to step 6

* merge main

* fix(org-delete): OldUserName rename to private

* fix linting

* cleanup

* fix: remove org test

* create prerelease

* chore: delete org-delete as prerelease

Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2022-11-30 17:01:17 +01:00
Livio Spring
441f215513
fix: join on instanceIDs in queries (#4612) 2022-10-27 06:08:36 +00:00
Livio Spring
aed7010508
fix: scheduling (#3978)
* fix: improve scheduling

* build pre-release

* fix: locker

* fix: user handler and print stack in case of panic in reducer

* chore: remove sentry

* fix: improve handler projection and implement tests

* more tests

* fix: race condition in tests

* Update internal/eventstore/repository/sql/query.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* fix: implemented suggested changes

* fix: lock statement

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-07-22 10:08:39 +00:00
Silvan
dd2f31683c
fix(query): realtime data on defined requests (#3726)
* feat: directly specify factors on addCustomLoginPolicy and return on LoginPolicy responses

* fix proto

* update login policy

* feat: directly specify idp on addCustomLoginPolicy and return on LoginPolicy responses

* fix: tests

* fix(projection): trigger bulk

* refactor: clean projection pkg

* instance should bulk

* fix(query): should trigger bulk on id calls

* tests

* build prerelease

* fix: add shouldTriggerBulk

* fix: test

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2022-06-14 07:51:00 +02:00