69 Commits

Author	SHA1	Message	Date
Silvan	1ee7a1ab7c	feat(eventstore): accept transaction in push (#8945) ... # Which Problems Are Solved Push is not capable of external transactions. # How the Problems Are Solved A new function `PushWithClient` is added to the eventstore framework which allows to pass a client which can either be a `*sql.Client` or `*sql.Tx` and is used during push. # Additional Changes Added interfaces to database package. # Additional Context - part of https://github.com/zitadel/zitadel/issues/8931 --------- Co-authored-by: Livio Spring <livio.a@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>	2024-11-22 17:25:28 +01:00
Tim Möhlmann	d4389ab359	feat(eventstore): add row locking option (#8939) ... # Which Problems Are Solved We need a reliable way to lock events that are being processed as part of a job queue. For example in the notification handlers. # How the Problems Are Solved Allow setting `FOR UPDATE [ NOWAIT | SKIP LOCKED ]` to the eventstore query builder using an open transaction. - NOWAIT returns an errors if the lock cannot be obtained - SKIP LOCKED only returns row which are not locked. - Default is to wait for the lock to be released. # Additional Changes - none # Additional Context - [Locking docs](https://www.postgresql.org/docs/17/sql-select.html#SQL-FOR-UPDATE-SHARE) - Related to https://github.com/zitadel/zitadel/issues/8931	2024-11-21 14:46:30 +00:00
Silvan	522c82876f	fix(eventstore): set application name during push to instance id (#8918) ... # Which Problems Are Solved Noisy neighbours can introduce projection latencies because the projections only query events older than the start timestamp of the oldest push transaction. # How the Problems Are Solved During push we set the application name to `zitadel_es_pusher_<instance_id>` instead of `zitadel_es_pusher` which is used to query events by projections.	2024-11-18 15:30:12 +00:00
Silvan	9c3e5e467b	perf(query): remove transactions for queries (#8614) ... # Which Problems Are Solved Queries currently execute 3 statements, begin, query, commit # How the Problems Are Solved remove transaction handling from query methods in database package # Additional Changes - Bump versions of `core_grpc_dependencies`-receipt in Makefile # Additional info During load tests we saw a lot of idle transactions of `zitadel_queries` application name which is the connection pool used to query data in zitadel. Executed query: `select query_start - xact_start, pid, application_name, backend_start, xact_start, query_start, state_change, wait_event_type, wait_event,substring(query, 1, 200) query from pg_stat_activity where datname = 'zitadel' and state <> 'idle';` Mostly the last query executed was `begin isolation level read committed read only`. example: ``` ?column? | pid | application_name | backend_start | xact_start | query_start | state_change | wait_event_type | wait_event | query -----------------+-------+----------------------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+--------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 00:00:00 | 33030 | zitadel_queries | 2024-10-16 16:25:53.906036+00 | 2024-10-16 16:30:19.191661+00 | 2024-10-16 16:30:19.191661+00 | 2024-10-16 16:30:19.19169+00 | Client | ClientRead | begin isolation level read committed read only 00:00:00 | 33035 | zitadel_queries | 2024-10-16 16:25:53.909629+00 | 2024-10-16 16:30:19.19179+00 | 2024-10-16 16:30:19.19179+00 | 2024-10-16 16:30:19.191805+00 | Client | ClientRead | begin isolation level read committed read only 00:00:00.00412 | 33028 | zitadel_queries | 2024-10-16 16:25:53.904247+00 | 2024-10-16 16:30:19.187734+00 | 2024-10-16 16:30:19.191854+00 | 2024-10-16 16:30:19.191964+00 | Client | ClientRead | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 00:00:00.084662 | 33134 | zitadel_es_pusher | 2024-10-16 16:29:54.979692+00 | 2024-10-16 16:30:19.178578+00 | 2024-10-16 16:30:19.26324+00 | 2024-10-16 16:30:19.263267+00 | Client | ClientRead | RELEASE SAVEPOINT cockroach_restart 00:00:00.084768 | 33139 | zitadel_es_pusher | 2024-10-16 16:29:54.979585+00 | 2024-10-16 16:30:19.180762+00 | 2024-10-16 16:30:19.26553+00 | 2024-10-16 16:30:19.265531+00 | LWLock | WALWriteLock | commit 00:00:00.077377 | 33136 | zitadel_es_pusher | 2024-10-16 16:29:54.978582+00 | 2024-10-16 16:30:19.187883+00 | 2024-10-16 16:30:19.26526+00 | 2024-10-16 16:30:19.265431+00 | Client | ClientRead | WITH existing AS ( + | | | | | | | | | (SELECT instance_id, aggregate_type, aggregate_id, "sequence" FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type = $2 AND aggregate_id = $3 ORDER BY "sequence" DE 00:00:00.012309 | 33123 | zitadel_es_pusher | 2024-10-16 16:29:54.963484+00 | 2024-10-16 16:30:19.175066+00 | 2024-10-16 16:30:19.187375+00 | 2024-10-16 16:30:19.187376+00 | IO | WalSync | commit 00:00:00 | 33034 | zitadel_queries | 2024-10-16 16:25:53.90791+00 | 2024-10-16 16:30:19.262921+00 | 2024-10-16 16:30:19.262921+00 | 2024-10-16 16:30:19.263133+00 | Client | ClientRead | begin isolation level read committed read only 00:00:00 | 33039 | zitadel_queries | 2024-10-16 16:25:53.914106+00 | 2024-10-16 16:30:19.191676+00 | 2024-10-16 16:30:19.191676+00 | 2024-10-16 16:30:19.191687+00 | Client | ClientRead | begin isolation level read committed read only 00:00:00.24539 | 33083 | zitadel_projection_spooler | 2024-10-16 16:27:49.895548+00 | 2024-10-16 16:30:19.020058+00 | 2024-10-16 16:30:19.265448+00 | 2024-10-16 16:30:19.26546+00 | Client | ClientRead | SAVEPOINT exec_stmt 00:00:00 | 33125 | zitadel_es_pusher | 2024-10-16 16:29:54.963859+00 | 2024-10-16 16:30:19.191715+00 | 2024-10-16 16:30:19.191715+00 | 2024-10-16 16:30:19.191729+00 | Client | ClientRead | begin 00:00:00.004292 | 33032 | zitadel_queries | 2024-10-16 16:25:53.906624+00 | 2024-10-16 16:30:19.187713+00 | 2024-10-16 16:30:19.192005+00 | 2024-10-16 16:30:19.192062+00 | Client | ClientRead | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 00:00:00 | 33031 | zitadel_queries | 2024-10-16 16:25:53.906422+00 | 2024-10-16 16:30:19.191625+00 | 2024-10-16 16:30:19.191625+00 | 2024-10-16 16:30:19.191645+00 | Client | ClientRead | begin isolation level read committed read only ``` The amount of idle transactions is significantly less if the query transactions are removed: example: ``` ?column? | pid | application_name | backend_start | xact_start | query_start | state_change | wait_event_type | wait_event | query -----------------+-------+----------------------------+-------------------------------+-------------------------------+-------------------------------+-------------------------------+-----------------+------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 00:00:00.000094 | 32741 | zitadel_queries | 2024-10-16 16:23:49.73935+00 | 2024-10-16 16:24:59.785589+00 | 2024-10-16 16:24:59.785683+00 | 2024-10-16 16:24:59.785684+00 | | | SELECT created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision FROM eventstore.events2 WHERE instance_id = $1 AND aggregate_type 00:00:00 | 32762 | zitadel_es_pusher | 2024-10-16 16:24:02.275136+00 | 2024-10-16 16:24:59.784586+00 | 2024-10-16 16:24:59.784586+00 | 2024-10-16 16:24:59.784607+00 | Client | ClientRead | begin 00:00:00.000167 | 32742 | zitadel_queries | 2024-10-16 16:23:49.740489+00 | 2024-10-16 16:24:59.784274+00 | 2024-10-16 16:24:59.784441+00 | 2024-10-16 16:24:59.784442+00 | | | with usr as ( + | | | | | | | | | select u.id, u.creation_date, u.change_date, u.sequence, u.state, u.resource_owner, u.username, n.login_name as preferred_login_name + | | | | | | | | | from projections.users13 u + | | | | | | | | | left join projections.l 00:00:00.256014 | 32759 | zitadel_projection_spooler | 2024-10-16 16:24:01.418429+00 | 2024-10-16 16:24:59.52959+00 | 2024-10-16 16:24:59.785604+00 | 2024-10-16 16:24:59.785649+00 | Client | ClientRead | UPDATE projections.milestones SET reached_date = $1 WHERE (instance_id = $2) AND (type = $3) AND (reached_date IS NULL) 00:00:00.014199 | 32773 | zitadel_es_pusher | 2024-10-16 16:24:02.320404+00 | 2024-10-16 16:24:59.769509+00 | 2024-10-16 16:24:59.783708+00 | 2024-10-16 16:24:59.783709+00 | IO | WalSync | commit 00:00:00 | 32765 | zitadel_es_pusher | 2024-10-16 16:24:02.28173+00 | 2024-10-16 16:24:59.780413+00 | 2024-10-16 16:24:59.780413+00 | 2024-10-16 16:24:59.780426+00 | Client | ClientRead | begin 00:00:00.012729 | 32777 | zitadel_es_pusher | 2024-10-16 16:24:02.339737+00 | 2024-10-16 16:24:59.767432+00 | 2024-10-16 16:24:59.780161+00 | 2024-10-16 16:24:59.780195+00 | Client | ClientRead | RELEASE SAVEPOINT cockroach_restart ``` --------- Co-authored-by: Tim Möhlmann <tim+github@zitadel.com> Co-authored-by: Livio Spring <livio.a@gmail.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Elio Bischof <elio@zitadel.com> Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com> Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com> Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com> Co-authored-by: Yxnt <Yxnt@users.noreply.github.com> Co-authored-by: Stefan Benz <stefan@caos.ch> Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com> Co-authored-by: Zach H <zhirschtritt@gmail.com>	2024-11-04 10:06:14 +01:00
Tim Möhlmann	32bad3feb3	perf(milestones): refactor (#8788) ... # Which Problems Are Solved Milestones used existing events from a number of aggregates. OIDC session is one of them. We noticed in load-tests that the reduction of the oidc_session.added event into the milestone projection is a costly business with payload based conditionals. A milestone is reached once, but even then we remain subscribed to the OIDC events. This requires the projections.current_states to be updated continuously. # How the Problems Are Solved The milestone creation is refactored to use dedicated events instead. The command side decides when a milestone is reached and creates the reached event once for each milestone when required. # Additional Changes In order to prevent reached milestones being created twice, a migration script is provided. When the old `projections.milestones` table exist, the state is read from there and `v2` milestone aggregate events are created, with the original reached and pushed dates. # Additional Context - Closes https://github.com/zitadel/zitadel/issues/8800	2024-10-28 08:29:34 +00:00
Tim Möhlmann	a84b259e8c	perf(oidc): nest position clause for session terminated query (#8738) ... # Which Problems Are Solved Optimize the query that checks for terminated sessions in the access token verifier. The verifier is used in auth middleware, userinfo and introspection. # How the Problems Are Solved The previous implementation built a query for certain events and then appended a single `PositionAfter` clause. This caused the postgreSQL planner to use indexes only for the instance ID, aggregate IDs, aggregate types and event types. Followed by an expensive sequential scan for the position. This resulting in internal over-fetching of rows before the final filter was applied.  Furthermore, the query was searching for events which are not always applicable. For example, there was always a session ID search and if there was a user ID, we would also search for a browser fingerprint in event payload (expensive). Even if those argument string would be empty. This PR changes: 1. Nest the position query, so that a full `instance_id, aggregate_id, aggregate_type, event_type, "position"` index can be matched. 2. Redefine the `es_wm` index to include the `position` column. 3. Only search for events for the IDs that actually have a value. Do not search (noop) if none of session ID, user ID or fingerpint ID are set. New query plan:  # Additional Changes - cleanup how we load multi-statement migrations and make that a bit more reusable. # Additional Context - Related to https://github.com/zitadel/zitadel/issues/7639	2024-10-07 12:49:55 +00:00
Tim Möhlmann	aeb379e7de	fix(eventstore): revert precise decimal (#8527) (#8679)	2024-09-24 18:43:29 +02:00
Silvan	b522588d98	fix(eventstore): precise decimal (#8527) ... # Which Problems Are Solved Float64 which was used for the event.Position field is [not precise in go and gets rounded](https://github.com/golang/go/issues/47300). This can lead to unprecies position tracking of events and therefore projections especially on cockcoachdb as the position used there is a big number. example of a unprecies position: exact: 1725257931223002628 float64: 1725257931223002624.000000 # How the Problems Are Solved The float64 was replaced by [github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal). # Additional Changes Correct behaviour of makefile for load tests. Rename `latestSequence`-queries to `latestPosition`	2024-09-06 12:19:19 +03:00
Tim Möhlmann	64a3bb3149	feat(v3alpha): web key resource (#8262) ... # Which Problems Are Solved Implement a new API service that allows management of OIDC signing web keys. This allows users to manage rotation of the instance level keys. which are currently managed based on expiry. The API accepts the generation of the following key types and parameters: - RSA keys with 2048, 3072 or 4096 bit in size and: - Signing with SHA-256 (RS256) - Signing with SHA-384 (RS384) - Signing with SHA-512 (RS512) - ECDSA keys with - P256 curve - P384 curve - P512 curve - ED25519 keys # How the Problems Are Solved Keys are serialized for storage using the JSON web key format from the `jose` library. This is the format that will be used by OIDC for signing, verification and publication. Each instance can have a number of key pairs. All existing public keys are meant to be used for token verification and publication the keys endpoint. Keys can be activated and the active private key is meant to sign new tokens. There is always exactly 1 active signing key: 1. When the first key for an instance is generated, it is automatically activated. 2. Activation of the next key automatically deactivates the previously active key. 3. Keys cannot be manually deactivated from the API 4. Active keys cannot be deleted # Additional Changes - Query methods that later will be used by the OIDC package are already implemented. Preparation for #8031 - Fix indentation in french translation for instance event - Move user_schema translations to consistent positions in all translation files # Additional Context - Closes #8030 - Part of #7809 --------- Co-authored-by: Elio Bischof <elio@zitadel.com>	2024-08-14 14:18:14 +00:00
Silvan	1d84635836	feat(eventstore): add search table (#8191) ... # Which Problems Are Solved To improve performance a new table and method is implemented on eventstore. The goal of this table is to index searchable fields on command side to use it on command and query side. The table allows to store one primitive value (numeric, text) per row. The eventstore framework is extended by the `Search`-method which allows to search for objects. The `Command`-interface is extended by the `SearchOperations()`-method which does manipulate the the `search`-table. # How the Problems Are Solved This PR adds the capability of improving performance for command and query side by using the `Search`-method of the eventstore instead of using one of the `Filter`-methods. # Open Tasks - [x] Add feature flag - [x] Unit tests - [ ] ~~Benchmarks if needed~~ - [x] Ensure no behavior change - [x] Add setup step to fill table with current data - [x] Add projection which ensures data added between setup and start of the new version are also added to the table # Additional Changes The `Search`-method is currently used by `ProjectGrant`-command side. # Additional Context - Closes https://github.com/zitadel/zitadel/issues/8094	2024-07-03 15:00:56 +00:00
Tim Möhlmann	f9742a58f4	fix(import): add tracing spans to all import related functions (#8160) ... # Which Problems Are Solved This fix adds tracing spans to all V1 API import related functions. This is to troubleshoot import related performance issues reported to us. # How the Problems Are Solved Add a tracing span to `api/grpc/admin/import.go` and all related functions that are called in the `command` package. # Additional Changes - none # Additional Context - Reported by internal communication	2024-06-19 12:56:33 +02:00
Silvan	4dc86c2415	fix(eventstore): order by sequence if filter by aggregate id (#8019) ... # Which Problems Are Solved Queriying events by an aggregate id can produce high loads on the database if the aggregate id contains many events (count > 1000000). # How the Problems Are Solved Instead of using the postion and in_tx_order columns we use the sequence column which guarantees correct ordering in a single aggregate and uses more optimised indexes. # Additional Context Closes https://github.com/zitadel/DevOps/issues/50 Co-authored-by: Livio Spring <livio.a@gmail.com>	2024-05-28 06:49:30 +00:00
Silvan	cff0f73e24	fix(init): add setting to enable durable locks on crdb (#7982) ... feat(init): add setting to enable durable locks on crdb	2024-05-27 09:03:34 +00:00
Stefan Benz	6dcdef0268	fix: add action v2 execution to features (#7597) ... * fix: add action v2 execution to features * fix: add action v2 execution to features * fix: add action v2 execution to features * fix: update internal/command/instance_features_model.go Co-authored-by: Tim Möhlmann <tim+github@zitadel.com> * fix: merge back main * fix: merge back main * fix: rename feature and service * fix: rename feature and service * fix: review changes * fix: review changes --------- Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>	2024-04-09 20:21:21 +03:00
Silvan	56df515e5f	chore: use pgx v5 (#7577) ... * chore: use pgx v5 * chore: update go version * remove direct pq dependency * remove unnecessary type * scan test * map scanner * converter * uint8 number array * duration * most unit tests work * unit tests work * chore: coverage * go 1.21 * linting * int64 gopfertammi * retry go 1.22 * retry go 1.22 * revert to go v1.21.5 * update go toolchain to 1.21.8 * go 1.21.8 * remove test flag * go 1.21.5 * linting * update toolchain * use correct array * use correct array * add byte array * correct value * correct error message * go 1.21 compatible	2024-03-27 15:48:22 +02:00
Silvan	4224c7ad3a	chore(core): update dependencies (#7517) ... * chore(core): update dependencies * chore(core): update dependencies	2024-03-11 10:43:56 +00:00
Elio Bischof	38777b478e	perf: filter events by instance ids (#7489) ... fix: filter events by instance ids	2024-03-04 07:56:48 +01:00
Silvan	17953e9040	fix(setup): init projections (#7194) ... Even though this is a feature it's released as fix so that we can back port to earlier revisions. As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version. The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.	2024-01-25 17:28:20 +01:00
Livio Spring	8470649ecb	chore: pin crdb version for unit tests (#7260) ... * chore: pin crdb version for unit tests * use latest 23.1 version * use latest available 23.1 version	2024-01-18 08:16:54 +00:00
Elio Bischof	ed0bc39ea4	feat: block instances (#7129) ... * docs: fix init description typos * feat: block instances using limits * translate * unit tests * fix translations * redirect /ui/login * fix http interceptor * cleanup * fix http interceptor * fix: delete cookies on gateway 200 * add integration tests * add command test * docs * fix integration tests * add bulk api and integration test * optimize bulk set limits * unit test bulk limits * fix broken link * fix assets middleware * fix broken link * validate instance id format * Update internal/eventstore/search_query.go Co-authored-by: Livio Spring <livio.a@gmail.com> * remove support for owner bulk limit commands * project limits to instances * migrate instances projection * Revert "migrate instances projection" This reverts commit 214218732a. * join limits, remove owner * remove todo * use optional bool * normally validate instance ids * use 302 * cleanup * cleanup * Update internal/api/grpc/system/limits_converter.go Co-authored-by: Livio Spring <livio.a@gmail.com> * remove owner * remove owner from reset --------- Co-authored-by: Livio Spring <livio.a@gmail.com>	2024-01-17 10:16:48 +00:00
Silvan	b7d027e2fd	fix(db): always use begin tx (#7142) ... * fix(db): always use begin tx * fix(handler): timeout for begin	2024-01-04 16:12:20 +00:00
Silvan	5ce542b959	fix(handler): allow uint32 offset for migration scenarios (#7103)	2023-12-21 10:40:51 +00:00
Tim Möhlmann	fe1337536f	fix(db): add additional connection pool for projection spooling (#7094) ... * fix(db): add additional connection pool for projection spooling * use correct connection pool for projections --------- Co-authored-by: Livio Spring <livio.a@gmail.com>	2023-12-20 16:13:04 +00:00
Tim Möhlmann	f680dd934d	refactor: rename package errors to zerrors (#7039) ... * chore: rename package errors to zerrors * rename package errors to gerrors * fix error related linting issues * fix zitadel error assertion * fix gosimple linting issues * fix deprecated linting issues * resolve gci linting issues * fix import structure --------- Co-authored-by: Elio Bischof <elio@zitadel.com>	2023-12-08 15:30:55 +01:00
Tim Möhlmann	ec03340b67	perf(oidc): optimize client verification (#6999) ... * fix some spelling errors * client credential auth * implementation of client auth * improve error handling * unit test command package * unit test database package * unit test query package * cleanup unused tracing func * fix integration tests * errz to zerrors * fix linting and import issues * fix another linting error * integration test with client secret * Revert "integration test with client secret" This reverts commit 0814ba522f. * add integration tests * client credentials integration test * resolve comments * pin oidc v3.5.0	2023-12-05 17:01:03 +00:00
Silvan	e3d1ca4d58	fix(eventstore): improve pagination of handler filter (#6968) ... * fix(setup): add filter_offset to `projections.current_states` * fix(eventstore): allow offset in query * fix(handler): offset for already processed events	2023-12-01 12:25:41 +00:00
Tim Möhlmann	2f91679623	chore(Makefile): add go generate target (#6944) ... This change adds a core_generate_all make target. It installs the required tools and runs generate on the complete project. `golang/mock` is no longer maintained and a fork is available from the Uber folks. So the latter is used as tool. All the mock files have been regenerated and are part of the PR. The obsolete `tools` directory has been removed, as all the tools are now part of specific make targets. Co-authored-by: Silvan <silvan.reusser@gmail.com>	2023-11-22 10:56:43 +00:00
Elio Bischof	1b6e3dcf27	fix: creation date argument in search events filters (#6855) ... * fix: creation date filter in event queries * fix: creation date with ordering filter * simplify code * simplify review --------- Co-authored-by: Livio Spring <livio.a@gmail.com>	2023-11-03 14:52:48 +00:00
Tim Möhlmann	ab79855cf0	fix(eventstore): prevent allocation of filtered events (#6749) ... * fix(eventstore): prevent allocation of filtered events Directly reduce each event obtained from a sql.Rows scan, so that we do not have to allocate all events in a slice. * reinstate the mutex as RWMutex * scan data directly * add todos * fix(writemodels): add reduce of parent * test: remove comment * update comments --------- Co-authored-by: adlerhurst <silvan.reusser@gmail.com>	2023-10-19 15:21:31 +00:00
Silvan	b5564572bc	feat(eventstore): increase parallel write capabilities (#5940) ... This implementation increases parallel write capabilities of the eventstore. Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006). The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`. If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.	2023-10-19 12:19:10 +02:00
Silvan	e64428b8d4	fix: TestCRDB_CreateInstance (#6522) ... (cherry picked from commit 856d0d12e8)	2023-09-12 17:06:01 +02:00
Silvan	22af4dcd97	feat(storage): read only transactions (#6417) ... feat(storage): read only transactions for queries (#6415) * fix: tests * bastle wie en grosse * fix(database): scan as callback * fix tests * fix merge failures * remove as of system time * refactor: remove unused test * refacotr: remove unused lines	2023-08-22 14:49:02 +02:00
Tim Möhlmann	09aafb35eb	feat(v2): implement user register OTP (#6030) ... * feat(v2): implement user register OTP * feat(v2): implement user verify OTP * session: retry on permission denied	2023-06-20 10:36:21 +00:00
Elio Bischof	61feb9d19f	fix: more silence (#5986)	2023-06-06 15:12:54 +00:00
Silvan	39bdef35e7	chore: merge (#5773) ... * feat: allow skip of success page for native apps (#5627) add possibility to return to callback directly after login without rendering the successful login page * build next * fix(console): disallow inline fonts, critical styles (#5714) fix: disallow inline * fix(setup): step 10 for postgres (#5717) * fix(setup): smaller transactions (#5743) * fix: order by sequence by default * test: add allowCreationDateFilter * fix(step10): separate executions (#5754) * feat: allow skip of success page for native apps (#5627) add possibility to return to callback directly after login without rendering the successful login page * build next * fix(console): disallow inline fonts, critical styles (#5714) fix: disallow inline * fix(setup): step 10 for postgres (#5717) * fix(setup): smaller transactions (#5743) * fix(step10): split statements * fix(step10): split into separate execs * chore: prerelease * add truncate before insert * fix: add truncate * Merge branch 'main' into optimise-step-10 * chore: reset release definition --------- Co-authored-by: Livio Spring <livio.a@gmail.com> Co-authored-by: Max Peintner <max@caos.ch> --------- Co-authored-by: Livio Spring <livio.a@gmail.com> Co-authored-by: Max Peintner <max@caos.ch>	2023-04-28 14:56:51 +00:00
James Schinner	2a79e77c7b	docs: Fix typo (#5727) ... Fix typo	2023-04-24 13:23:38 +02:00
Silvan	8da8fbe6ce	fix(eventstore): correct creation date of events (#5683) ... * fix: add setup step to correct creation dates * fix(eventstore): replace now with statement ts * fix(step10): correct number * fix: handle wrong instance domain removed events	2023-04-18 19:29:04 +02:00
Silvan	4c1169b562	feat(eventstore): order by creation_date and sequence (#5568) ... * feat(eventstore): order by `creation_date` and `sequence` * fix(logstore): use correct event type --------- Co-authored-by: Livio Spring <livio.a@gmail.com>	2023-04-05 09:34:24 +00:00
Elio Bischof	62bd606593	feat: enable handling projections for inactive instances (#5523) ... * fix: don't ignore failed events in handlers * question * fix retries * don't instance ids query * statements can be nil * make unit tests pass * add comments * spool only active instances * feat(config): handle inactive instances * customizable HandleInactiveInstances * test: handling with and w/o inactive instances * docs: describe projection options * enable global handling of inactive instances * accept NowFunc, not Clock interface * add comment about stringer usage * remove enum stringer implementations * fix enum format types * Update internal/eventstore/repository/mock/repository.mock.impl.go Co-authored-by: Silvan <silvan.reusser@gmail.com> --------- Co-authored-by: Silvan <silvan.reusser@gmail.com>	2023-03-27 12:34:01 +00:00
Silvan	e38abdcdf3	perf: query data AS OF SYSTEM TIME (#5231) ... Queries the data in the storage layser at the timestamp when the call hit the API layer	2023-02-27 22:36:43 +01:00
Stefan Benz	5d17da542d	fix(user): add search query for login name (#4173) ... * fix(user): add search query for login name * fix(user): change login name query to IN from EXISTS * fix(loginname): include InQuery into ListQuery with SubSelect as possible datasource * fix(user): apply suggestions from code review Co-authored-by: Livio Spring <livio.a@gmail.com> * fix: correct unit test for search query Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com> Co-authored-by: Livio Spring <livio.a@gmail.com>	2022-10-31 13:03:23 +00:00
Livio Spring	d721f725fd	fix: instance remove (#4602)	2022-10-26 13:06:48 +00:00
Livio Spring	5aa91ad105	fix: improve performance (#4300) ... ## Note This release requires a setup step to fully improve performance. Be sure to start ZITADEL with an appropriate command (zitadel start-from-init / start-from-setup) ## Changes - fix: only run projection scheduler on active instances - fix: set default for concurrent instances of projections to 1 (for scheduling) - fix: create more indexes on eventstore.events table - fix: get current sequence for token check (improve reread performance)	2022-09-02 14:05:13 +00:00
Silvan	77b4fc5487	feat(database): support for postgres (#3998) ... * beginning with postgres statements * try pgx * use pgx * database * init works for postgres * arrays working * init for cockroach * init * start tests * tests * TESTS * ch * ch * chore: use go 1.18 * read stmts * fix typo * tests * connection string * add missing error handler * cleanup * start all apis * go mod tidy * old update * switch back to minute * on conflict * replace string slice with `database.StringArray` in db models * fix tests and start * update go version in dockerfile * setup go * clean up * remove notification migration * update * docs: add deploy guide for postgres * fix: revert sonyflake * use `database.StringArray` for daos * use `database.StringArray` every where * new tables * index naming, metadata primary key, project grant role key type * docs(postgres): change to beta * chore: correct compose * fix(defaults): add empty postgres config * refactor: remove unused code * docs: add postgres to self hosted * fix broken link * so? * change title * add mdx to link * fix stmt * update goreleaser in test-code * docs: improve postgres example * update more projections * fix: add beta log for postgres * revert index name change * prerelease * fix: add sequence to v1 "reduce paniced" * log if nil * add logging * fix: log output * fix(import): check if org exists and user * refactor: imports * fix(user): ignore malformed events * refactor: method naming * fix: test * refactor: correct errors.Is call * ci: don't build dev binaries on main * fix(go releaser): update version to 1.11.0 * fix(user): projection should not break * fix(user): handle error properly * docs: correct config example * Update .releaserc.js * Update .releaserc.js Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Elio Bischof <eliobischof@gmail.com>	2022-08-31 07:52:43 +00:00
Livio Spring	aed7010508	fix: scheduling (#3978) ... * fix: improve scheduling * build pre-release * fix: locker * fix: user handler and print stack in case of panic in reducer * chore: remove sentry * fix: improve handler projection and implement tests * more tests * fix: race condition in tests * Update internal/eventstore/repository/sql/query.go Co-authored-by: Silvan <silvan.reusser@gmail.com> * fix: implemented suggested changes * fix: lock statement Co-authored-by: Silvan <silvan.reusser@gmail.com>	2022-07-22 10:08:39 +00:00
Livio Spring	12d4d3ea0b	fix: enable env vars in setup steps (and deprecate admin subcommand) (#3871) ... * fix: enable env vars in setup steps (and deprecate admin subcommand) * fix tests and error text	2022-06-27 10:32:34 +00:00
Silvan	a95b1ab3d0	fix(storage): resolve deadlock occuring durring projection (#3671)	2022-05-19 11:44:16 +00:00
Florian Forster	fa9f581d56	chore(v2): move to new org (#3499) ... * chore: move to new org * logging * fix: org rename caos -> zitadel Co-authored-by: adlerhurst <silvan.reusser@gmail.com>	2022-04-26 23:01:45 +00:00
Livio Amstutz	1305c14e49	feat: handle instanceID in projections (#3442) ... * feat: handle instanceID in projections * rename functions * fix key lock * fix import	2022-04-19 08:26:12 +02:00
Silvan	db554536a1	fix: v2 setup sequence (#3437) ... * add/register human command done * validations * crypto * move clientid * keys * fix: clientID * remove v2 package * tests * tests running * fix: add init instance to eventstore * fix: mig * test(eventstore): create instance * revert old code * instance domain from ctx * chore: rename zitadel app ids * comments * fix: test * fix: mock * fix: test	2022-04-13 05:42:48 +00:00