mirror of
https://github.com/zitadel/zitadel.git
synced 2024-12-14 03:54:21 +00:00
02c98f570b
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Silvan
|
1d84635836
|
feat(eventstore): add search table (#8191)
# Which Problems Are Solved To improve performance a new table and method is implemented on eventstore. The goal of this table is to index searchable fields on command side to use it on command and query side. The table allows to store one primitive value (numeric, text) per row. The eventstore framework is extended by the `Search`-method which allows to search for objects. The `Command`-interface is extended by the `SearchOperations()`-method which does manipulate the the `search`-table. # How the Problems Are Solved This PR adds the capability of improving performance for command and query side by using the `Search`-method of the eventstore instead of using one of the `Filter`-methods. # Open Tasks - [x] Add feature flag - [x] Unit tests - [ ] ~~Benchmarks if needed~~ - [x] Ensure no behavior change - [x] Add setup step to fill table with current data - [x] Add projection which ensures data added between setup and start of the new version are also added to the table # Additional Changes The `Search`-method is currently used by `ProjectGrant`-command side. # Additional Context - Closes https://github.com/zitadel/zitadel/issues/8094 |
||
|
Tim Möhlmann
|
f680dd934d
|
refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors * rename package errors to gerrors * fix error related linting issues * fix zitadel error assertion * fix gosimple linting issues * fix deprecated linting issues * resolve gci linting issues * fix import structure --------- Co-authored-by: Elio Bischof <elio@zitadel.com> |
||
|
Silvan
|
e3d1ca4d58
|
fix(eventstore): improve pagination of handler filter (#6968)
* fix(setup): add filter_offset to `projections.current_states` * fix(eventstore): allow offset in query * fix(handler): offset for already processed events |
||
|
Silvan
|
b5564572bc
|
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore. Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006). The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`. If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events. |
||
|
Silvan
|
c5b99274d7
|
feat(cli): setup (#3267)
* commander * commander * selber! * move to packages * fix(errors): implement Is interface * test: command * test: commands * add init steps * setup tenant * add default step yaml * possibility to set password * merge v2 into v2-commander * fix: rename iam command side to instance * fix: rename iam command side to instance * fix: rename iam command side to instance * fix: rename iam command side to instance * fix: search query builder can filter events in memory * fix: filters for add member * fix(setup): add `ExternalSecure` to config * chore: name iam to instance * fix: matching * remove unsued func * base url * base url * test(command): filter funcs * test: commands * fix: rename orgiampolicy to domain policy * start from init * commands * config * fix indexes and add constraints * fixes * fix: merge conflicts * fix: protos * fix: md files * setup * add deprecated org iam policy again * typo * fix search query * fix filter * Apply suggestions from code review * remove custom org from org setup * add todos for verification * change apps creation * simplify package structure * fix error * move preparation helper for tests * fix unique constraints * fix config mapping in setup * fix error handling in encryption_keys.go * fix projection config * fix query from old views to projection * fix setup of mgmt api * set iam project and fix instance projection * imports Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: fabi <fabienne.gerschwiler@gmail.com> |
||
|
Silvan
|
09be70949f
|
refactor(eventstore): rename EventPusher to Command, EventReader to Event, PushEvents to Push and FilterEvents to Filter (#2907) | ||
|
Silvan
|
37ee5b4bab
|
feat: jobs for projection tables (#1730)
* job queue * wg improvements * start handler * statement * statements * imporve handler * improve statement * statement in seperate file * move handlers * move query/old to query * handler * read models * bulk works * cleanup * contrib * rename readmodel to projection * rename read_models schema to projections * rename read_models schema to projections * search query as func, bulk iterates as long as new events * add event sequence less query * update checks for events between current sequence and sequence of first statement if it has previous sequence 0 * cleanup crdb projection * refactor projection handler * start with testing * tests for handler * remove todo * refactor statement: remove table name, add tests * improve projection handler shutdown, no savepoint if noop stmt, tests for stmt handler * tests * start failed events * seperate branch for contrib * move statement constructors to crdb pkg * correct import * Subscribe for eventtypes (#1800) * fix: is default (#1737) * fix: use email as username on global org (#1738) * fix: use email as username on global org * Update user_human.go * Update register_handler.go * chore(deps): update docusaurus (#1739) * chore: remove PAT and use GH Token (#1716) * chore: remove PAT and use GH Token * fix env * fix env * fix env * md lint * trigger ci * change user * fix GH bug * replace login part * chore: add GH Token to sem rel (#1746) * chore: add GH Token to sem rel * try branch * add GH Token * remove test branch again * docs: changes acme to acme-caos (#1744) * changes acme to acme-caos * Apply suggestions from code review Co-authored-by: Florian Forster <florian@caos.ch> Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com> Co-authored-by: Florian Forster <florian@caos.ch> * feat: add additional origins on applications (#1691) * feat: add additional origins on applications * app additional redirects * chore(deps-dev): bump @angular/cli from 11.2.8 to 11.2.11 in /console (#1706) * fix: show org with regex (#1688) * fix: flag mapping (#1699) * chore(deps-dev): bump @angular/cli from 11.2.8 to 11.2.11 in /console Bumps [@angular/cli](https://github.com/angular/angular-cli) from 11.2.8 to 11.2.11. - [Release notes](https://github.com/angular/angular-cli/releases) - [Commits](https://github.com/angular/angular-cli/compare/v11.2.8...v11.2.11) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Silvan <silvan.reusser@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump stylelint from 13.10.0 to 13.13.1 in /console (#1703) * fix: show org with regex (#1688) * fix: flag mapping (#1699) * chore(deps-dev): bump stylelint from 13.10.0 to 13.13.1 in /console Bumps [stylelint](https://github.com/stylelint/stylelint) from 13.10.0 to 13.13.1. - [Release notes](https://github.com/stylelint/stylelint/releases) - [Changelog](https://github.com/stylelint/stylelint/blob/master/CHANGELOG.md) - [Commits](https://github.com/stylelint/stylelint/compare/13.10.0...13.13.1) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Silvan <silvan.reusser@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @types/node from 14.14.37 to 15.0.1 in /console (#1702) * fix: show org with regex (#1688) * fix: flag mapping (#1699) * chore(deps-dev): bump @types/node from 14.14.37 to 15.0.1 in /console Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 14.14.37 to 15.0.1. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Silvan <silvan.reusser@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump ts-protoc-gen from 0.14.0 to 0.15.0 in /console (#1701) * fix: show org with regex (#1688) * fix: flag mapping (#1699) * chore(deps): bump ts-protoc-gen from 0.14.0 to 0.15.0 in /console Bumps [ts-protoc-gen](https://github.com/improbable-eng/ts-protoc-gen) from 0.14.0 to 0.15.0. - [Release notes](https://github.com/improbable-eng/ts-protoc-gen/releases) - [Changelog](https://github.com/improbable-eng/ts-protoc-gen/blob/master/CHANGELOG.md) - [Commits](https://github.com/improbable-eng/ts-protoc-gen/compare/0.14.0...0.15.0) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Silvan <silvan.reusser@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @types/jasmine from 3.6.9 to 3.6.10 in /console (#1682) Bumps [@types/jasmine](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jasmine) from 3.6.9 to 3.6.10. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jasmine) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump @types/google-protobuf in /console (#1681) Bumps [@types/google-protobuf](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/google-protobuf) from 3.7.4 to 3.15.2. - [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases) - [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/google-protobuf) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps): bump grpc from 1.24.5 to 1.24.7 in /console (#1666) Bumps [grpc](https://github.com/grpc/grpc-node) from 1.24.5 to 1.24.7. - [Release notes](https://github.com/grpc/grpc-node/releases) - [Commits](https://github.com/grpc/grpc-node/compare/grpc@1.24.5...grpc@1.24.7) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * lock * chore(deps-dev): bump @angular/language-service from 11.2.9 to 11.2.12 in /console (#1704) * fix: show org with regex (#1688) * fix: flag mapping (#1699) * chore(deps-dev): bump @angular/language-service in /console Bumps [@angular/language-service](https://github.com/angular/angular/tree/HEAD/packages/language-service) from 11.2.9 to 11.2.12. - [Release notes](https://github.com/angular/angular/releases) - [Changelog](https://github.com/angular/angular/blob/master/CHANGELOG.md) - [Commits](https://github.com/angular/angular/commits/11.2.12/packages/language-service) Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Silvan <silvan.reusser@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * package lock * downgrade grpc * downgrade protobuf types * revert npm packs 🥸 Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Silvan <silvan.reusser@gmail.com> * docs: update run and start section texts (#1745) * update run and start section texts * adds showcase Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com> * fix: additional origin list (#1753) * fix: handle api configs in authz handler (#1755) * fix(console): add model for api keys, fix toast, binding (#1757) * fix: add model for api keys, fix toast, binding * show api clientid * fix: missing patchvalue (#1758) * feat: refresh token (#1728) * begin refresh tokens * refresh tokens * list and revoke refresh tokens * handle remove * tests for refresh tokens * uniqueness and default expiration * rename oidc token methods * cleanup * migration version * Update internal/static/i18n/en.yaml Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com> * fixes * feat: update oidc pkg for refresh tokens Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com> * fix: correct json name of clientId in key.json (#1760) * fix: migration version (#1767) * start subscription * eventtypes * fix(login): links (#1778) * fix(login): href for help * fix(login): correct link to tos * fix: access tokens for service users and refresh token infos (#1779) * fix: access token for service user * handle info from refresh request * uniqueness * postpone access token uniqueness change * chore(coc): recommend code of conduct (#1782) * subscribe for events * feat(console): refresh toggle out of granttype context (#1785) * refresh toggle * disable if not code flow, lint * lint * fix: change oidc config order * accept refresh option within flow Co-authored-by: Livio Amstutz <livio.a@gmail.com> * fix: refresh token activation (#1795) * fix: oidc grant type check * docs: add offline_access scope * docs: update refresh token status in supported grant types * fix: update oidc pkg * fix: check refresh token grant type (#1796) * configuration structs * org admins * failed events * fixes Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Florian Forster <florian@caos.ch> Co-authored-by: mffap <mpa@caos.ch> Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com> * remove comment * aggregate reducer * remove eventtypes * add protoc-get-validate to mod * fix transaltion * upsert * add gender on org admins, allow to retry failed stmts after configurable time * remove if * sub queries * fix: tests * add builder to tests * new search query * rename searchquerybuilder to builder * remove comment from code * test with multiple queries * add filters test * current sequences * make org and org_admins work again * add aggregate type to current sequence * fix(contibute): listing * add validate module * fix: search queries * feat(eventstore): previous aggregate root sequence (#1810) * feat(eventstore): previous aggregate root sequence * fix tests * fix: eventstore v1 test * add col to all mocked rows * next try * fix mig * rename aggregate root to aggregate type * update comment Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Livio Amstutz <livio.a@gmail.com> * small refactorings * allow update multiple current sequences * unique log id * fix migrations * rename org admin to org owner * improve error handling and logging * fix(migration): optimize prev agg root seq * fix: projection handler test * fix: sub queries * small fixes * additional event types * correct org owner projection * fix primary key * feat(eventstore): jobs for projections (#2026) * fix: template names in login (#1974) * fix: template names in login * fix: error.html * fix: check for features on mgmt only (#1976) * fix: add sentry in ui, http and projection handlers (#1977) * fix: add sentry in ui, http and projection handlers * fix test * fix(eventstore): sub queries (#1805) * sub queries * fix: tests * add builder to tests * new search query * rename searchquerybuilder to builder * remove comment from code * test with multiple queries * add filters test * fix(contibute): listing * add validate module * fix: search queries * remove unused event type in query * ignore query if error in marshal * go mod tidy * update privacy policy query * update queries Co-authored-by: Livio Amstutz <livio.a@gmail.com> * feat: Extend oidc idp with oauth endpoints (#1980) * feat: add oauth attributes to oidc idp configuration * feat: return idpconfig id on create idp * feat: tests * feat: descriptions * feat: docs * feat: tests * docs: update to beta 3 (#1984) * fix: role assertion (#1986) * fix: enum to display access token role assertion * improve assertion descriptions * fix nil pointer * docs: eventstore (#1982) * docs: eventstore * Apply suggestions from code review Co-authored-by: Florian Forster <florian@caos.ch> Co-authored-by: Florian Forster <florian@caos.ch> * fix(sentry): trigger sentry release (#1989) * feat(send sentry release): send sentry release * fix(moved step and added releasetag): moved step and added releasetag * fix: set version for sentry release (#1990) * feat(send sentry release): send sentry release * fix(moved step and added releasetag): moved step and added releasetag * fix(corrected var name): corrected var name Co-authored-by: Livio Amstutz <livio.a@gmail.com> * fix: log error reason on terminate session (#1973) * fix: return default language file, if requested lang does not exist for default login texts (#1988) * fix: return default language file, if requested lang doesnt exists * feat: read default translation file * feat: docs * fix: race condition in auth request unmarshalling (#1993) * feat: handle ui_locales in login (#1994) * fix: handle ui_locales in login * move supportedlanguage func into i18n package * update oidc pkg * fix: handle closed channels on unsubscribe (#1995) * fix: give restore more time (#1997) * fix: translation file read (#2009) * feat: translation file read * feat: readme * fix: enable idp add button for iam users (#2010) * fix: filter event_data (#2011) * feat: Custom message files (#1992) * feat: add get custom message text to admin api * feat: read custom message texts from files * feat: get languages in apis * feat: get languages in apis * feat: get languages in apis * feat: pr feedback * feat: docs * feat: merge main * fix: sms notification (#2013) * fix: phone verifications * feat: fix password reset as sms * fix: phone verification * fix: grpc status in sentry and validation interceptors (#2012) * fix: remove oauth endpoints from oidc config proto (#2014) * try with view * fix(console): disable sw (#2021) * fix: disable sw * angular.json disable sw * project projections * fix typos * customize projections * customizable projections, add change date to projects Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com> Co-authored-by: Florian Forster <florian@caos.ch> Co-authored-by: mffap <mpa@caos.ch> Co-authored-by: Christian Jakob <47860090+thesephirot@users.noreply.github.com> Co-authored-by: Elio Bischof <eliobischof@gmail.com> * env file * typo * correct users * correct migration * fix: merge fail * fix test * fix(tests): unordered matcher * improve currentSequenceMatcher * correct certs * correct certs * add zitadel database on database list * refctor switch in match * enable all handlers * Delete io.env * cleanup * add handlers * rename view to projection * rename view to projection * fix type typo * remove unnecessary logs * refactor stmts * simplify interval calculation * fix tests * fix unlock test * fix migration * migs * fix(operator): update cockroach and flyway versions (#2138) * chore(deps): bump k8s.io/apiextensions-apiserver from 0.19.2 to 0.21.3 Bumps [k8s.io/apiextensions-apiserver](https://github.com/kubernetes/apiextensions-apiserver) from 0.19.2 to 0.21.3. - [Release notes](https://github.com/kubernetes/apiextensions-apiserver/releases) - [Commits](https://github.com/kubernetes/apiextensions-apiserver/compare/v0.19.2...v0.21.3) --- updated-dependencies: - dependency-name: k8s.io/apiextensions-apiserver dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * chore(deps): bump google.golang.org/api from 0.34.0 to 0.52.0 Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client) from 0.34.0 to 0.52.0. - [Release notes](https://github.com/googleapis/google-api-go-client/releases) - [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md) - [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.34.0...v0.52.0) --- updated-dependencies: - dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * start update dependencies * update mods and otlp * fix(build): update to go 1.16 * old version for k8s mods * update k8s versions * update orbos * fix(operator): update cockroach and flyway version * Update images.go Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Stefan Benz <stefan@caos.ch> * fix import * fix typo * fix(migration): add org projection * fix(projection): correct table for org events in org owners * better insert stmt * fix typo * fix typo * set max connection lifetime * set max conns and conn lifetime in eventstore v1 * configure sql connection settings * add mig for agg type index * fix replace tab in yaml * check requeue at least 500ms * split column in column and condition * remove useless comment * mig versions * fix migs Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Florian Forster <florian@caos.ch> Co-authored-by: mffap <mpa@caos.ch> Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com> Co-authored-by: Christian Jakob <47860090+thesephirot@users.noreply.github.com> Co-authored-by: Elio Bischof <eliobischof@gmail.com> Co-authored-by: Stefan Benz <stefan@caos.ch> |
||
|
Fabi
|
667cc30291
|
feat: asset storage (#1696)
* feat: remove assets * feat: minio implementation * fix: remove assets from tests * feat: minio implementation * feat: Env vars * fix: sprintf * fix: sprintf * Update internal/eventstore/repository/repository.go Co-authored-by: Livio Amstutz <livio.a@gmail.com> * fix: error handling Co-authored-by: Livio Amstutz <livio.a@gmail.com> |
||
|
Fabi
|
f51f0ede5c
|
feat: add assets to eventstore and event (#1674)
* fix: add assets to eventstore and event * fix: project member, grant member, app changed tests * fix: asset migrations * feat: add asset tests * feat: add asset tests * Update internal/eventstore/repository/repository.go Co-authored-by: Livio Amstutz <livio.a@gmail.com> * feat: add asset tests Co-authored-by: Livio Amstutz <livio.a@gmail.com> |
||
|
Fabi
|
c0f55e7209
|
feat: pub sub (#1341)
* fix: pub sub * fix: adaot config to commands (and queries) * remove dependency on vv2 in v1 * fix: pub sub in new eventstore * fix tests Co-authored-by: Livio Amstutz <livio.a@gmail.com> |
||
|
Fabi
|
d8e42744b4
|
fix: move v2 pkgs (#1331)
* fix: move eventstore pkgs * fix: move eventstore pkgs * fix: remove v2 view * fix: remove v2 view |