Silvan
77cd430b3a
refactor(handler): cache active instances ( #9008 )
...
# Which Problems Are Solved
Scheduled handlers use `eventstore.InstanceIDs` to get the all active
instances within a given timeframe. This function scrapes through all
events written within that time frame which can cause heavy load on the
database.
# How the Problems Are Solved
A new query cache `activeInstances` is introduced which caches the ids
of all instances queried by id or host within the configured timeframe.
# Additional Changes
- Changed `default.yaml`
- Removed `HandleActiveInstances` from custom handler configs
- Added `MaxActiveInstances` to define the maximal amount of cached
instance ids
- fixed start-from-init and start-from-setup to start auth and admin
projections twice
- fixed org cache invalidation to use correct index
# Additional Context
- part of #8999
2024-12-06 11:32:53 +00:00
Silvan
dab5d9e756
refactor(eventstore): move push logic to sql ( #8816 )
...
# Which Problems Are Solved
If many events are written to the same aggregate id it can happen that
zitadel [starts to retry the push
transaction](48ffc902cc/internal/eventstore/eventstore.go (L101)48ffc902cc/internal/eventstore/v3/sequence.go (L25)https://github.com/zitadel/zitadel/issues/8931
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Miguel Cabrerizo <30386061+doncicuto@users.noreply.github.com>
Co-authored-by: Joakim Lodén <Loddan@users.noreply.github.com>
Co-authored-by: Yxnt <Yxnt@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Zach H <zhirschtritt@gmail.com>
2024-12-04 13:51:40 +00:00
Silvan
7714af6f5b
fix(eventstore): correct database type in PushWithClient ( #8949 )
...
# Which Problems Are Solved
`eventstore.PushWithClient` required the wrong type of for the client
parameter.
# How the Problems Are Solved
Changed type of client from `database.Client` to
`database.QueryExecutor`
2024-11-25 07:02:59 +01:00
Silvan
1ee7a1ab7c
feat(eventstore): accept transaction in push ( #8945 )
...
# Which Problems Are Solved
Push is not capable of external transactions.
# How the Problems Are Solved
A new function `PushWithClient` is added to the eventstore framework
which allows to pass a client which can either be a `*sql.Client` or
`*sql.Tx` and is used during push.
# Additional Changes
Added interfaces to database package.
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/8931
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2024-11-22 17:25:28 +01:00
Tim Möhlmann
32bad3feb3
perf(milestones): refactor ( #8788 )
...
ZITADEL CI/CD / core (push) Waiting to run
ZITADEL CI/CD / console (push) Waiting to run
ZITADEL CI/CD / version (push) Waiting to run
ZITADEL CI/CD / compile (push) Blocked by required conditions
ZITADEL CI/CD / core-unit-test (push) Blocked by required conditions
ZITADEL CI/CD / core-integration-test (push) Blocked by required conditions
ZITADEL CI/CD / lint (push) Blocked by required conditions
ZITADEL CI/CD / container (push) Blocked by required conditions
ZITADEL CI/CD / e2e (push) Blocked by required conditions
ZITADEL CI/CD / release (push) Blocked by required conditions
Code Scanning / CodeQL-Build (go) (push) Waiting to run
Code Scanning / CodeQL-Build (javascript) (push) Waiting to run
# Which Problems Are Solved
Milestones used existing events from a number of aggregates. OIDC
session is one of them. We noticed in load-tests that the reduction of
the oidc_session.added event into the milestone projection is a costly
business with payload based conditionals. A milestone is reached once,
but even then we remain subscribed to the OIDC events. This requires the
projections.current_states to be updated continuously.
# How the Problems Are Solved
The milestone creation is refactored to use dedicated events instead.
The command side decides when a milestone is reached and creates the
reached event once for each milestone when required.
# Additional Changes
In order to prevent reached milestones being created twice, a migration
script is provided. When the old `projections.milestones` table exist,
the state is read from there and `v2` milestone aggregate events are
created, with the original reached and pushed dates.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8800
2024-10-28 08:29:34 +00:00
Tim Möhlmann
aeb379e7de
fix(eventstore): revert precise decimal ( #8527 ) ( #8679 )
2024-09-24 18:43:29 +02:00
Silvan
b522588d98
fix(eventstore): precise decimal ( #8527 )
...
# Which Problems Are Solved
Float64 which was used for the event.Position field is [not precise in
go and gets rounded](https://github.com/golang/go/issues/47300 ). This
can lead to unprecies position tracking of events and therefore
projections especially on cockcoachdb as the position used there is a
big number.
example of a unprecies position:
exact: 1725257931223002628
float64: 1725257931223002624.000000
# How the Problems Are Solved
The float64 was replaced by
[github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal ).
# Additional Changes
Correct behaviour of makefile for load tests.
Rename `latestSequence`-queries to `latestPosition`
2024-09-06 12:19:19 +03:00
Silvan
1d84635836
feat(eventstore): add search table ( #8191 )
...
# Which Problems Are Solved
To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.
The table allows to store one primitive value (numeric, text) per row.
The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.
# How the Problems Are Solved
This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.
# Open Tasks
- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table
# Additional Changes
The `Search`-method is currently used by `ProjectGrant`-command side.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8094
2024-07-03 15:00:56 +00:00
Silvan
56df515e5f
chore: use pgx v5 ( #7577 )
...
* chore: use pgx v5
* chore: update go version
* remove direct pq dependency
* remove unnecessary type
* scan test
* map scanner
* converter
* uint8 number array
* duration
* most unit tests work
* unit tests work
* chore: coverage
* go 1.21
* linting
* int64 gopfertammi
* retry go 1.22
* retry go 1.22
* revert to go v1.21.5
* update go toolchain to 1.21.8
* go 1.21.8
* remove test flag
* go 1.21.5
* linting
* update toolchain
* use correct array
* use correct array
* add byte array
* correct value
* correct error message
* go 1.21 compatible
2024-03-27 15:48:22 +02:00
Silvan
ec3076c3c8
fix(ListEvents): add aggregate types to filter if not set ( #7490 )
2024-03-05 15:44:51 +00:00
Tim Möhlmann
1890e28f79
fix(eventstore): retry push on primary key sequence collision ( #7420 )
...
* fix(eventstore): retry push on primary key sequence collision
* MaxRetries config option and unit test
2024-02-23 08:29:10 +00:00
Silvan
17953e9040
fix(setup): init projections ( #7194 )
...
Even though this is a feature it's released as fix so that we can back port to earlier revisions.
As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.
The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
2024-01-25 17:28:20 +01:00
Tim Möhlmann
ab79855cf0
fix(eventstore): prevent allocation of filtered events ( #6749 )
...
* fix(eventstore): prevent allocation of filtered events
Directly reduce each event obtained from a sql.Rows scan,
so that we do not have to allocate all events in a slice.
* reinstate the mutex as RWMutex
* scan data directly
* add todos
* fix(writemodels): add reduce of parent
* test: remove comment
* update comments
---------
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-10-19 15:21:31 +00:00
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities ( #5940 )
...
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005 ) and [06](https://zitadel.com/docs/support/advisory/a10006 ).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Silvan
f72560291e
fix(eventstore): cache instances ( #6501 )
...
* fix(eventstore): cache instances
* fix: consider succeeded once during instance ids query
* fix(eventstore): return correct instances
(cherry picked from commit 0f06e84f40
2023-09-07 06:23:28 +02:00
Livio Spring
a1942ecdaa
feat(api): add and remove OTP (SMS and email) ( #6295 )
...
* refactor: rename otp to totp
* feat: add otp sms and email
* implement tests
2023-08-02 18:57:53 +02:00
Tim Möhlmann
5819924275
feat: device authorization RFC 8628 ( #5646 )
...
* device auth: implement the write events
* add grant type device code
* fix(init): check if default value implements stringer
---------
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-04-19 08:46:02 +00:00
Silvan
7b5135e637
fix(adminAPI): localize event type ( #5059 )
...
* fix(adminAPI): localisation of event types, aggregate types
* fix(adminAPI): validations of ListEvent request
* implement caching of editor user information
2023-01-19 15:50:05 +00:00
Silvan
1bf1f335dc
feat(admin-api): list events ( #4989 )
...
* docs: update cockroachdb version to 22.2
* feat(adminAPI): ListEventTypes returns the list of event types ZITADEL implements
* feat(adminAPI): ListAggregateTypes returns the list of aggregate types ZITADEL implements
* feat(adminAPI): ListEvents allows `IAM_OWNERS` to search for events
2023-01-16 11:30:03 +00:00
Livio Spring
d21bb902f1
fix: push timeout ( #4882 ) ( #4885 )
...
* push with timeout
* test: config for eventstore
(cherry picked from commit b9156da76d
2022-12-15 09:40:13 +00:00
Livio Spring
d721f725fd
fix: instance remove ( #4602 )
2022-10-26 13:06:48 +00:00
Livio Spring
aed7010508
fix: scheduling ( #3978 )
...
* fix: improve scheduling
* build pre-release
* fix: locker
* fix: user handler and print stack in case of panic in reducer
* chore: remove sentry
* fix: improve handler projection and implement tests
* more tests
* fix: race condition in tests
* Update internal/eventstore/repository/sql/query.go
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* fix: implemented suggested changes
* fix: lock statement
Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-07-22 10:08:39 +00:00
Livio Amstutz
d401439427
fix: unique constraints on instance domain events ( #3635 )
2022-05-16 11:52:54 +02:00
Florian Forster
fa9f581d56
chore(v2): move to new org ( #3499 )
...
* chore: move to new org
* logging
* fix: org rename caos -> zitadel
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2022-04-26 23:01:45 +00:00
Silvan
db554536a1
fix: v2 setup sequence ( #3437 )
...
* add/register human command done
* validations
* crypto
* move clientid
* keys
* fix: clientID
* remove v2 package
* tests
* tests running
* fix: add init instance to eventstore
* fix: mig
* test(eventstore): create instance
* revert old code
* instance domain from ctx
* chore: rename zitadel app ids
* comments
* fix: test
* fix: mock
* fix: test
2022-04-13 05:42:48 +00:00
Silvan
cea2567e22
fix: v2 human command ( #3435 )
...
* add/register human command done
* validations
* crypto
* move clientid
* keys
* fix: clientID
* remove v2 package
* tests
* tests running
* revert old code
* instance domain from ctx
* chore: rename zitadel app ids
* comments
* fix: test
2022-04-12 16:20:17 +02:00
Livio Amstutz
958362e6c9
feat: handle instance from context ( #3382 )
...
* commander
* commander
* selber!
* move to packages
* fix(errors): implement Is interface
* test: command
* test: commands
* add init steps
* setup tenant
* add default step yaml
* possibility to set password
* merge v2 into v2-commander
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: search query builder can filter events in memory
* fix: filters for add member
* fix(setup): add `ExternalSecure` to config
* chore: name iam to instance
* fix: matching
* remove unsued func
* base url
* base url
* test(command): filter funcs
* test: commands
* fix: rename orgiampolicy to domain policy
* start from init
* commands
* config
* fix indexes and add constraints
* fixes
* fix: merge conflicts
* fix: protos
* fix: md files
* setup
* add deprecated org iam policy again
* typo
* fix search query
* fix filter
* Apply suggestions from code review
* remove custom org from org setup
* add todos for verification
* change apps creation
* simplify package structure
* fix error
* move preparation helper for tests
* fix unique constraints
* fix config mapping in setup
* fix error handling in encryption_keys.go
* fix projection config
* fix query from old views to projection
* fix setup of mgmt api
* set iam project and fix instance projection
* fix tokens view
* fix steps.yaml and defaults.yaml
* fix projections
* change instance context to interface
* instance interceptors and additional events in setup
* cleanup
* tests for interceptors
* fix label policy
* add todo
* single api endpoint in environment.json
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-03-29 11:53:19 +02:00
Silvan
c5b99274d7
feat(cli): setup ( #3267 )
...
* commander
* commander
* selber!
* move to packages
* fix(errors): implement Is interface
* test: command
* test: commands
* add init steps
* setup tenant
* add default step yaml
* possibility to set password
* merge v2 into v2-commander
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: search query builder can filter events in memory
* fix: filters for add member
* fix(setup): add `ExternalSecure` to config
* chore: name iam to instance
* fix: matching
* remove unsued func
* base url
* base url
* test(command): filter funcs
* test: commands
* fix: rename orgiampolicy to domain policy
* start from init
* commands
* config
* fix indexes and add constraints
* fixes
* fix: merge conflicts
* fix: protos
* fix: md files
* setup
* add deprecated org iam policy again
* typo
* fix search query
* fix filter
* Apply suggestions from code review
* remove custom org from org setup
* add todos for verification
* change apps creation
* simplify package structure
* fix error
* move preparation helper for tests
* fix unique constraints
* fix config mapping in setup
* fix error handling in encryption_keys.go
* fix projection config
* fix query from old views to projection
* fix setup of mgmt api
* set iam project and fix instance projection
* imports
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-03-28 10:05:09 +02:00
Livio Amstutz
56b916a2b0
feat: projections auto create their tables ( #3324 )
...
* begin init checks for projections
* first projection checks
* debug notification providers with query fixes
* more projections and first index
* more projections
* more projections
* finish projections
* fix tests (remove db name)
* create tables in setup
* fix logging / error handling
* add tenant to views
* rename tenant to instance_id
* add instance_id to all projections
* add instance_id to all queries
* correct instance_id on projections
* add instance_id to failed_events
* use separate context for instance
* implement features projection
* implement features projection
* remove unique constraint from setup when migration failed
* add error to failed setup event
* add instance_id to primary keys
* fix IAM projection
* remove old migrations folder
* fix keysFromYAML test
2022-03-23 09:02:39 +01:00
Fabi
5132ebe07c
feat: add tenant column to eventstore ( #3314 )
...
* feat: add tenant column to eventstore
* feat: read tenant from context on push and filter
* Update 07_events_table.sql
* pass tenant to queryFactory
* fix some query tests
* init in tests
* add missing sql files
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2022-03-15 07:19:02 +01:00
Livio Amstutz
19b095e6c2
fix: check resourceowner not empty string ( #2922 )
...
* fix: check resourceowner not empty string
* fix test
* fix tests
* fix tests of command pkg
* enable RoleSelfManagementGlobal as org member role
* fix tests of query pkg
* Update eventstore_test.go
* update docusaurus
2022-01-06 08:29:58 +01:00
Silvan
09be70949f
refactor(eventstore): rename EventPusher to Command, EventReader to Event, PushEvents to Push and FilterEvents to Filter ( #2907 )
2022-01-03 09:19:07 +01:00
Livio Amstutz
27f87df676
fix: setup step 21 ( #2864 )
2021-12-17 10:11:57 +00:00
Silvan
296f1c3c71
fix(eventstore): fill new column with data ( #2288 )
...
* fix: smaller outage on events migration first part
* fix: fill old events with sequence
* fix: migration add transactions
* fix: mig
* rename mig
* replace migration with setup step
* regenerate mock
* add step 20 to config
* log
* simplify step
* limit 1000
* limit 500
2021-09-01 09:25:52 +00:00
Silvan
37ee5b4bab
feat: jobs for projection tables ( #1730 )
...
* job queue
* wg improvements
* start handler
* statement
* statements
* imporve handler
* improve statement
* statement in seperate file
* move handlers
* move query/old to query
* handler
* read models
* bulk works
* cleanup
* contrib
* rename readmodel to projection
* rename read_models schema to projections
* rename read_models schema to projections
* search query as func,
bulk iterates as long as new events
* add event sequence less query
* update checks for events between current sequence and sequence of first statement if it has previous sequence 0
* cleanup crdb projection
* refactor projection handler
* start with testing
* tests for handler
* remove todo
* refactor statement: remove table name,
add tests
* improve projection handler shutdown,
no savepoint if noop stmt,
tests for stmt handler
* tests
* start failed events
* seperate branch for contrib
* move statement constructors to crdb pkg
* correct import
* Subscribe for eventtypes (#1800 )
* fix: is default (#1737 )
* fix: use email as username on global org (#1738 )
* fix: use email as username on global org
* Update user_human.go
* Update register_handler.go
* chore(deps): update docusaurus (#1739 )
* chore: remove PAT and use GH Token (#1716 )
* chore: remove PAT and use GH Token
* fix env
* fix env
* fix env
* md lint
* trigger ci
* change user
* fix GH bug
* replace login part
* chore: add GH Token to sem rel (#1746 )
* chore: add GH Token to sem rel
* try branch
* add GH Token
* remove test branch again
* docs: changes acme to acme-caos (#1744 )
* changes acme to acme-caos
* Apply suggestions from code review
Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com>
Co-authored-by: Florian Forster <florian@caos.ch>
* feat: add additional origins on applications (#1691 )
* feat: add additional origins on applications
* app additional redirects
* chore(deps-dev): bump @angular/cli from 11.2.8 to 11.2.11 in /console (#1706 )
* fix: show org with regex (#1688 )
* fix: flag mapping (#1699 )
* chore(deps-dev): bump @angular/cli from 11.2.8 to 11.2.11 in /console
Bumps [@angular/cli](https://github.com/angular/angular-cli ) from 11.2.8 to 11.2.11.
- [Release notes](https://github.com/angular/angular-cli/releases )
- [Commits](https://github.com/angular/angular-cli/compare/v11.2.8...v11.2.11 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps-dev): bump stylelint from 13.10.0 to 13.13.1 in /console (#1703 )
* fix: show org with regex (#1688 )
* fix: flag mapping (#1699 )
* chore(deps-dev): bump stylelint from 13.10.0 to 13.13.1 in /console
Bumps [stylelint](https://github.com/stylelint/stylelint ) from 13.10.0 to 13.13.1.
- [Release notes](https://github.com/stylelint/stylelint/releases )
- [Changelog](https://github.com/stylelint/stylelint/blob/master/CHANGELOG.md )
- [Commits](https://github.com/stylelint/stylelint/compare/13.10.0...13.13.1 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps-dev): bump @types/node from 14.14.37 to 15.0.1 in /console (#1702 )
* fix: show org with regex (#1688 )
* fix: flag mapping (#1699 )
* chore(deps-dev): bump @types/node from 14.14.37 to 15.0.1 in /console
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node ) from 14.14.37 to 15.0.1.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases )
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump ts-protoc-gen from 0.14.0 to 0.15.0 in /console (#1701 )
* fix: show org with regex (#1688 )
* fix: flag mapping (#1699 )
* chore(deps): bump ts-protoc-gen from 0.14.0 to 0.15.0 in /console
Bumps [ts-protoc-gen](https://github.com/improbable-eng/ts-protoc-gen ) from 0.14.0 to 0.15.0.
- [Release notes](https://github.com/improbable-eng/ts-protoc-gen/releases )
- [Changelog](https://github.com/improbable-eng/ts-protoc-gen/blob/master/CHANGELOG.md )
- [Commits](https://github.com/improbable-eng/ts-protoc-gen/compare/0.14.0...0.15.0 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps-dev): bump @types/jasmine from 3.6.9 to 3.6.10 in /console (#1682 )
Bumps [@types/jasmine](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/jasmine ) from 3.6.9 to 3.6.10.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases )
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/jasmine )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump @types/google-protobuf in /console (#1681 )
Bumps [@types/google-protobuf](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/google-protobuf ) from 3.7.4 to 3.15.2.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases )
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/google-protobuf )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps): bump grpc from 1.24.5 to 1.24.7 in /console (#1666 )
Bumps [grpc](https://github.com/grpc/grpc-node ) from 1.24.5 to 1.24.7.
- [Release notes](https://github.com/grpc/grpc-node/releases )
- [Commits](https://github.com/grpc/grpc-node/compare/grpc@1.24.5...grpc@1.24.7 )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* lock
* chore(deps-dev): bump @angular/language-service from 11.2.9 to 11.2.12 in /console (#1704 )
* fix: show org with regex (#1688 )
* fix: flag mapping (#1699 )
* chore(deps-dev): bump @angular/language-service in /console
Bumps [@angular/language-service](https://github.com/angular/angular/tree/HEAD/packages/language-service ) from 11.2.9 to 11.2.12.
- [Release notes](https://github.com/angular/angular/releases )
- [Changelog](https://github.com/angular/angular/blob/master/CHANGELOG.md )
- [Commits](https://github.com/angular/angular/commits/11.2.12/packages/language-service )
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* package lock
* downgrade grpc
* downgrade protobuf types
* revert npm packs 🥸
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* docs: update run and start section texts (#1745 )
* update run and start section texts
* adds showcase
Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com>
* fix: additional origin list (#1753 )
* fix: handle api configs in authz handler (#1755 )
* fix(console): add model for api keys, fix toast, binding (#1757 )
* fix: add model for api keys, fix toast, binding
* show api clientid
* fix: missing patchvalue (#1758 )
* feat: refresh token (#1728 )
* begin refresh tokens
* refresh tokens
* list and revoke refresh tokens
* handle remove
* tests for refresh tokens
* uniqueness and default expiration
* rename oidc token methods
* cleanup
* migration version
* Update internal/static/i18n/en.yaml
Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
* fixes
* feat: update oidc pkg for refresh tokens
Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
* fix: correct json name of clientId in key.json (#1760 )
* fix: migration version (#1767 )
* start subscription
* eventtypes
* fix(login): links (#1778 )
* fix(login): href for help
* fix(login): correct link to tos
* fix: access tokens for service users and refresh token infos (#1779 )
* fix: access token for service user
* handle info from refresh request
* uniqueness
* postpone access token uniqueness change
* chore(coc): recommend code of conduct (#1782 )
* subscribe for events
* feat(console): refresh toggle out of granttype context (#1785 )
* refresh toggle
* disable if not code flow, lint
* lint
* fix: change oidc config order
* accept refresh option within flow
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* fix: refresh token activation (#1795 )
* fix: oidc grant type check
* docs: add offline_access scope
* docs: update refresh token status in supported grant types
* fix: update oidc pkg
* fix: check refresh token grant type (#1796 )
* configuration structs
* org admins
* failed events
* fixes
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: mffap <mpa@caos.ch>
Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
* remove comment
* aggregate reducer
* remove eventtypes
* add protoc-get-validate to mod
* fix transaltion
* upsert
* add gender on org admins,
allow to retry failed stmts after configurable time
* remove if
* sub queries
* fix: tests
* add builder to tests
* new search query
* rename searchquerybuilder to builder
* remove comment from code
* test with multiple queries
* add filters test
* current sequences
* make org and org_admins work again
* add aggregate type to current sequence
* fix(contibute): listing
* add validate module
* fix: search queries
* feat(eventstore): previous aggregate root sequence (#1810 )
* feat(eventstore): previous aggregate root sequence
* fix tests
* fix: eventstore v1 test
* add col to all mocked rows
* next try
* fix mig
* rename aggregate root to aggregate type
* update comment
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* small refactorings
* allow update multiple current sequences
* unique log id
* fix migrations
* rename org admin to org owner
* improve error handling and logging
* fix(migration): optimize prev agg root seq
* fix: projection handler test
* fix: sub queries
* small fixes
* additional event types
* correct org owner projection
* fix primary key
* feat(eventstore): jobs for projections (#2026 )
* fix: template names in login (#1974 )
* fix: template names in login
* fix: error.html
* fix: check for features on mgmt only (#1976 )
* fix: add sentry in ui, http and projection handlers (#1977 )
* fix: add sentry in ui, http and projection handlers
* fix test
* fix(eventstore): sub queries (#1805 )
* sub queries
* fix: tests
* add builder to tests
* new search query
* rename searchquerybuilder to builder
* remove comment from code
* test with multiple queries
* add filters test
* fix(contibute): listing
* add validate module
* fix: search queries
* remove unused event type in query
* ignore query if error in marshal
* go mod tidy
* update privacy policy query
* update queries
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* feat: Extend oidc idp with oauth endpoints (#1980 )
* feat: add oauth attributes to oidc idp configuration
* feat: return idpconfig id on create idp
* feat: tests
* feat: descriptions
* feat: docs
* feat: tests
* docs: update to beta 3 (#1984 )
* fix: role assertion (#1986 )
* fix: enum to display access token role assertion
* improve assertion descriptions
* fix nil pointer
* docs: eventstore (#1982 )
* docs: eventstore
* Apply suggestions from code review
Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: Florian Forster <florian@caos.ch>
* fix(sentry): trigger sentry release (#1989 )
* feat(send sentry release): send sentry release
* fix(moved step and added releasetag): moved step and added releasetag
* fix: set version for sentry release (#1990 )
* feat(send sentry release): send sentry release
* fix(moved step and added releasetag): moved step and added releasetag
* fix(corrected var name): corrected var name
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* fix: log error reason on terminate session (#1973 )
* fix: return default language file, if requested lang does not exist for default login texts (#1988 )
* fix: return default language file, if requested lang doesnt exists
* feat: read default translation file
* feat: docs
* fix: race condition in auth request unmarshalling (#1993 )
* feat: handle ui_locales in login (#1994 )
* fix: handle ui_locales in login
* move supportedlanguage func into i18n package
* update oidc pkg
* fix: handle closed channels on unsubscribe (#1995 )
* fix: give restore more time (#1997 )
* fix: translation file read (#2009 )
* feat: translation file read
* feat: readme
* fix: enable idp add button for iam users (#2010 )
* fix: filter event_data (#2011 )
* feat: Custom message files (#1992 )
* feat: add get custom message text to admin api
* feat: read custom message texts from files
* feat: get languages in apis
* feat: get languages in apis
* feat: get languages in apis
* feat: pr feedback
* feat: docs
* feat: merge main
* fix: sms notification (#2013 )
* fix: phone verifications
* feat: fix password reset as sms
* fix: phone verification
* fix: grpc status in sentry and validation interceptors (#2012 )
* fix: remove oauth endpoints from oidc config proto (#2014 )
* try with view
* fix(console): disable sw (#2021 )
* fix: disable sw
* angular.json disable sw
* project projections
* fix typos
* customize projections
* customizable projections,
add change date to projects
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: mffap <mpa@caos.ch>
Co-authored-by: Christian Jakob <47860090+thesephirot@users.noreply.github.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
* env file
* typo
* correct users
* correct migration
* fix: merge fail
* fix test
* fix(tests): unordered matcher
* improve currentSequenceMatcher
* correct certs
* correct certs
* add zitadel database on database list
* refctor switch in match
* enable all handlers
* Delete io.env
* cleanup
* add handlers
* rename view to projection
* rename view to projection
* fix type typo
* remove unnecessary logs
* refactor stmts
* simplify interval calculation
* fix tests
* fix unlock test
* fix migration
* migs
* fix(operator): update cockroach and flyway versions (#2138 )
* chore(deps): bump k8s.io/apiextensions-apiserver from 0.19.2 to 0.21.3
Bumps [k8s.io/apiextensions-apiserver](https://github.com/kubernetes/apiextensions-apiserver ) from 0.19.2 to 0.21.3.
- [Release notes](https://github.com/kubernetes/apiextensions-apiserver/releases )
- [Commits](https://github.com/kubernetes/apiextensions-apiserver/compare/v0.19.2...v0.21.3 )
---
updated-dependencies:
- dependency-name: k8s.io/apiextensions-apiserver
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* chore(deps): bump google.golang.org/api from 0.34.0 to 0.52.0
Bumps [google.golang.org/api](https://github.com/googleapis/google-api-go-client ) from 0.34.0 to 0.52.0.
- [Release notes](https://github.com/googleapis/google-api-go-client/releases )
- [Changelog](https://github.com/googleapis/google-api-go-client/blob/master/CHANGES.md )
- [Commits](https://github.com/googleapis/google-api-go-client/compare/v0.34.0...v0.52.0 )
---
updated-dependencies:
- dependency-name: google.golang.org/api
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* start update dependencies
* update mods and otlp
* fix(build): update to go 1.16
* old version for k8s mods
* update k8s versions
* update orbos
* fix(operator): update cockroach and flyway version
* Update images.go
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
* fix import
* fix typo
* fix(migration): add org projection
* fix(projection): correct table for org events in org owners
* better insert stmt
* fix typo
* fix typo
* set max connection lifetime
* set max conns and conn lifetime in eventstore v1
* configure sql connection settings
* add mig for agg type index
* fix replace tab in yaml
* check requeue at least 500ms
* split column in column and condition
* remove useless comment
* mig versions
* fix migs
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: mffap <mpa@caos.ch>
Co-authored-by: Maximilian Panne <maximilian.panne@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
Co-authored-by: Christian Jakob <47860090+thesephirot@users.noreply.github.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
2021-08-19 08:31:56 +02:00
Silvan
5349d96ce4
fix(eventstore): sub queries ( #1805 )
...
* sub queries
* fix: tests
* add builder to tests
* new search query
* rename searchquerybuilder to builder
* remove comment from code
* test with multiple queries
* add filters test
* fix(contibute): listing
* add validate module
* fix: search queries
* remove unused event type in query
* ignore query if error in marshal
* go mod tidy
* update privacy policy query
* update queries
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2021-07-06 13:55:57 +02:00
Fabi
667cc30291
feat: asset storage ( #1696 )
...
* feat: remove assets
* feat: minio implementation
* fix: remove assets from tests
* feat: minio implementation
* feat: Env vars
* fix: sprintf
* fix: sprintf
* Update internal/eventstore/repository/repository.go
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* fix: error handling
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2021-05-03 10:15:50 +02:00
Fabi
f51f0ede5c
feat: add assets to eventstore and event ( #1674 )
...
* fix: add assets to eventstore and event
* fix: project member, grant member, app changed tests
* fix: asset migrations
* feat: add asset tests
* feat: add asset tests
* Update internal/eventstore/repository/repository.go
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* feat: add asset tests
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2021-04-27 12:58:18 +02:00
Fabi
2bd255106a
fix: new es testing ( #1411 )
...
* fix: org tests
* fix: org tests
* fix: user grant test
* fix: user grant test
* fix: project and project role test
* fix: project grant test
* fix: project grant test
* fix: project member, grant member, app changed tests
* fix: application tests
* fix: application tests
* fix: add oidc app test
* fix: add oidc app test
* fix: add api keys test
* fix: iam policies
* fix: iam and org member tests
* fix: clock skew validation
* revert crypto changes
* fix: tests
* fix project grant member commands
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2021-03-15 12:51:15 +01:00
Fabi
c0f55e7209
feat: pub sub ( #1341 )
...
* fix: pub sub
* fix: adaot config to commands (and queries)
* remove dependency on vv2 in v1
* fix: pub sub in new eventstore
* fix tests
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2021-02-24 13:24:33 +01:00
Fabi
d8e42744b4
fix: move v2 pkgs ( #1331 )
...
* fix: move eventstore pkgs
* fix: move eventstore pkgs
* fix: remove v2 view
* fix: remove v2 view
2021-02-23 15:13:04 +01:00
Livio Amstutz
5b84c9b619
Merge branch 'master' into new-eventstore
...
# Conflicts:
# go.mod
# internal/admin/repository/eventsourcing/eventstore/iam.go
# internal/authz/repository/eventsourcing/repository.go
# internal/eventstore/eventstore.go
# internal/setup/config.go
# pkg/grpc/management/mock/management.proto.mock.go
2021-01-05 09:27:42 +01:00
Silvan
dd5e4acd24
fix(event handling): use internal pubsub for view update ( #1118 )
...
* start sub
* start implement subsciptions
* start subscription
* implementation for member done
* admin done
* fix: tests
* extend handlers
* prepary notification
* no errors in adminapi
* changed current sequence in all packages
* ignore mocks
* works
* subscriptions as singleton
* tests
* refactor: rename function scope var
2020-12-18 16:47:45 +01:00
adlerhurst
23e6cad703
start implementing new eventstore on iam
2020-11-12 22:50:01 +01:00
Silvan
41e1a7cc7b
fix(spooler): correct workers ( #508 )
...
* one concurrent task
* disable spooler
* fix: improve concurrency in spooler
* fix: dont block lock
* fix: break if lock failed
* fix: check if handler is working
* fix: worker id
* fix: test
* fix: use limit for spoolers configured in startup.yaml
* fix test
* fix: factory
* fix(key): only reduce if not expired
* fix(searchQueryFactory): check for string-slice in aggregateID
* fix(migrations): combine migrations
* fix: allow saving multiple objects in one request
* fix(eventstore): logging
* fix(eventstore): rethink insert i locks table
* fix: ignore failed tests for the moment
* fix: tuubel
* fix: for tests in io
* fix: ignore tests for io
* fix: rename concurrent tasks to workers
* fix: incomment tests and remove some tests
* fix: refert changes for io
* refactor(eventstore): combine types of sql in one file
* refactor(eventstore): logs, TODO's, tests
* fix(eventstore): sql package
* test(eventstore): add tests for search query factory
* chore: logs
* fix(spooler): optimize lock query
chore(migrations): rename locks.object_type to view_name
chore(migrations): refactor migrations
* test: incomment tests
* fix: rename PrepareSaves to PrepareBulkSave
* chore: go dependencies
* fix(migrations): add id in events table
* refactor(lock): less magic numbers
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2020-07-28 09:42:21 +02:00
Fabi
d447f68d78
fix: User checks ( #139 )
...
* check uniqueness on create and register user
* change user email, reserve release unique email
* usergrant unique aggregate
* usergrant uniqueness
* validate UserGrant
* fix tests
2020-05-28 06:49:22 +02:00
Silvan
191690d905
feat(eventstore): sdk ( #39 )
...
* sdk
* fix(sdk): return correct error type
* AppendEventError instead of Aggregater error
* fix(tests): tests
* fix(tests): wantErr to is error func
2020-04-07 18:36:37 +02:00
Fabi
c07ed83c41
Project commands ( #26 )
...
* feat: eventstore repository
* fix: remove gorm
* version
* feat: pkg
* feat: add some files for project
* feat: eventstore without eventstore-lib
* rename files
* gnueg
* fix: key json
* fix: add object
* fix: change imports
* fix: internal models
* fix: some imports
* fix: global model
* fix: add some functions on repo
* feat(eventstore): sdk
* fix(eventstore): search query
* fix(eventstore): rename app to eventstore
* delete empty test
* remove unused func
* merge master
* fix(eventstore): tests
* fix(models): delete unused struct
* fix: some funcitons
* feat(eventstore): implemented push events
* fix: move project eventstore to project package
* fix: change project eventstore funcs
* feat(eventstore): overwrite context data
* fix: change project eventstore
* fix: add project repo to mgmt server
* feat(types): SQL-config
* fix: commented code
* feat(eventstore): options to overwrite editor
* feat: auth interceptor and cockroach migrations
* fix: migrations
* fix: fix filter
* fix: not found on getbyid
* fix: add sequence
* fix: add some tests
* fix(eventstore): nullable sequence
* fix: add some tests
* merge
* fix: add some tests
* fix(migrations): correct statements for sequence
* fix: add some tests
* fix: add some tests
* fix: changes from mr
* Update internal/eventstore/models/field.go
Co-Authored-By: livio-a <livio.a@gmail.com>
* fix(eventstore): code quality
* fix: add types to aggregate/Event-types
* fix(eventstore): rename modifier* to editor*
* fix(eventstore): delete editor_org
* fix(migrations): remove editor_org field,
rename modifier_* to editor_*
* fix: generate files
* fix(eventstore): tests
* fix(eventstore): rename modifier to editor
* fix(migrations): add cluster migration,
fix(migrations): fix typo of host in clean clsuter
* fix(eventstore): move health
* fix(eventstore): AggregateTypeFilter aggregateType as param
* code quality
* feat: start implementing project members
* feat: remove member funcs
* feat: remove member model
* feat: remove member events
* feat: remove member repo model
* fix: better error func testing
* Update docs/local.md
Co-Authored-By: Silvan <silvan.reusser@gmail.com>
* Update docs/local.md
Co-Authored-By: Silvan <silvan.reusser@gmail.com>
* fix: mr requests
* fix: md file
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: livio-a <livio.a@gmail.com>
2020-04-07 13:23:04 +02:00
Silvan
fbeab4c582
Eventstore ( #22 )
...
* feat: eventstore repository
* fix: remove gorm
* version
* feat: pkg
* feat: eventstore without eventstore-lib
* rename files
* gnueg
* fix: add object
* fix: global model
* feat(eventstore): sdk
* fix(eventstore): search query
* fix(eventstore): rename app to eventstore
* delete empty test
* remove unused func
* merge master
* fix(eventstore): tests
* fix(models): delete unused struct
* feat(eventstore): implemented push events
* feat(eventstore): overwrite context data
* feat(types): SQL-config
* feat(eventstore): options to overwrite editor
* Update internal/eventstore/models/field.go
Co-Authored-By: livio-a <livio.a@gmail.com>
* fix(eventstore): code quality
* fix(eventstore): rename modifier* to editor*
* fix(eventstore): delete editor_org
Co-authored-by: Fabiennne <fabienne.gerschwiler@gmail.com>
Co-authored-by: livio-a <livio.a@gmail.com>
2020-04-06 06:42:21 +02:00
