Commit Graph

20 Commits

Author SHA1 Message Date
Silvan
99e1c654a3
feat(storage): read only transactions for queries (#6415)
* fix: tests

* bastle wie en grosse

* fix(database): scan as callback

* fix tests

* fix merge failures

* remove as of system time

* refactor: remove unused test

* refacotr: remove unused lines
2023-08-22 10:49:22 +00:00
Livio Spring
cd5e176e30
fix: user grant by id (#6242) 2023-07-21 11:04:55 +00:00
Tim Möhlmann
c0e45b63d8
fix: reset the call timestamp after a bulk trigger (#6080)
* reproduce #5808

Add an integration test that imports and gets N amount of human users.
- With N set to 1-10 the operation seems to succeed always
- With N set to 100 the operation seems to fail between 1 and 7 times.

* fix merge issue

* fix: reset the call timestamp after a bulk trigger

With the use of `AS OF SYSTEM TIME` in queries,
there was a change for the query package not
finding the latest projection verson after
a bulk trigger.
If events where processed in the bulk trigger,
the resulting row timestamp would be after the call
start timestamp.
This sometimes resulted in consistency issues when
Set and Get API methods are called in short succession.
For example a Import and Get user could sometimes result in a Not Found
error.

Although the issue was reported for the Management API user import,
it is likely this bug contributed to the flaky integration and e2e tests.

Fixes #5808

* trigger bulk action in GetSession

* don't use the new context in handler schedule

* disable reproduction test

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-07 08:15:05 +00:00
Silvan
e688954308
feat: role claims for service user tokens (#5577)
tokens of service users can now contain role claims by requesting them through scopes
2023-04-03 14:26:51 +02:00
Silvan
eb4f7c5d7c
fix(auth): update user grants before check (#5406) 2023-03-13 08:03:49 +01:00
Silvan
20e4f1ce57
feat(actions): add fields to complement token flow (#5336)
* deprecated `ctx.v1.userinfo`-field in "pre userinfo creation" trigger in favour of `ctx.v1.claims`. The trigger now behaves the same as "pre access token creation"
* added `ctx.v1.claims` to "complement tokens" flow
* added `ctx.v1.grants` to "complement tokens" flow
* document `ctx.v1.getUser()` in "complement tokens" flow

* feat(actions): add getUser() and grant

* map user grants

* map claims

* feat(actions): claims in complement token ctx

* docs(actions): add new fields of complement token

* docs(actions): additions to complement token

* docs(actions): correct field names
2023-03-08 15:26:28 +01:00
Silvan
e38abdcdf3
perf: query data AS OF SYSTEM TIME (#5231)
Queries the data in the storage layser at the timestamp when the call hit the API layer
2023-02-27 22:36:43 +01:00
Silvan
28760ab4b3
fix(query): add tracing for each method (#4777)
* fix(query): add tracing for each method
2022-12-01 09:18:53 +01:00
Silvan
f3e6f3b23b
feat: remove org (#4148)
* feat(command): remove org

* refactor: imports, unused code, error handling

* reduce org removed in action

* add org deletion to projections

* add org removal to projections

* add org removal to projections

* org removed projection

* lint import

* projections

* fix: table names in tests

* fix: table names in tests

* logging

* add org state

* fix(domain): add Owner removed to object details

* feat(ListQuery): add with owner removed

* fix(org-delete): add bool to functions to select with owner removed

* fix(org-delete): add bools to user grants with events to determine if dependencies lost owner

* fix(org-delete): add unit tests for owner removed and org removed events

* fix(org-delete): add handling of org remove for grants and members

* fix(org-delete): correction of unit tests for owner removed

* fix(org-delete): update projections, unit tests and get functions

* fix(org-delete): add change date to authnkeys and owner removed to org metadata

* fix(org-delete): include owner removed for login names

* fix(org-delete): some column fixes in projections and build for queries with owner removed

* indexes

* fix(org-delete): include review changes

* fix(org-delete): change user projection name after merge

* fix(org-delete): include review changes for project grant where no project owner is necessary

* fix(org-delete): include auth and adminapi tables with owner removed information

* fix(org-delete): cleanup username and orgdomain uniqueconstraints when org is removed

* fix(org-delete): add permissions for org.remove

* remove unnecessary unique constraints

* fix column order in primary keys

* fix(org-delete): include review changes

* fix(org-delete): add owner removed indexes and chang setup step to create tables

* fix(org-delete): move PK order of instance_id and change added user_grant from review

* fix(org-delete): no params for prepareUserQuery

* change to step 6

* merge main

* fix(org-delete): OldUserName rename to private

* fix linting

* cleanup

* fix: remove org test

* create prerelease

* chore: delete org-delete as prerelease

Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2022-11-30 17:01:17 +01:00
Livio Spring
441f215513
fix: join on instanceIDs in queries (#4612) 2022-10-27 06:08:36 +00:00
Silvan
77b4fc5487
feat(database): support for postgres (#3998)
* beginning with postgres statements

* try pgx

* use pgx

* database

* init works for postgres

* arrays working

* init for cockroach

* init

* start tests

* tests

* TESTS

* ch

* ch

* chore: use go 1.18

* read stmts

* fix typo

* tests

* connection string

* add missing error handler

* cleanup

* start all apis

* go mod tidy

* old update

* switch back to minute

* on conflict

* replace string slice with `database.StringArray` in db models

* fix tests and start

* update go version in dockerfile

* setup go

* clean up

* remove notification migration

* update

* docs: add deploy guide for postgres

* fix: revert sonyflake

* use `database.StringArray` for daos

* use `database.StringArray` every where

* new tables

* index naming,
metadata primary key,
project grant role key type

* docs(postgres): change to beta

* chore: correct compose

* fix(defaults): add empty postgres config

* refactor: remove unused code

* docs: add postgres to self hosted

* fix broken link

* so?

* change title

* add mdx to link

* fix stmt

* update goreleaser in test-code

* docs: improve postgres example

* update more projections

* fix: add beta log for postgres

* revert index name change

* prerelease

* fix: add sequence to v1 "reduce paniced"

* log if nil

* add logging

* fix: log output

* fix(import): check if org exists and user

* refactor: imports

* fix(user): ignore malformed events

* refactor: method naming

* fix: test

* refactor: correct errors.Is call

* ci: don't build dev binaries on main

* fix(go releaser): update version to 1.11.0

* fix(user): projection should not break

* fix(user): handle error properly

* docs: correct config example

* Update .releaserc.js

* Update .releaserc.js

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
2022-08-31 07:52:43 +00:00
Livio Spring
aed7010508
fix: scheduling (#3978)
* fix: improve scheduling

* build pre-release

* fix: locker

* fix: user handler and print stack in case of panic in reducer

* chore: remove sentry

* fix: improve handler projection and implement tests

* more tests

* fix: race condition in tests

* Update internal/eventstore/repository/sql/query.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* fix: implemented suggested changes

* fix: lock statement

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-07-22 10:08:39 +00:00
Silvan
dd2f31683c
fix(query): realtime data on defined requests (#3726)
* feat: directly specify factors on addCustomLoginPolicy and return on LoginPolicy responses

* fix proto

* update login policy

* feat: directly specify idp on addCustomLoginPolicy and return on LoginPolicy responses

* fix: tests

* fix(projection): trigger bulk

* refactor: clean projection pkg

* instance should bulk

* fix(query): should trigger bulk on id calls

* tests

* build prerelease

* fix: add shouldTriggerBulk

* fix: test

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2022-06-14 07:51:00 +02:00
Florian Forster
fa9f581d56
chore(v2): move to new org (#3499)
* chore: move to new org

* logging

* fix: org rename caos -> zitadel

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2022-04-26 23:01:45 +00:00
Livio Amstutz
958362e6c9
feat: handle instance from context (#3382)
* commander

* commander

* selber!

* move to packages

* fix(errors): implement Is interface

* test: command

* test: commands

* add init steps

* setup tenant

* add default step yaml

* possibility to set password

* merge v2 into v2-commander

* fix: rename iam command side to instance

* fix: rename iam command side to instance

* fix: rename iam command side to instance

* fix: rename iam command side to instance

* fix: search query builder can filter events in memory

* fix: filters for add member

* fix(setup): add `ExternalSecure` to config

* chore: name iam to instance

* fix: matching

* remove unsued func

* base url

* base url

* test(command): filter funcs

* test: commands

* fix: rename orgiampolicy to domain policy

* start from init

* commands

* config

* fix indexes and add constraints

* fixes

* fix: merge conflicts

* fix: protos

* fix: md files

* setup

* add deprecated org iam policy again

* typo

* fix search query

* fix filter

* Apply suggestions from code review

* remove custom org from org setup

* add todos for verification

* change apps creation

* simplify package structure

* fix error

* move preparation helper for tests

* fix unique constraints

* fix config mapping in setup

* fix error handling in encryption_keys.go

* fix projection config

* fix query from old views to projection

* fix setup of mgmt api

* set iam project and fix instance projection

* fix tokens view

* fix steps.yaml and defaults.yaml

* fix projections

* change instance context to interface

* instance interceptors and additional events in setup

* cleanup

* tests for interceptors

* fix label policy

* add todo

* single api endpoint in environment.json

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-03-29 11:53:19 +02:00
Livio Amstutz
504fe5b761
cherry pick changes from main (#3371)
* feat: remove exif data from uploaded images (#3221)

* feat: remove exif tags from images

* feat: remove exif data

* feat: remove exif

* fix: add preferredLoginName to user grant response (#3271)

* chore: log webauthn parse error (#3272)

* log error

* log error

* feat: Help link in privacy policy

* fix: convert correct detail data on organization (#3279)

* fix: handle empty editor users

* fix: add some missing translations (#3291)

* fix: org policy translations

* fix: metadata event types translation

* fix: translations

* fix: filter resource owner correctly on project grant members (#3281)

* fix: filter resource owner correctly on project grant members

* fix: filter resource owner correctly on project grant members

* fix: add orgIDs to zitadel permissions request

Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>

* fix: get IAM memberships correctly in MyZitadelPermissions (#3309)

* fix: correct login names on auth and notification users (#3349)

* fix: correct login names on auth and notification users

* fix: migration

* fix: handle resource owner in action flows (#3361)

* fix merge

* fix: exchange exif library (#3366)

* fix: exchange exif library

* ignore tiffs

* requested fixes

* feat: Help link in privacy policy

Co-authored-by: Fabi <38692350+fgerschwiler@users.noreply.github.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-03-24 14:00:24 +01:00
Livio Amstutz
56b916a2b0
feat: projections auto create their tables (#3324)
* begin init checks for projections

* first projection checks

* debug notification providers with query fixes

* more projections and first index

* more projections

* more projections

* finish projections

* fix tests (remove db name)

* create tables in setup

* fix logging / error handling

* add tenant to views

* rename tenant to instance_id

* add instance_id to all projections

* add instance_id to all queries

* correct instance_id on projections

* add instance_id to failed_events

* use separate context for instance

* implement features projection

* implement features projection

* remove unique constraint from setup when migration failed

* add error to failed setup event

* add instance_id to primary keys

* fix IAM projection

* remove old migrations folder

* fix keysFromYAML test
2022-03-23 09:02:39 +01:00
Livio Amstutz
e99b7f4972
fix: move activity log to queries and remove old code (#3096)
* move changes to queries and remove old code

* fix changes query

* remove unused code

* fix sorting

* fix sorting

* refactor and remove old code

* remove accidental go.mod replace

* add missing file

* remove listDetail from ChangesResponse
2022-01-26 10:16:33 +01:00
Livio Amstutz
44d78df4d4
feat: user query (#3075)
* user queries

* user query

* user query

* user tests

* remove old code

* user metadata

* cleanup

* fix merge

* cleanup

* cleanup

* fixes
2022-01-20 14:40:25 +00:00
Silvan
c542cab4f8
feat(queries): user grants (#2838)
* refactor(domain): add user type

* fix(projections): start with login names

* fix(login_policy): correct handling of user domain claimed event

* fix(projections): add members

* refactor: simplify member projections

* add migration for members

* add metadata to member projections

* refactor: login name projection

* fix: set correct suffixes on login name projections

* test(projections): login name reduces

* fix: correct cols in reduce member

* test(projections): org, iam, project members

* member additional cols and conds as opt,
add project grant members

* fix(migration): members

* fix(migration): correct database name

* migration version

* migs

* better naming for member cond and col

* split project and project grant members

* prepare member columns

* feat(queries): membership query

* test(queries): membership prepare

* fix(queries): multiple projections for latest sequence

* fix(api): use query for membership queries in auth and management

* feat: org member queries

* fix(api): use query for iam member calls

* fix(queries): org members

* fix(queries): project members

* fix(queries): project grant members

* fix(query): member queries and user avatar column

* member cols

* fix(queries): membership stmt

* fix user test

* fix user test

* fix(projections): add user grant projection

* fix(user_grant): handle state changes

* add state to migration

* fix(management): use query for user grant requests

* merge eventstore-naming into user-grant-projection

* feat(queries): user grants

* fix(migrations): version

* fix(api): user query for user grants

* fix(query): event mappers for usergrant aggregate

* fix(projection): correct aggregate for user grants

* fix(queries): user grant roles as list contains

* cleanup reducers

* fix avater_key to avatar_key

* tests

* cleanup

* cleanup

* add resourceowner query

* fix: user grant project name search query

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-01-14 09:45:50 +00:00