Commit Graph

443 Commits

Author SHA1 Message Date
Livio Spring
6ab06aa249
fix: improve secret generation for apple idp (#7843)
* fix: improve secret generation for apple idp

* remove accidental commit

* change exp time

* change exp time

* change exp time

* change exp time
2024-04-26 15:46:15 +00:00
Livio Spring
4f3564e4e9
fix: disable auth cache by default (#7845) 2024-04-26 09:30:35 +02:00
Livio Spring
d016379e2a
feat: pass and handle auth request context for email links (#7815)
* pass and handle auth request context

* tests and cleanup

* cleanup
2024-04-24 17:50:58 +02:00
Silvan
25030c69b9
perf: cache auth request in memory (#7824)
* perf: cache auth request in memory
2024-04-23 11:23:50 +00:00
Elio Bischof
5b34aabbdd
fix(setup): decode role mappings (#7760) 2024-04-14 12:55:54 +03:00
Livio Spring
153df2e12f
feat: provide option to limit (T)OTP checks (#7693)
* feat: provide option to limit (T)OTP checks

* fix requests in console

* update errors pkg

* cleanup

* cleanup

* improve naming of existing config
2024-04-10 09:14:55 +00:00
Stefan Benz
6dcdef0268
fix: add action v2 execution to features (#7597)
* fix: add action v2 execution to features

* fix: add action v2 execution to features

* fix: add action v2 execution to features

* fix: update internal/command/instance_features_model.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: merge back main

* fix: merge back main

* fix: rename feature and service

* fix: rename feature and service

* fix: review changes

* fix: review changes

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-04-09 20:21:21 +03:00
Tim Möhlmann
2089992d75
feat(crypto): use passwap for machine and app secrets (#7657)
* feat(crypto): use passwap for machine and app secrets

* fix command package tests

* add hash generator command test

* naming convention, fix query tests

* rename PasswordHasher and cleanup start commands

* add reducer tests

* fix intergration tests, cleanup old config

* add app secret unit tests

* solve setup panics

* fix push of updated events

* add missing event translations

* update documentation

* solve linter errors

* remove nolint:SA1019 as it doesn't seem to help anyway

* add nolint to deprecated filter usage

* update users migration version

* remove unused ClientSecret from APIConfigChangedEvent

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-04-05 09:35:49 +00:00
Tim Möhlmann
25ef3da9d5
refactor(fmt): run gci on complete project (#7557)
chore(fmt): run gci on complete project

Fix global import formatting in go code by running the `gci` command. This allows us to just use the command directly, instead of fixing the import order manually for the linter, on each PR.

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-04-03 10:43:43 +00:00
Stefan Benz
217703395e
feat: add user v2 pw change required information on query (#7603)
* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute

* fix: review changes

* fix: review changes

* fix: review changes

* fix: review changes

* fix: add password change required to user v2 get and list

* fix: update unit tests for query side with new column and projection

* fix: change projection in setup steps

* fix: change projection in setup steps

* fix: remove setup step 25

* fix: add password_change_required into ListUsers response

* fix: correct SetUserPassword parameters

* fix: rollback to change setup instead of projection directly

* fix: rollback to change setup instead of projection directly

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-03-28 06:21:21 +00:00
Silvan
56df515e5f
chore: use pgx v5 (#7577)
* chore: use pgx v5

* chore: update go version

* remove direct pq dependency

* remove unnecessary type

* scan test

* map scanner

* converter

* uint8 number array

* duration

* most unit tests work

* unit tests work

* chore: coverage

* go 1.21

* linting

* int64 gopfertammi

* retry go 1.22

* retry go 1.22

* revert to go v1.21.5

* update go toolchain to 1.21.8

* go 1.21.8

* remove test flag

* go 1.21.5

* linting

* update toolchain

* use correct array

* use correct array

* add byte array

* correct value

* correct error message

* go 1.21 compatible
2024-03-27 15:48:22 +02:00
Livio Spring
d313e6d498
fix(setup): enable init-projection by default (#7616)
* fix(setup): enable init-projection by default

* update A10008
2024-03-23 12:52:52 +01:00
Livio Spring
7494a7b6d9
feat(api): add possibility to retrieve user schemas (#7614)
This PR extends the user schema service (V3 API) with the possibility to ListUserSchemas and GetUserSchemaByID.
The previously started guide is extended to demonstrate how to retrieve the schema(s) and notes the generated revision property.
2024-03-22 13:26:13 +00:00
Livio Spring
7e24a1adbc
fix: allow login by email case-insensitive (#7578)
A customer noted that the login by email was case-sensitive, which differs to the handling of the loginname.

This PR changes the email check to be case-insensitive (which it was already in same parts) and improve the search for this as well.
2024-03-20 15:51:26 +00:00
Tim Möhlmann
6398349c24
feat(oidc): token exchange impersonation (#7516)
* add token exchange feature flag

* allow setting reason and actor to access tokens

* impersonation

* set token types and scopes in response

* upgrade oidc to working draft state

* fix tests

* audience and scope validation

* id toke and jwt as input

* return id tokens

* add grant type  token exchange to app config

* add integration tests

* check and deny actors in api calls

* fix instance setting tests by triggering projection on write and cleanup

* insert sleep statements again

* solve linting issues

* add translations

* pin oidc v3.15.0

* resolve comments, add event translation

* fix refreshtoken test

* use ValidateAuthReqScopes from oidc

* apparently the linter can't make up its mind

* persist actor thru refresh tokens and check in tests

* remove unneeded triggers
2024-03-20 10:18:46 +00:00
Stefan Benz
fb3c6f791b
feat: query side for executions and targets for actions v2 (#7524)
* feat: add projections and query side to executions and targets

* feat: add list and get endpoints for targets

* feat: add integration tests for query endpoints target and execution

* fix: linting

* fix: linting

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: review changes, renames and corrections

* fix: remove position from list details
2024-03-14 09:56:23 +00:00
Livio Spring
0e181b218c
feat: implement user schema management (#7416)
This PR adds the functionality to manage user schemas through the new user schema service.
It includes the possibility to create a basic JSON schema and also provides a way on defining permissions (read, write) for owner and self context with an annotation.

Further annotations for OIDC claims and SAML attribute mappings will follow.

A guide on how to create a schema and assign permissions has been started. It will be extended though out the process of implementing the schema and users based on those.

Note:
This feature is in an early stage and therefore not enabled by default. To test it out, please enable the UserSchema feature flag on your instance / system though the feature service.
2024-03-12 13:50:13 +00:00
Silvan
60ee2610f2
fix(eventstore): consider IsGlobal-flag of constraints (#7518)
* fix(eventstore): consider `IsGlobal`-flag of constraints

* fix(setup): set `instance_domain`-constraint global
2024-03-08 13:33:53 +00:00
Elio Bischof
3ae4e9e6cf
docs: describe DefaultInstance vs FirstInstance (#7487)
* docs: describe DefaultInstance vs FirstInstance

* link to docs

* add better searchable tip to the docs

* add better searchable tip to the docs

* add link
2024-03-05 08:37:12 +01:00
Elio Bischof
f4c72cbe14
feat: improve instance not found error (#7413)
* feat: improve instance not found error

* unit tests

* check if is templatable

* lint

* assert

* compile tests

* remove error templates

* link to instance not found page

* fmt

* cleanup

* lint
2024-02-28 10:49:57 +00:00
Tim Möhlmann
062d153cfe
feat: impersonation roles (#7442)
* partial work done

* test IAM membership roles

* org membership tests

* console :(, translations and docs

* fix integration test

* fix tests

* add EnableImpersonation to security policy API

* fix integration test timestamp checking

* add security policy tests and fix projections

* add impersonation setting in console

* add security settings to the settings v2 API

* fix typo

* move impersonation to instance

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-28 10:21:11 +00:00
Tim Möhlmann
26d1563643
feat(api): feature flags (#7356)
* feat(api): feature API proto definitions

* update proto based on discussion with @livio-a

* cleanup old feature flag stuff

* authz instance queries

* align defaults

* projection definitions

* define commands and event reducers

* implement system and instance setter APIs

* api getter implementation

* unit test repository package

* command unit tests

* unit test Get queries

* grpc converter unit tests

* migrate the V1 features

* migrate oidc to dynamic features

* projection unit test

* fix instance by host

* fix instance by id data type in sql

* fix linting errors

* add system projection test

* fix behavior inversion

* resolve proto file comments

* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event

* use write models and conditional set events

* system features integration tests

* instance features integration tests

* error on empty request

* documentation entry

* typo in feature.proto

* fix start unit tests

* solve linting error on key case switch

* remove system defaults after discussion with @eliobischof

* fix system feature projection

* resolve comments in defaults.yaml

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-28 10:55:54 +02:00
Stefan Benz
2731099db3
feat: add executions for actions v2 (#7433)
* feat: add events for execution

* feat: add events for execution and command side

* feat: add events for execution and command side

* feat: add api endpoints for set and delete executions with integration tests

* feat: add integration and unit tests and more existence checks

* feat: add integration and unit tests and more existence checks

* feat: unit tests for includes in executions

* feat: integration tests for includes in executions

* fix: linting

* fix: update internal/api/api.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: update internal/command/command.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: apply suggestions from code review

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: change api return

* fix: change aggregateID with prefix of execution type and add to documentation

* fix: change body in proto for documentation and correct linting

* fix: changed existing check to single query in separate writemodel

* fix: linter changes and list endpoints for conditions in executions

* fix: remove writemodel query on exeuction set as state before is irrelevant

* fix: testing for exists write models and correction

* fix: translations for errors and event types

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-02-26 12:49:43 +02:00
Tim Möhlmann
1890e28f79
fix(eventstore): retry push on primary key sequence collision (#7420)
* fix(eventstore): retry push on primary key sequence collision

* MaxRetries config option and unit test
2024-02-23 08:29:10 +00:00
David Schneider
ffdde825ec
fix(backend): respect start flags in all commands (#6621)
* fix(backend): respect start flags in all commands

Currently flags like --externalDomain do only work in the last
registered command which currently is start-from-setup.
This creates the flags globally in the init function in uses them for
all start commands.

* fix(backend): remove viper defaults in start flags

At this point viper is not yet initialized so this defaults would have
not effect either.

* Remove flag name variables and run go mod tidy

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-02-21 12:26:51 +00:00
Elio Bischof
19af2f7372
feat: support whole config as env (#6336)
* fix existing env vars

* feat: support all config by env

* cleanup

* remove system users hook

* decode system users in setup
2024-02-16 16:04:42 +00:00
Stefan Benz
198bc017b8
feat: actions v2 execution targets command side (#7384)
Adds the API to create, update, delete targets for execution in a new ExecutionService (v3alpha)
2024-02-15 05:39:10 +00:00
Tim Möhlmann
df57a64ed7
fix(oidc): ignore public key expiry for ID Token hints (#7293)
* fix(oidc): ignore public key expiry for ID Token hints

This splits the key sets used for access token and ID token hints.
ID Token hints should be able to be verified by with public keys that are already expired.
However, we do not want to change this behavior for Access Tokens,
where an error for an expired public key is still returned.

The public key cache is modified to purge public keys based on last use,
instead of expiry.
The cache is shared between both verifiers.

* resolve review comments

* pin oidc 3.11
2024-01-29 15:11:52 +00:00
Silvan
17953e9040
fix(setup): init projections (#7194)
Even though this is a feature it's released as fix so that we can back port to earlier revisions.

As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.

The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
2024-01-25 17:28:20 +01:00
Elio Bischof
ed0bc39ea4
feat: block instances (#7129)
* docs: fix init description typos

* feat: block instances using limits

* translate

* unit tests

* fix translations

* redirect /ui/login

* fix http interceptor

* cleanup

* fix http interceptor

* fix: delete cookies on gateway 200

* add integration tests

* add command test

* docs

* fix integration tests

* add bulk api and integration test

* optimize bulk set limits

* unit test bulk limits

* fix broken link

* fix assets middleware

* fix broken link

* validate instance id format

* Update internal/eventstore/search_query.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove support for owner bulk limit commands

* project limits to instances

* migrate instances projection

* Revert "migrate instances projection"

This reverts commit 214218732a.

* join limits, remove owner

* remove todo

* use optional bool

* normally validate instance ids

* use 302

* cleanup

* cleanup

* Update internal/api/grpc/system/limits_converter.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove owner

* remove owner from reset

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-01-17 10:16:48 +00:00
Stefan Benz
d9d376a275
feat: user v2 service query (#7095)
* feat: add query endpoints for user v2 api

* fix: correct integration tests

* fix: correct linting

* fix: correct linting

* fix: comment out permission check on user get and list

* fix: permission check on user v2 query

* fix: merge back origin/main

* fix: add search query in user emails

* fix: reset count for SearchUser if users are removed due to permissions

* fix: reset count for SearchUser if users are removed due to permissions

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-01-17 10:00:10 +01:00
Elio Bischof
29b386005d
fix(origin): fall back to ExternalSecure (#7228)
* fix(origin): fall back to ExternalSecure

* avoid middleware.Middleware

* avoid else

* lint
2024-01-15 16:44:35 +00:00
Silvan
43f1d59649
fix(auth): efficient user session projection (#7187)
* fix(auth): cache users during session projection

* fix(auth.user_sessions): add index for more efficient by user search
2024-01-09 18:36:46 +00:00
Silvan
a5d4b08a99
fix(cleanup): cleanup all stuck states (#7145)
* fix(setup): unmarshal of failed step

* fix(cleanup): cleanup all stuck states

* use lastRun for repeatable steps

* typo

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-01-05 09:01:48 +00:00
Silvan
b7d027e2fd
fix(db): always use begin tx (#7142)
* fix(db): always use begin tx

* fix(handler): timeout for begin
2024-01-04 16:12:20 +00:00
Silvan
cc2dd8b20b
fix(eventstore): increase performance on push (#7125) 2023-12-31 15:30:25 +01:00
Tim Möhlmann
fe1337536f
fix(db): add additional connection pool for projection spooling (#7094)
* fix(db): add additional connection pool for projection spooling

* use correct connection pool for projections

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-12-20 16:13:04 +00:00
Livio Spring
edaa41903e
fix(projections): handle every instance by default and randomize start (#7093) 2023-12-19 13:32:08 +02:00
Fabi
2c4e7070ea
docs: correct env var docs (#7082)
Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-18 10:54:43 +00:00
Tim Möhlmann
f680dd934d
refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors

* rename package errors to gerrors

* fix error related linting issues

* fix zitadel error assertion

* fix gosimple linting issues

* fix deprecated linting issues

* resolve gci linting issues

* fix import structure

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-08 15:30:55 +01:00
Silvan
ddbea119f1
fix(query): user performance (#6537)
* start user by id

* ignore debug bin

* use new user by id

* new sql

* fix(sql): replace STRING with text for psql compatabilit

* some changes

* fix: correct user queries

* fix tests

* unify sql statements

* use specific get user methods

* search login name case insensitive

* refactor: optimise user statements

* add index

* fix queries

* fix: correct domain segregation

* return all login names

* fix queries

* improve readability

* query should be correct now

* cleanup statements

* fix username / loginname handling

* fix: psql doesn't support create view if not exists

* fix: create pre-release

* ignore release comments

* add lower fields

* fix: always to lower

* update to latest projection

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-12-08 13:14:22 +01:00
Tim Möhlmann
ec03340b67
perf(oidc): optimize client verification (#6999)
* fix some spelling errors

* client credential auth

* implementation of client auth

* improve error handling

* unit test command package

* unit test database package

* unit test query package

* cleanup unused tracing func

* fix integration tests

* errz to zerrors

* fix linting and import issues

* fix another linting error

* integration test with client secret

* Revert "integration test with client secret"

This reverts commit 0814ba522f.

* add integration tests

* client credentials integration test

* resolve comments

* pin oidc v3.5.0
2023-12-05 17:01:03 +00:00
Elio Bischof
dd33538c0a
feat: restrict languages (#6931)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* configure supported languages

* fix allowed languages

* fix tests

* default lang must not be restricted

* preferred language must be allowed

* change preferred languages

* check languages everywhere

* lint

* test command side

* lint

* add integration test

* add integration test

* restrict supported ui locales

* lint

* lint

* cleanup

* lint

* allow undefined preferred language

* fix integration tests

* update main

* fix env var

* ignore linter

* ignore linter

* improve integration test config

* reduce cognitive complexity

* compile

* check for duplicates

* remove useless restriction checks

* review

* revert restriction renaming

* fix language restrictions

* lint

* generate

* allow custom texts for supported langs for now

* fix tests

* cleanup

* cleanup

* cleanup

* lint

* unsupported preferred lang is allowed

* fix integration test

* finish reverting to old property name

* finish reverting to old property name

* load languages

* refactor(i18n): centralize translators and fs

* lint

* amplify no validations on preferred languages

* fix integration test

* lint

* fix resetting allowed languages

* test unchanged restrictions
2023-12-05 11:12:01 +00:00
Tim Zook
302b4b90d4
fix(init): correct quoting of database and user (#6928)
* fix(init): correct quoting

* quote username in logstore migration

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-12-03 08:30:08 +00:00
Silvan
e3d1ca4d58
fix(eventstore): improve pagination of handler filter (#6968)
* fix(setup): add filter_offset to `projections.current_states`

* fix(eventstore): allow offset in query

* fix(handler): offset for already processed events
2023-12-01 12:25:41 +00:00
Stefan Benz
ef11609142
fix: add https status to activity log (#6978)
* fix: add https status to activity log

* create prerelease

* create RC

* pass info from gateway to grpc server

* fix: update releaserc to create RC version

* cleanup

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-11-28 16:56:29 +01:00
Silvan
2ca88956b4
docs: add tracing.endpoint to defaults.yaml (#6824)
* docs: add `tracing.endpoint` to defaults.yaml

* docs: describe tracing types in defaults.yaml
2023-11-24 13:38:52 +01:00
Livio Spring
b563041103
fix: ensure uniqueness (#6956)
* fix: ensure uniqueness

* only update wrong ones

* Update cmd/setup/16.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-11-22 12:05:14 +00:00
Elio Bischof
76fe032b5f
feat: option to disallow public org registration (#6917)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* lint
2023-11-22 09:29:38 +00:00
Tim Möhlmann
ba9b807854
perf(oidc): optimize the introspection endpoint (#6909)
* get key by id and cache them

* userinfo from events for v2 tokens

* improve keyset caching

* concurrent token and client checks

* client and project in single query

* logging and otel

* drop owner_removed column on apps and authN tables

* userinfo and project roles in go routines

* get  oidc user info from projections and add actions

* add avatar URL

* some cleanup

* pull oidc work branch

* remove storage from server

* add config flag for experimental introspection

* legacy introspection flag

* drop owner_removed column on user projections

* drop owner_removed column on useer_metadata

* query userinfo unit test

* query introspection client test

* add user_grants to the userinfo query

* handle PAT scopes

* bring triggers back

* test instance keys query

* add userinfo unit tests

* unit test keys

* go mod tidy

* solve some bugs

* fix missing preferred login name

* do not run triggers in go routines, they seem to deadlock

* initialize the trigger handlers late with a sync.OnceValue

* Revert "do not run triggers in go routines, they seem to deadlock"

This reverts commit 2a03da2127.

* add missing translations

* chore: update go version for linting

* pin oidc version

* parse a global time location for query test

* fix linter complains

* upgrade go lint

* fix more linting issues

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-11-21 13:11:38 +01:00