Commit Graph

97 Commits

Author SHA1 Message Date
Tim Möhlmann
26d1563643
feat(api): feature flags (#7356)
* feat(api): feature API proto definitions

* update proto based on discussion with @livio-a

* cleanup old feature flag stuff

* authz instance queries

* align defaults

* projection definitions

* define commands and event reducers

* implement system and instance setter APIs

* api getter implementation

* unit test repository package

* command unit tests

* unit test Get queries

* grpc converter unit tests

* migrate the V1 features

* migrate oidc to dynamic features

* projection unit test

* fix instance by host

* fix instance by id data type in sql

* fix linting errors

* add system projection test

* fix behavior inversion

* resolve proto file comments

* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event

* use write models and conditional set events

* system features integration tests

* instance features integration tests

* error on empty request

* documentation entry

* typo in feature.proto

* fix start unit tests

* solve linting error on key case switch

* remove system defaults after discussion with @eliobischof

* fix system feature projection

* resolve comments in defaults.yaml

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-28 10:55:54 +02:00
Stefan Benz
2731099db3
feat: add executions for actions v2 (#7433)
* feat: add events for execution

* feat: add events for execution and command side

* feat: add events for execution and command side

* feat: add api endpoints for set and delete executions with integration tests

* feat: add integration and unit tests and more existence checks

* feat: add integration and unit tests and more existence checks

* feat: unit tests for includes in executions

* feat: integration tests for includes in executions

* fix: linting

* fix: update internal/api/api.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: update internal/command/command.go

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: apply suggestions from code review

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix: change api return

* fix: change aggregateID with prefix of execution type and add to documentation

* fix: change body in proto for documentation and correct linting

* fix: changed existing check to single query in separate writemodel

* fix: linter changes and list endpoints for conditions in executions

* fix: remove writemodel query on exeuction set as state before is irrelevant

* fix: testing for exists write models and correction

* fix: translations for errors and event types

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2024-02-26 12:49:43 +02:00
Elio Bischof
585988bd83
chore(ci): merge test runs (#7388)
* chore(ci): merge test runs

* docs: improve CONTRIBUTING.md

* expect failure

* expect failure unit

* cleanup

* fix: correct binding in integration tests

* fix: correct binding in integration tests

---------

Co-authored-by: Stefan Benz <stefan@caos.ch>
2024-02-19 07:50:37 +01:00
Stefan Benz
198bc017b8
feat: actions v2 execution targets command side (#7384)
Adds the API to create, update, delete targets for execution in a new ExecutionService (v3alpha)
2024-02-15 05:39:10 +00:00
Silvan
17953e9040
fix(setup): init projections (#7194)
Even though this is a feature it's released as fix so that we can back port to earlier revisions.

As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.

The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
2024-01-25 17:28:20 +01:00
Elio Bischof
ed0bc39ea4
feat: block instances (#7129)
* docs: fix init description typos

* feat: block instances using limits

* translate

* unit tests

* fix translations

* redirect /ui/login

* fix http interceptor

* cleanup

* fix http interceptor

* fix: delete cookies on gateway 200

* add integration tests

* add command test

* docs

* fix integration tests

* add bulk api and integration test

* optimize bulk set limits

* unit test bulk limits

* fix broken link

* fix assets middleware

* fix broken link

* validate instance id format

* Update internal/eventstore/search_query.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove support for owner bulk limit commands

* project limits to instances

* migrate instances projection

* Revert "migrate instances projection"

This reverts commit 214218732a.

* join limits, remove owner

* remove todo

* use optional bool

* normally validate instance ids

* use 302

* cleanup

* cleanup

* Update internal/api/grpc/system/limits_converter.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* remove owner

* remove owner from reset

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-01-17 10:16:48 +00:00
Stefan Benz
d9d376a275
feat: user v2 service query (#7095)
* feat: add query endpoints for user v2 api

* fix: correct integration tests

* fix: correct linting

* fix: correct linting

* fix: comment out permission check on user get and list

* fix: permission check on user v2 query

* fix: merge back origin/main

* fix: add search query in user emails

* fix: reset count for SearchUser if users are removed due to permissions

* fix: reset count for SearchUser if users are removed due to permissions

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-01-17 10:00:10 +01:00
Tim Möhlmann
45ccdcfa99
fix(oidc): nil check for client secret (#7115)
This fixes a nil pointer panic when client basic auth is attempted on a client without secret in introspection.
2023-12-28 13:31:41 +00:00
Yordis Prieto
9d5d1cf3ea
feat: allow glob redirects (#7091)
fixes #5110
2023-12-28 11:25:18 +02:00
Stefan Benz
a0a82b59e1
feat: user service v2 create, update and remove (#6996)
* feat: user service v2 remove user

* feat: user service v2 add user human

* feat: user service v2 change user human

* feat: user service v2 change user human unit tests

* feat: user service v2 reactivate, deactivate, lock, unlock user

* feat: user service v2 integration tests

* fix: merge back origin/main

* lint: linter corrections

* fix: move permission check for isVerfied and password change

* fix: add deprecated notices and other review comments

* fix: consistent naming in proto

* fix: errors package renaming

* fix: remove / delete user renaming in integration test

* fix: machine user status changes through user v2 api

* fix: linting changes

* fix: linting changes

* fix: changes from review

* fix: changes from review

* fix: changes from review

* fix: changes from review

* fix: changes from review

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-12-21 10:03:37 +01:00
Tim Möhlmann
f680dd934d
refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors

* rename package errors to gerrors

* fix error related linting issues

* fix zitadel error assertion

* fix gosimple linting issues

* fix deprecated linting issues

* resolve gci linting issues

* fix import structure

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-08 15:30:55 +01:00
Elio Bischof
8c85318fbd
fix: restrict languages in console (#6964)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* configure supported languages

* fix allowed languages

* fix tests

* default lang must not be restricted

* preferred language must be allowed

* change preferred languages

* check languages everywhere

* lint

* test command side

* lint

* add integration test

* add integration test

* restrict supported ui locales

* lint

* lint

* cleanup

* lint

* allow undefined preferred language

* fix integration tests

* update main

* fix env var

* ignore linter

* ignore linter

* improve integration test config

* reduce cognitive complexity

* compile

* fix(console): switch back to saved language

* feat(API): get allowed languages

* fix(console): only make allowed languages selectable

* warn when editing not allowed languages

* check for duplicates

* remove useless restriction checks

* review

* revert restriction renaming

* fix language restrictions

* lint

* generate

* allow custom texts for supported langs for now

* fix tests

* cleanup

* cleanup

* cleanup

* lint

* unsupported preferred lang is allowed

* fix integration test

* allow unsupported preferred languages

* lint

* load languages for tests

* cleanup

* lint

* cleanup

* get allowed only on admin

* cleanup

* reduce flakiness on very limited postgres

* simplify langSvc

* refactor according to suggestions in pr

* lint

* set first allowed language as default

* selectionchange for language in msg texts

* initialize login texts

* init message texts

* lint

---------

Co-authored-by: peintnermax <max@caos.ch>
2023-12-07 08:43:23 +00:00
Tim Möhlmann
ec03340b67
perf(oidc): optimize client verification (#6999)
* fix some spelling errors

* client credential auth

* implementation of client auth

* improve error handling

* unit test command package

* unit test database package

* unit test query package

* cleanup unused tracing func

* fix integration tests

* errz to zerrors

* fix linting and import issues

* fix another linting error

* integration test with client secret

* Revert "integration test with client secret"

This reverts commit 0814ba522f.

* add integration tests

* client credentials integration test

* resolve comments

* pin oidc v3.5.0
2023-12-05 17:01:03 +00:00
Elio Bischof
dd33538c0a
feat: restrict languages (#6931)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* configure supported languages

* fix allowed languages

* fix tests

* default lang must not be restricted

* preferred language must be allowed

* change preferred languages

* check languages everywhere

* lint

* test command side

* lint

* add integration test

* add integration test

* restrict supported ui locales

* lint

* lint

* cleanup

* lint

* allow undefined preferred language

* fix integration tests

* update main

* fix env var

* ignore linter

* ignore linter

* improve integration test config

* reduce cognitive complexity

* compile

* check for duplicates

* remove useless restriction checks

* review

* revert restriction renaming

* fix language restrictions

* lint

* generate

* allow custom texts for supported langs for now

* fix tests

* cleanup

* cleanup

* cleanup

* lint

* unsupported preferred lang is allowed

* fix integration test

* finish reverting to old property name

* finish reverting to old property name

* load languages

* refactor(i18n): centralize translators and fs

* lint

* amplify no validations on preferred languages

* fix integration test

* lint

* fix resetting allowed languages

* test unchanged restrictions
2023-12-05 11:12:01 +00:00
Elio Bischof
60688757fa
test(postgres): always test against latest release (#6972)
* test(postgres): always test against latest

* Update CONTRIBUTING.md

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* Update internal/integration/config/docker-compose.yaml

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-11-24 13:56:58 +00:00
Tim Möhlmann
72bc3ffe14
fix(oidc): add missing fields to introspection (#6967)
during QA I found some user info and org ID was missing.
This change adds those missing fields.
2023-11-23 16:17:50 +02:00
Elio Bischof
76fe032b5f
feat: option to disallow public org registration (#6917)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* lint
2023-11-22 09:29:38 +00:00
Tim Möhlmann
ba9b807854
perf(oidc): optimize the introspection endpoint (#6909)
* get key by id and cache them

* userinfo from events for v2 tokens

* improve keyset caching

* concurrent token and client checks

* client and project in single query

* logging and otel

* drop owner_removed column on apps and authN tables

* userinfo and project roles in go routines

* get  oidc user info from projections and add actions

* add avatar URL

* some cleanup

* pull oidc work branch

* remove storage from server

* add config flag for experimental introspection

* legacy introspection flag

* drop owner_removed column on user projections

* drop owner_removed column on useer_metadata

* query userinfo unit test

* query introspection client test

* add user_grants to the userinfo query

* handle PAT scopes

* bring triggers back

* test instance keys query

* add userinfo unit tests

* unit test keys

* go mod tidy

* solve some bugs

* fix missing preferred login name

* do not run triggers in go routines, they seem to deadlock

* initialize the trigger handlers late with a sync.OnceValue

* Revert "do not run triggers in go routines, they seem to deadlock"

This reverts commit 2a03da2127.

* add missing translations

* chore: update go version for linting

* pin oidc version

* parse a global time location for query test

* fix linter complains

* upgrade go lint

* fix more linting issues

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-11-21 13:11:38 +01:00
Livio Spring
f3b8a3aece
feat: add possibility to set an expiration to a session (#6851)
* add lifetime to session api

* extend session with lifetime

* check session token expiration

* fix typo

* integration test to check session token expiration

* integration test to check session token expiration

* i18n

* cleanup

* improve tests

* prevent negative lifetime

* fix error message

* fix lifetime check
2023-11-06 10:48:28 +01:00
adlerhurst
bd23a7a56f merge main into next 2023-10-19 12:34:00 +02:00
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Stefan Benz
15fd3045e0
feat: add SAML as identity provider (#6454)
* feat: first implementation for saml sp

* fix: add command side instance and org for saml provider

* fix: add query side instance and org for saml provider

* fix: request handling in event and retrieval of finished intent

* fix: add review changes and integration tests

* fix: add integration tests for saml idp

* fix: correct unit tests with review changes

* fix: add saml session unit test

* fix: add saml session unit test

* fix: add saml session unit test

* fix: changes from review

* fix: changes from review

* fix: proto build error

* fix: proto build error

* fix: proto build error

* fix: proto require metadata oneof

* fix: login with saml provider

* fix: integration test for saml assertion

* lint client.go

* fix json tag

* fix: linting

* fix import

* fix: linting

* fix saml idp query

* fix: linting

* lint: try all issues

* revert linting config

* fix: add regenerate endpoints

* fix: translations

* fix mk.yaml

* ignore acs path for user agent cookie

* fix: add AuthFromProvider test for saml

* fix: integration test for saml retrieve information

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-09-29 11:26:14 +02:00
Elio Bischof
1a49b7d298
perf: project quotas and usages (#6441)
* project quota added

* project quota removed

* add periods table

* make log record generic

* accumulate usage

* query usage

* count action run seconds

* fix filter in ReportQuotaUsage

* fix existing tests

* fix logstore tests

* fix typo

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* move notifications into debouncer and improve limit querying

* cleanup

* comment

* fix: add quota unit tests command side

* fix remaining quota usage query

* implement InmemLogStorage

* cleanup and linting

* improve test

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* action notifications and fixes for notifications query

* revert console prefix

* fix: add quota unit tests command side

* fix: add quota integration tests

* improve accountable requests

* improve accountable requests

* fix: add quota integration tests

* fix: add quota integration tests

* fix: add quota integration tests

* comment

* remove ability to store logs in db and other changes requested from review

* changes requested from review

* changes requested from review

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* tests: fix quotas integration tests

* improve incrementUsageStatement

* linting

* fix: delete e2e tests as intergation tests cover functionality

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* backup

* fix conflict

* create rc

* create prerelease

* remove issue release labeling

* fix tracing

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-09-15 16:58:45 +02:00
Livio Spring
be81570fb5
feat(api): move resource apis to beta (#6530)
Moves UserService, SessionService, SettingsService and OIDCService to beta state. This includes gRPC and HTTP path changes.
2023-09-13 12:43:01 +00:00
Livio Spring
a9fb2a6e5c
fix(api): naming cleanup in user and session service (#6379)
* fix(api): rename first and last name to given and family name, intent to idp_intent, remove _ actions

* fix merge

* fully rename intent to idp intent in api

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-08-22 10:05:45 +00:00
Stefan Benz
52f68f8db8
feat: add ldap external idp to login api (#5938)
* fix: handling of ldap login through separate endpoint

* fix: handling of ldap login through separate endpoint

* fix: handling of ldap login through separate endpoint

* fix: successful intent for ldap

* fix: successful intent for ldap

* fix: successful intent for ldap

* fix: add changes from code review

* fix: remove set intent credentials and handle ldap errors

* fix: remove set intent credentials and handle ldap errors

* refactor into separate methods and fix merge

* remove mocks

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-16 11:29:57 +00:00
Tim Möhlmann
0017542aa2
feat(api/v2): implement TOTP session check (#6362)
* feat(api/v2): implement TOTP session check

* add integration test

* correct typo in projection test

* fix event type typos

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-15 09:50:42 +00:00
Tim Möhlmann
86af67d1be
feat(api/v2): implement U2F session check (#6339) 2023-08-11 15:36:18 +00:00
Livio Spring
372755bddd
feat(api): add organisation service (#6340)
* setup org with multiple admins

* tests

* add missing proto

* remove machine users (for now)

* update tests with idp case

* fix package

* organisation -> organization

* fix test
2023-08-11 14:19:14 +00:00
Stefan Benz
ef012d0081
feat: user v2 phone verification (#6309)
* feat: add phone change and code verification for user v2 api

* feat: add phone change and code verification for user v2 api

* fix: add ignored phone.proto

* fix: integration tests

* Update proto/zitadel/user/v2alpha/user_service.proto

* Update idp_template.go

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-03 06:42:59 +02:00
Livio Spring
a1942ecdaa
feat(api): add and remove OTP (SMS and email) (#6295)
* refactor: rename otp to totp

* feat: add otp sms and email

* implement tests
2023-08-02 18:57:53 +02:00
Livio Spring
59f3c328ec
feat(OIDC): add support for end_session for V2 tokens (#6226)
This PR adds support for the OIDC end_session_endpoint for V2 tokens. Sending an id_token_hint as parameter will directly terminate the underlying (SSO) session and all its tokens. Without this param, the user will be redirected to the Login UI, where he will able to choose if to logout.
2023-07-19 13:17:39 +02:00
Livio Spring
80961125a7
feat(API): support V2 token and session token usage (#6180)
This PR adds support for userinfo and introspection of V2 tokens. Further V2 access tokens and session tokens can be used for authentication on the ZITADEL API (like the current access tokens).
2023-07-14 11:16:16 +00:00
Livio Spring
14b8cf4894
feat(api): add OIDC session service (#6157)
This PR starts the OIDC implementation for the API V2 including the Implicit and Code Flow.


Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-07-10 13:27:00 +00:00
Livio Spring
6319fdda9e
fix: add scope profile to PAT (#6154)
Co-authored-by: Fabi <fabienne@zitadel.com>
2023-07-07 17:31:42 +02:00
Elio Bischof
bb756482c7
feat: push telemetry (#6027)
* document analytics config

* rework configuration and docs

* describe HandleActiveInstances better

* describe active instances on quotas better

* only projected events are considered

* cleanup

* describe changes at runtime

* push milestones

* stop tracking events

* calculate and push 4 in 6 milestones

* reduce milestone pushed

* remove docs

* fix scheduled pseudo event projection

* push 5 in 6 milestones

* push 6 in 6 milestones

* ignore client ids

* fix text array contains

* push human readable milestone type

* statement unit tests

* improve dev and db performance

* organize imports

* cleanup

* organize imports

* test projection

* check rows.Err()

* test search query

* pass linting

* review

* test 4 milestones

* simplify milestone by instance ids query

* use type NamespacedCondition

* cleanup

* lint

* lint

* dont overwrite original error

* no opt-in in examples

* cleanup

* prerelease

* enable request headers

* make limit configurable

* review fixes

* only requeue special handlers secondly

* include integration tests

* Revert "include integration tests"

This reverts commit 96db9504ec.

* pass reducers

* test handlers

* fix unit test

* feat: increment version

* lint

* remove prerelease

* fix integration tests
2023-07-06 08:38:13 +02:00
Livio Spring
bd5defa96a
fix: provide domain in session, passkey and u2f (#6097)
This fix provides a possibility to pass a domain on the session, which
will be used (as rpID) to create a passkey / u2f assertion and
attestation. This is useful in cases where the login UI is served under
a different domain / origin than the ZITADEL API.
2023-06-27 14:36:07 +02:00
Stefan Benz
1b5d6ce89e
feat: session checks with intent (#6031)
* feat: session checks with intent

* feat: session checks with intent

* fix: integration tests for intent session

* fix: integration tests for intent session

* fix merge

* fix: integration tests for intent session

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-06-21 14:06:18 +00:00
Livio Spring
7046194530
feat(api): list authentication method types in user api v2 (#6058) 2023-06-20 16:23:28 +00:00
Tim Möhlmann
f456168a74
feat: session v2 passkey authentication (#5952) 2023-06-07 17:28:42 +02:00
Tim Möhlmann
5f0c1b5290 resolve comments 2023-05-03 15:10:27 +02:00
Tim Möhlmann
c839cb3ce0 tie loose ends, documentation 2023-05-02 19:24:24 +03:00
Tim Möhlmann
4b7f5ae186 AddHumanUser tests 2023-04-28 14:39:53 +03:00
Tim Möhlmann
596900aba6 nolint contextcheck in NewTester 2023-04-26 20:49:33 +03:00
Tim Möhlmann
90ba3a8d92 poll on test start 2023-04-26 19:54:47 +03:00
Tim Möhlmann
79084089ea add github action for integration tests 2023-04-25 18:11:04 +03:00
Tim Möhlmann
a22b58f1c0 simple test of a health endpoint 2023-04-25 15:04:35 +03:00