183 Commits

Author SHA1 Message Date
Livio Spring
7015d347e1
fix(query): user performance (#7081)
* fix(query): user performance (#6537)

* start user by id

* ignore debug bin

* use new user by id

* new sql

* fix(sql): replace STRING with text for psql compatabilit

* some changes

* fix: correct user queries

* fix tests

* unify sql statements

* use specific get user methods

* search login name case insensitive

* refactor: optimise user statements

* add index

* fix queries

* fix: correct domain segregation

* return all login names

* fix queries

* improve readability

* query should be correct now

* cleanup statements

* fix username / loginname handling

* fix: psql doesn't support create view if not exists

* fix: create pre-release

* ignore release comments

* add lower fields

* fix: always to lower

* update to latest projection

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>

(cherry picked from commit ddbea119f1c49845a3e364fa6dc565ab3180989a)

* update projection versions

* trigger workflow

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-12-14 11:17:35 +00:00
Tim Möhlmann
2f91679623
chore(Makefile): add go generate target (#6944)
This change adds a core_generate_all make target.
It installs the required tools and runs generate on the complete project.

`golang/mock` is no longer maintained and a fork is available
from the Uber folks. So the latter is used as tool.
All the mock files have been regenerated and are part of the PR.

The obsolete `tools` directory has been removed,
as all the tools are now part of specific make targets.

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-11-22 10:56:43 +00:00
Elio Bischof
76fe032b5f
feat: option to disallow public org registration (#6917)
* feat: return 404 or 409 if org reg disallowed

* fix: system limit permissions

* feat: add iam limits api

* feat: disallow public org registrations on default instance

* add integration test

* test: integration

* fix test

* docs: describe public org registrations

* avoid updating docs deps

* fix system limits integration test

* silence integration tests

* fix linting

* ignore strange linter complaints

* review

* improve reset properties naming

* redefine the api

* use restrictions aggregate

* test query

* simplify and test projection

* test commands

* fix unit tests

* move integration test

* support restrictions on default instance

* also test GetRestrictions

* self review

* lint

* abstract away resource owner

* fix tests

* lint
2023-11-22 09:29:38 +00:00
Tim Möhlmann
ba9b807854
perf(oidc): optimize the introspection endpoint (#6909)
* get key by id and cache them

* userinfo from events for v2 tokens

* improve keyset caching

* concurrent token and client checks

* client and project in single query

* logging and otel

* drop owner_removed column on apps and authN tables

* userinfo and project roles in go routines

* get  oidc user info from projections and add actions

* add avatar URL

* some cleanup

* pull oidc work branch

* remove storage from server

* add config flag for experimental introspection

* legacy introspection flag

* drop owner_removed column on user projections

* drop owner_removed column on useer_metadata

* query userinfo unit test

* query introspection client test

* add user_grants to the userinfo query

* handle PAT scopes

* bring triggers back

* test instance keys query

* add userinfo unit tests

* unit test keys

* go mod tidy

* solve some bugs

* fix missing preferred login name

* do not run triggers in go routines, they seem to deadlock

* initialize the trigger handlers late with a sync.OnceValue

* Revert "do not run triggers in go routines, they seem to deadlock"

This reverts commit 2a03da2127b7dc74552ec25d4772282a82cc1cba.

* add missing translations

* chore: update go version for linting

* pin oidc version

* parse a global time location for query test

* fix linter complains

* upgrade go lint

* fix more linting issues

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-11-21 13:11:38 +01:00
Stefan Benz
0ec7a74877
perf: remove owner removed columns from projections for oidc (#6925)
* fix: remove owner removed columns from login names projection

* fix: remove owner removed columns from flow projection

* fix: remove owner removed columns from project, projectgrant and member projections

* fix: correct unit tests for session projection

* fix: correct unit tests for session projection
2023-11-20 17:21:08 +02:00
Livio Spring
2e8c3b5a53
feat: allow session deletion without session token (#6889)
* fix: add resource owner of user and change the one of session to instance

* use user resource owner from session projection

* fix session permission check

* integration tests and fixes

* update api docs
2023-11-16 07:35:50 +01:00
Livio Spring
f3b8a3aece
feat: add possibility to set an expiration to a session (#6851)
* add lifetime to session api

* extend session with lifetime

* check session token expiration

* fix typo

* integration test to check session token expiration

* integration test to check session token expiration

* i18n

* cleanup

* improve tests

* prevent negative lifetime

* fix error message

* fix lifetime check
2023-11-06 10:48:28 +01:00
Stefan Benz
cb7b50b513
feat: add attribute to only enable specific themes (#6798)
* feat: enable only specific themes in label policy

* feat: enable only specific themes in label policy

* feat: enable only specific themes in label policy

* feat: enable only specific themes in label policy

* add management in console

* pass enabledTheme

* render login ui based on enabled theme

* add in branding / settings service and name consistently

* update console to latest proto state

* fix console linting

* fix linting

* cleanup

* add translations

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-10-26 05:54:09 +00:00
Elio Bischof
385a55bd21
feat: limit audit trail (#6744)
* feat: enable limiting audit trail

* support AddExclusiveQuery

* fix invalid condition

* register event mappers

* fix NullDuration validity

* test query side for limits

* lint

* acceptance test audit trail limit

* fix acceptance test

* translate limits not found

* update tests

* fix linting

* add audit log retention to default instance

* fix tests

* update docs

* remove todo

* improve test name
2023-10-25 11:42:00 +00:00
adlerhurst
bd23a7a56f merge main into next 2023-10-19 12:34:00 +02:00
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Stefan Benz
15fd3045e0
feat: add SAML as identity provider (#6454)
* feat: first implementation for saml sp

* fix: add command side instance and org for saml provider

* fix: add query side instance and org for saml provider

* fix: request handling in event and retrieval of finished intent

* fix: add review changes and integration tests

* fix: add integration tests for saml idp

* fix: correct unit tests with review changes

* fix: add saml session unit test

* fix: add saml session unit test

* fix: add saml session unit test

* fix: changes from review

* fix: changes from review

* fix: proto build error

* fix: proto build error

* fix: proto build error

* fix: proto require metadata oneof

* fix: login with saml provider

* fix: integration test for saml assertion

* lint client.go

* fix json tag

* fix: linting

* fix import

* fix: linting

* fix saml idp query

* fix: linting

* lint: try all issues

* revert linting config

* fix: add regenerate endpoints

* fix: translations

* fix mk.yaml

* ignore acs path for user agent cookie

* fix: add AuthFromProvider test for saml

* fix: integration test for saml retrieve information

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-09-29 11:26:14 +02:00
Elio Bischof
ae1af6bc8c
fix: set quotas (#6597)
* feat: set quotas

* fix: start new period on younger anchor

* cleanup e2e config

* fix set notifications

* lint

* test: fix quota projection tests

* fix add quota tests

* make quota fields nullable

* enable amount 0

* fix initial setup

* create a prerelease

* avoid success comments

* fix quota projection primary key

* Revert "fix quota projection primary key"

This reverts commit e72f4d7fa17d03d36493912168490350a320e04f.

* simplify write model

* fix aggregate id

* avoid push without changes

* test set quota lifecycle

* test set quota mutations

* fix quota unit test

* fix: quotas

* test quota.set event projection

* use SetQuota in integration tests

* fix: release quotas 3

* reset releaserc

* fix comment

* test notification order doesn't matter

* test notification order doesn't matter

* test with unmarshalled events

* test with unmarshalled events
2023-09-22 09:37:16 +00:00
Elio Bischof
1a49b7d298
perf: project quotas and usages (#6441)
* project quota added

* project quota removed

* add periods table

* make log record generic

* accumulate usage

* query usage

* count action run seconds

* fix filter in ReportQuotaUsage

* fix existing tests

* fix logstore tests

* fix typo

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* move notifications into debouncer and improve limit querying

* cleanup

* comment

* fix: add quota unit tests command side

* fix remaining quota usage query

* implement InmemLogStorage

* cleanup and linting

* improve test

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* action notifications and fixes for notifications query

* revert console prefix

* fix: add quota unit tests command side

* fix: add quota integration tests

* improve accountable requests

* improve accountable requests

* fix: add quota integration tests

* fix: add quota integration tests

* fix: add quota integration tests

* comment

* remove ability to store logs in db and other changes requested from review

* changes requested from review

* changes requested from review

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* tests: fix quotas integration tests

* improve incrementUsageStatement

* linting

* fix: delete e2e tests as intergation tests cover functionality

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* backup

* fix conflict

* create rc

* create prerelease

* remove issue release labeling

* fix tracing

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-09-15 16:58:45 +02:00
Livio Spring
62d679e553
fix: handle password data correct on user creation (#6515) 2023-09-07 14:06:11 +00:00
Miguel Cabrerizo
dd80109969
feat: delete organizations (#6083)
* feat: delete organizations

* feat: tests and delete all that depends on org

* fix: grpc delete description

* fix: get back reduce OrgRemovedEvent

* fix: add @muhlemmer review suggestions

* fix: new e2e for add/delete org
2023-09-07 04:54:51 +00:00
Livio Spring
e17b49e4ca
feat: add apple as idp (#6442)
* feat: manage apple idp

* handle apple idp callback

* add tests for provider

* basic console implementation

* implement flow for login UI and add logos / styling

* tests

* cleanup

* add upload button

* begin i18n

* apple logo positioning, file upload component

* fix add apple instance idp

* add missing apple logos for login

* update to go 1.21

* fix slice compare

* revert permission changes

* concrete error messages

* translate login apple logo -y-2px

* change form parsing

* sign in button

* fix tests

* lint console

---------

Co-authored-by: peintnermax <max@caos.ch>
2023-08-31 08:39:16 +02:00
Miguel Cabrerizo
fd00ac533a
feat: add reply-to header in email notification (#6393)
* feat: add reply-to header to smtp messages

* fix: grpc reply_to_address min 0 and js var name

* fix: add missing translations

* fix merge and linting

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-29 09:08:24 +02:00
Livio Spring
bb40e173bd
feat(api): add otp (sms and email) checks in session api (#6422)
* feat: add otp (sms and email) checks in session api

* implement sending

* fix tests

* add tests

* add integration tests

* fix merge main and add tests

* put default OTP Email url into config

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-08-24 09:41:52 +00:00
Livio Spring
8b44794c75
fix: delete SMTP correctly (#6391) 2023-08-18 14:22:57 +02:00
Livio Spring
7c494fd219
feat(login): add OTP (email and sms) (#6353)
* feat: login with otp

* fix(i18n): japanese translation

* add missing files

* fix provider change

* add event types translations to en

* add tests

* resourceOwner

* remove unused handler

* fix: secret generators and add comments

* add setup step

* rename

* linting

* fix setup

* improve otp handling

* fix autocomplete

* translations for login and notifications

* translations for event types

* changes from review

* check selected mfa type
2023-08-15 12:47:05 +00:00
Tim Möhlmann
0017542aa2
feat(api/v2): implement TOTP session check (#6362)
* feat(api/v2): implement TOTP session check

* add integration test

* correct typo in projection test

* fix event type typos

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-15 09:50:42 +00:00
Tim Möhlmann
86af67d1be
feat(api/v2): implement U2F session check (#6339) 2023-08-11 15:36:18 +00:00
Elio Bischof
343a9428b3
feat: SMS and email OTP texts (#6281)
* manage 2 custom texts proto

* implement methods

* default texts

* console

* improve translations

* lint

* test: fix e2e timeout

* fix translations

* add missing console translations

* remove unused text parts

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-09 05:49:12 +00:00
Livio Spring
45262e6829
fix: migrate external id of federated users (#6312)
* feat: migrate external id

* implement tests and some renaming

* fix projection

* cleanup

* i18n

* fix event type

* handle migration for new services as well

* typo
2023-08-04 11:35:36 +02:00
Livio Spring
a1942ecdaa
feat(api): add and remove OTP (SMS and email) (#6295)
* refactor: rename otp to totp

* feat: add otp sms and email

* implement tests
2023-08-02 18:57:53 +02:00
Elio Bischof
31ec1d83b9
feat: enable otp email and sms (#6260)
* feat: enable otp email and sms

* feat: enable otp factors in login settings

* remove tests without value

* translate second factors

* don't add new factors yet

* add comment

* add factors to docs

* backward compatible settings api

* compile tests

* add available 2fa types

* test: add mapping tests

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-28 07:39:30 +02:00
Livio Spring
fed15574f6
feat: allow to force MFA local only (#6234)
This PR adds an option to the LoginPolicy to "Force MFA for local users", so that users authenticated through an IDP must not configure (and verify) an MFA.
2023-07-20 04:06:16 +00:00
Livio Spring
14b8cf4894
feat(api): add OIDC session service (#6157)
This PR starts the OIDC implementation for the API V2 including the Implicit and Code Flow.


Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-07-10 13:27:00 +00:00
Elio Bischof
bb756482c7
feat: push telemetry (#6027)
* document analytics config

* rework configuration and docs

* describe HandleActiveInstances better

* describe active instances on quotas better

* only projected events are considered

* cleanup

* describe changes at runtime

* push milestones

* stop tracking events

* calculate and push 4 in 6 milestones

* reduce milestone pushed

* remove docs

* fix scheduled pseudo event projection

* push 5 in 6 milestones

* push 6 in 6 milestones

* ignore client ids

* fix text array contains

* push human readable milestone type

* statement unit tests

* improve dev and db performance

* organize imports

* cleanup

* organize imports

* test projection

* check rows.Err()

* test search query

* pass linting

* review

* test 4 milestones

* simplify milestone by instance ids query

* use type NamespacedCondition

* cleanup

* lint

* lint

* dont overwrite original error

* no opt-in in examples

* cleanup

* prerelease

* enable request headers

* make limit configurable

* review fixes

* only requeue special handlers secondly

* include integration tests

* Revert "include integration tests"

This reverts commit 96db9504ecdb4e73451f09554fd749cd7c27341f.

* pass reducers

* test handlers

* fix unit test

* feat: increment version

* lint

* remove prerelease

* fix integration tests
2023-07-06 08:38:13 +02:00
Livio Spring
bd5defa96a
fix: provide domain in session, passkey and u2f (#6097)
This fix provides a possibility to pass a domain on the session, which
will be used (as rpID) to create a passkey / u2f assertion and
attestation. This is useful in cases where the login UI is served under
a different domain / origin than the ZITADEL API.
2023-06-27 14:36:07 +02:00
Stefan Benz
1b5d6ce89e
feat: session checks with intent (#6031)
* feat: session checks with intent

* feat: session checks with intent

* fix: integration tests for intent session

* fix: integration tests for intent session

* fix merge

* fix: integration tests for intent session

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-06-21 14:06:18 +00:00
Livio Spring
82e7333169
feat(api): add password reset and change to user service (#6036)
* feat(api): add password reset and change to user service

* integration tests

* invalidate password check after password change

* handle notification type

* fix proto
2023-06-20 17:34:06 +02:00
Stefan Benz
66e639b5ad
fix: handling of org idp migrations and google events (#5992) 2023-06-08 14:08:13 +00:00
Stefan Benz
5562ee94a6
feat: migrate external idp to other types (#5984)
* feat: migrate instance oidc to azureAD

* feat: migrate instance oidc to azureAD

* feat: migrate org oidc to azureAD

* feat: migrate oidc to google

* fix: correct idp writemodels

* fix: review changes
2023-06-08 00:50:53 +02:00
Tim Möhlmann
f456168a74
feat: session v2 passkey authentication (#5952) 2023-06-07 17:28:42 +02:00
Livio Spring
c2cb84cd24
feat(api): new session service (#5801)
* backup new protoc plugin

* backup

* session

* backup

* initial implementation

* change to specific events

* implement tests

* cleanup

* refactor: use new protoc plugin for api v2

* change package

* simplify code

* cleanup

* cleanup

* fix merge

* start queries

* fix tests

* improve returned values

* add token to projection

* tests

* test db map

* update query

* permission checks

* fix tests and linting

* rework token creation

* i18n

* refactor token check and fix tests

* session to PB test

* request to query tests

* cleanup proto

* test user check

* add comment

* simplify database map type

* Update docs/docs/guides/integrate/access-zitadel-system-api.md

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix test

* cleanup

* docs

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-05-05 15:34:53 +00:00
Tim Möhlmann
5819924275
feat: device authorization RFC 8628 (#5646)
* device auth: implement the write events

* add grant type device code

* fix(init): check if default value implements stringer

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-04-19 08:46:02 +00:00
Livio Spring
429a91518b
fix: remove idp templates when using old / deprecated delete method (#5685)
* fix: remove idp templates when using old / deprecated delete method

* fix: projection version
2023-04-17 08:20:49 +00:00
Livio Spring
8bf36301ed
feat: allow skip of success page for native apps (#5627)
add possibility to return to callback directly after login without rendering the successful login page
2023-04-11 15:07:32 +00:00
Livio Spring
ea9223a2b0
fix: remove instance IDPs correctly from org policies (#5609)
Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-04-05 11:18:00 +00:00
Elio Bischof
887e2f474d
fix: use fixed active instances duration (#5567)
* fix: use fixed active instances duration

* fix active instances tests

* fix syntax error

* run pipeline

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-30 13:01:27 +02:00
Elio Bischof
cccccd005c
feat: call webhooks at least once (#5454)
* feat: call webhooks at least once

* self review

* feat: improve notification observability

* feat: add notification tracing

* test(e2e): test at-least-once webhook delivery

* fix webhook notifications

* dedicated quota notifications handler

* fix linting

* fix e2e test

* wait less in e2e test

* fix: don't ignore failed events in handlers

* fix: don't ignore failed events in handlers

* faster requeues

* question

* fix retries

* fix retries

* retry

* don't instance ids query

* revert handler_projection

* statements can be nil

* cleanup

* make unit tests pass

* add comments

* add comments

* lint

* spool only active instances

* feat(config): handle inactive instances

* customizable HandleInactiveInstances

* call inactive instances quota webhooks

* test: handling with and w/o inactive instances

* omit retrying noop statements

* docs: describe projection options

* enable global handling of inactive instances

* self review

* requeue quota notifications every 5m

* remove caos_errors reference

* fix comment styles

* make handlers package flat

* fix linting

* fix repeating quota notifications

* test with more usage

* debug log channel init failures
2023-03-28 22:09:06 +00:00
Miguel Cabrerizo
1b9cea0e0c
feat: add Help/Support e-mail for instance/org (#5445)
feat: help and support email in privacy policy
2023-03-28 21:36:52 +02:00
Elio Bischof
62bd606593
feat: enable handling projections for inactive instances (#5523)
* fix: don't ignore failed events in handlers

* question

* fix retries

* don't instance ids query

* statements can be nil

* make unit tests pass

* add comments

* spool only active instances

* feat(config): handle inactive instances

* customizable HandleInactiveInstances

* test: handling with and w/o inactive instances

* docs: describe projection options

* enable global handling of inactive instances

* accept NowFunc, not Clock interface

* add comment about stringer usage

* remove enum stringer implementations

* fix enum format types

* Update internal/eventstore/repository/mock/repository.mock.impl.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-27 12:34:01 +00:00
Stefan Benz
41ff0bbc63
feat: ldap provider login (#5448)
Add the logic to configure and use LDAP provider as an external IDP with a dedicated login GUI.
2023-03-24 15:18:56 +00:00
Livio Spring
fca6dd9613
fix: reduce correct type for jwt idps (#5519) 2023-03-22 15:46:59 +00:00
Livio Spring
1896f13952
fix: use idToken for mapping when using old configs (#5458)
* fix: use idToken for mapping when using old configs

* fix events and add tests
2023-03-16 16:47:22 +01:00
Livio Spring
5a307afe62
feat: add azure provider templates (#5441)
Adds possibility to manage and use Microsoft Azure template based providers
2023-03-15 07:48:37 +01:00
Elio Bischof
e00cc187fa
fix: make user creation errors helpful (#5382)
* fix: make user creation errors helpful

* fix linting and unit testing errors

* fix linting

* make zitadel config reusable

* fix human validations

* translate ssr errors

* make zitadel config reusable

* cover more translations for ssr

* handle email validation message centrally

* fix unit tests

* fix linting

* align signatures

* use more precise wording

* handle phone validation message centrally

* fix: return specific profile errors

* docs: edit comments

* fix unit tests

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-14 19:20:38 +00:00