* fix: poc outlook.com now works login auth
* fix: remove port arg from smtpAuth
* fix: add outlook provider and custom email placeholder
* fix: minor typo in contributing docs
* fix: use zerrors package
* fix: typo for idp and smtp providers
---------
Co-authored-by: Max Peintner <max@caos.ch>
* docs: translate missing event types
* fix: wrong example in api docs
* Update internal/static/i18n/cs.yaml
Co-authored-by: Livio Spring <livio.a@gmail.com>
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* Allow using a local RSA key for machine keys
* Add check for key validity
* Fix naming error
* docs: provide translations of invalid key
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
We are trying to reproduce a few 500 responses we observe on zitadel cloud's token endpoint.
As in the past these were caused by wrongly encoded or encrypted refresh tokens, I created a integration test which tries to reproduce 500 errors by sending invalid refresh tokens.
The added test does not reproduce 500s, all returned errors are in the 400 range as they should. However, as the test is already written, we might as well include them.
Related to #7765
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix(oidc): roles in userinfo for client credentials token
When tokens were obtained using the client credentials grant,
with audience and role scopes, userinfo would not return the role claims. This had multiple causes:
1. There is no auth request flow, so for legacy userinfo project data was never attached to the token
2. For optimized userinfo, there is no client ID that maps to an application. The client ID for client credentials is the machine user's name. There we can't obtain a project ID. When the project ID remained empty, we always ignored the roleAudience.
This PR fixes situation 2, by always taking the roleAudience into account, even when the projectID is empty. The code responsible for the bug is also refactored to be more readable and understandable, including additional godoc.
The fix only applies to the optimized userinfo code introduced in #7706 and released in v2.50 (currently in RC). Therefore it can't be back-ported to earlier versions.
Fixes#6662
* chore(deps): update all go deps (#7764)
This change updates all go modules, including oidc, a major version of go-jose and the go 1.22 release.
* Revert "chore(deps): update all go deps" (#7772)
Revert "chore(deps): update all go deps (#7764)"
This reverts commit 6893e7d060.
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
This fixes the projection of events that have a null audience or scope.
As audience was added in v2.50, legacy events do not have an audience, this made replay of the old events not possible after an upgrade.
* feat: smtp templates poc
* feat: add isActive & ProviderType to SMTP backend
* feat: change providertype to uint32 and fix tests
* feat: minimal smtp provider component
* feat: woking on diiferent providers
* feat: keep working on providers
* feat: initial stepper for new provider
* fix: settings list and working on stepper
* feat: step 1 and 2 form inputs
* feat: starter for smtp test step
* fix: misspelled SMPT
* fix: remove tests for now
* feat: add tls toggle remove old google provider
* feat: working on add smtp and table
* fix: duplicated identifiers
* fix: settings list
* fix: add missing smtp config properties
* fix: add configID to smtp config table
* fix: working on listproviders
* feat: working in listSMTPConfigs
* fix: add count to listsmtpconfigs
* fix: getting empty results from listSMTPConfigs
* feat: table now shows real data
* fix: remaining styles for smtp-table
* fix: remove old notification-smtp-provider-component
* feat: delete smtp configuration
* feat: deactivate smtp config
* feat: replace isActive with state for smtp config
* feat: activate smtp config
* fix: remaining errors after main merge
* fix: list smtp providers panic and material mdc
* feat: refactor to only one provider component
* feat: current provider details view
* fix: refactor AddSMTPConfig and ChangeSMTPConfig
* fix: smtp config reduce issue
* fix: recover domain in NewIAMSMTPConfigWriteModel
* fix: add code needed by SetUpInstance
* fix: go tests and warn about passing context to InstanceAggregateFromWriteModel
* fix: i18n and add missing trans for fr, it, zh
* fix: add e2e tests
* docs: add smtp templates
* fix: remove provider_type, add description
* fix: remaining error from merge main
* fix: add @stebenz change for primary key
* fix: inactive placed after removed to prevent deleted configs to show as inactive
* fix: smtp provider id can be empty (migrated)
* feat: add mailchimp transactional template
* feat: add Brevo (Sendinblue) template
* feat: change brevo logo, add color to tls icon
* fix: queries use resourceowner, id must not be empty
* fix: deal with old smtp settings and tests
* fix: resourceOwner is the instanceID
* fix: remove aggregate_id, rename SMTPConfigByAggregateID with SMTPConfigActive
* fix: add tests for multiple configs with different IDs
* fix: conflict
* fix: remove notification-smtp-provider
* fix: add @peintnermax suggestions, rename module and fix e2e tests
* fix: remove material legacy modules
* fix: remove ctx as parameter for InstanceAggregateFromWriteModel
* fix: add Id to SMTPConfigToPb
* fix: change InstanceAggregateFromWriteModel to avoid linter errors
* fix import
* rm unused package-lock
* update yarn lock
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
* init auto linking
* prompt handling
* working
* translations
* console
* fixes
* unify
* custom texts
* fix tests
* linting
* fix check of existing user
* fix bg translation
* set unspecified as default in the form
* fix: add action v2 execution to features
* fix: add action v2 execution to features
* fix: add action v2 execution to features
* fix: update internal/command/instance_features_model.go
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: merge back main
* fix: merge back main
* fix: rename feature and service
* fix: rename feature and service
* fix: review changes
* fix: review changes
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* feat(oidc): optimize the userinfo endpoint
* store project ID in the access token
* query for projectID if not in token
* add scope based tests
* Revert "store project ID in the access token"
This reverts commit 5f0262f239.
* query project role assertion
* use project role assertion setting to return roles
* workaround eventual consistency and handle PAT
* do not append empty project id
* fix(oidc): return bad request for base64 errors
We've recently noticed an increased amount of 500: internal server error status returns on zitadel cloud.
The source of these errors appear to be erroneous input in fields that are supposed to be bas64 formatted.
```
time=2024-04-08T14:05:47.600Z level=ERROR msg="request error" oidc_error.parent="ID=OIDC-AhX2u Message=Errors.Internal Parent=(illegal base64 data at input byte 8)" oidc_error.description=Errors.Internal oidc_error.type=server_error status_code=500
```
Within the possible code paths of the token endpoint there are a couple of uses of base64.Encoding.DecodeString of which a returned error was not properly wrapped, but returned as-is.
This causes the oidc error handler to return a 500 with the `OIDC-AhX2u` ID.
We were not able to pinpoint the exact errors that are happening to any one call of `DecodeString`.
This fix wraps all errors from `DecodeString` so that proper 400: bad request is returned with information about the error. Each wrapper now has an unique error ID, so that logs will contain the source of the error as well.
This bug was reported internally by the ops team.
* catch op.ErrInvalidRefreshToken
* update url to redirect to name change url
* Update register_option.html
* Update register_option.html
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix(idp): do not call userinfo when mapping from ID token is configured
This change prevents the call of the Userinfo endpoint of a OIDC IDP if the IDP is configured to use the ID token for user information instead.
A unit test has been added to confirm the corrected behavior.
Closes#7373
* video for e2e
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
chore(fmt): run gci on complete project
Fix global import formatting in go code by running the `gci` command. This allows us to just use the command directly, instead of fixing the import order manually for the linter, on each PR.
Co-authored-by: Elio Bischof <elio@zitadel.com>
feat(db): wrap BeginTx in spans to get acquire metrics
This changes adds a span around most db.BeginTx calls so we can get tracings about the connection pool acquire process.
This might help us pinpoint why sometimes some query package traces show longer execution times, while this was not reflected on database side execution times.
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute
* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute
* fix: add resource owner as query for user v2 ListUsers and clean up deprecated attribute
* fix: review changes
* fix: review changes
* fix: review changes
* fix: review changes
* fix: add password change required to user v2 get and list
* fix: update unit tests for query side with new column and projection
* fix: change projection in setup steps
* fix: change projection in setup steps
* fix: remove setup step 25
* fix: add password_change_required into ListUsers response
* fix: correct SetUserPassword parameters
* fix: rollback to change setup instead of projection directly
* fix: rollback to change setup instead of projection directly
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: remove resourceowner read from context in user v2 api
* fix: lint
* fix: remove orgID in addIDPLink
* fix: remove comment as unnecessary
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* chore: use pgx v5
* chore: update go version
* remove direct pq dependency
* remove unnecessary type
* scan test
* map scanner
* converter
* uint8 number array
* duration
* most unit tests work
* unit tests work
* chore: coverage
* go 1.21
* linting
* int64 gopfertammi
* retry go 1.22
* retry go 1.22
* revert to go v1.21.5
* update go toolchain to 1.21.8
* go 1.21.8
* remove test flag
* go 1.21.5
* linting
* update toolchain
* use correct array
* use correct array
* add byte array
* correct value
* correct error message
* go 1.21 compatible
* docs(oidc): token exchange guide
This change adds a token exchange guide which includes "simple" and impersonation examples.
The endpoint, claims and grant type documentation also has been amended with token exchange specifics.
* solve suggestions
* fix impersonated event type
* add link to event store concept
* fix links build error
* add to sidebar and update some info boxes
This PR extends the user schema service (V3 API) with the possibility to ListUserSchemas and GetUserSchemaByID.
The previously started guide is extended to demonstrate how to retrieve the schema(s) and notes the generated revision property.
fix(oidc): define audience inside auth request instead off token creation
When using the v1 OIDC Code flow, tokens would not carry the correct audience when returned as JWT. This applies to access tokens as JWT and ID tokens.
Introspection would still show the correct audience.
This happened because project audience was appended at token creation time. This stored the appended audience, used later in introspection or token refresh. However, the OIDC library still only had a view of the original auth request with the original audience.
When signing JWTs it would use this outdated information.
This change moves audience modifications to the auth request creation. This is was already the way it was done for v2 login and now v1 follows the same method.
Co-authored-by: Livio Spring <livio.a@gmail.com>
Add organizationID as query for ListUsers and clean up the deprecated Organisation attributes in other queries.
This PR removes the following fields from API requests (user service v2):
organisation from AddHumanUser (deprecated some time ago, organization still exists)
organization from GetUserByID
A customer noted that the login by email was case-sensitive, which differs to the handling of the loginname.
This PR changes the email check to be case-insensitive (which it was already in same parts) and improve the search for this as well.
It was noticed multiple time (incl. customers) that the loginname is sometimes not rendered in the UI.
This PR fixes such an issue after registration of a new user from an IdP.
* add token exchange feature flag
* allow setting reason and actor to access tokens
* impersonation
* set token types and scopes in response
* upgrade oidc to working draft state
* fix tests
* audience and scope validation
* id toke and jwt as input
* return id tokens
* add grant type token exchange to app config
* add integration tests
* check and deny actors in api calls
* fix instance setting tests by triggering projection on write and cleanup
* insert sleep statements again
* solve linting issues
* add translations
* pin oidc v3.15.0
* resolve comments, add event translation
* fix refreshtoken test
* use ValidateAuthReqScopes from oidc
* apparently the linter can't make up its mind
* persist actor thru refresh tokens and check in tests
* remove unneeded triggers
feat: updated russian translations by native speaker with a user polite approach
Co-authored-by: Pavel Girilyuk <pavel.girilyuk@digitalchief.ru>
Co-authored-by: Fabi <fabienne@zitadel.com>
* feat: add projections and query side to executions and targets
* feat: add list and get endpoints for targets
* feat: add integration tests for query endpoints target and execution
* fix: linting
* fix: linting
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: remove position from list details
* feat: add ExpirationDate to MachineKey JSON detail
* fix: include time in expiration date column for machine keys table
* fix: show expiration date in ShowKeyDialog if available
* fix: add machine key expiration date note
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
This PR adds the functionality to manage user schemas through the new user schema service.
It includes the possibility to create a basic JSON schema and also provides a way on defining permissions (read, write) for owner and self context with an annotation.
Further annotations for OIDC claims and SAML attribute mappings will follow.
A guide on how to create a schema and assign permissions has been started. It will be extended though out the process of implementing the schema and users based on those.
Note:
This feature is in an early stage and therefore not enabled by default. To test it out, please enable the UserSchema feature flag on your instance / system though the feature service.
Add additional permission integration tests to the user v2 query endpoints including some fixes to correctly check the permissions after the data is known which you want to query.
* fix: projection reduce correction with unit tests
* fix: remove eventcout variable as not used anymore
* fix: add errors if resoureowner is not found in user grants reduce
fix(query): optimize instance by domain query
On zitadel cloud we noticed an increase in database CPU usage and slightly higher response times.
By analyzes we found that the instance by domain query was wrongly joining all instance_feature rows against all instances.
This PR adds an additional CTE to limit the join set to only the features that apply to the found instance.
The query was introduced with https://github.com/zitadel/zitadel/pull/7356 and part of the v2.47 release.
* fix: assign instance ID to aggregate ID when converting from v1 to v2 feature
This change fixes a mismatch between v1 and v2 aggregate IDs for instance feature events.
The old v1 used a random aggregate ID, while v2 uses the instance ID as aggregate ID.
The adapter was not correctly mapping, which resulted in the projections.instance_features table being filled with wrong instance IDs.
Closes#7501
* fix unit test
* fix: if device Code is selected Auth Code no longer a hard requirement
* fix: create device code app without auth code as grant type
* fix: don't show redirect uris in overview for devicecode app wizard
* feat: reload component when oidc app is updated
* fix: oidcapp is valid grantTypes cannot be empty
* fix: auth code not mandatory if refresh token and device code combined
* feat: improve instance not found error
* unit tests
* check if is templatable
* lint
* assert
* compile tests
* remove error templates
* link to instance not found page
* fmt
* cleanup
* lint
* partial work done
* test IAM membership roles
* org membership tests
* console :(, translations and docs
* fix integration test
* fix tests
* add EnableImpersonation to security policy API
* fix integration test timestamp checking
* add security policy tests and fix projections
* add impersonation setting in console
* add security settings to the settings v2 API
* fix typo
* move impersonation to instance
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(api): feature API proto definitions
* update proto based on discussion with @livio-a
* cleanup old feature flag stuff
* authz instance queries
* align defaults
* projection definitions
* define commands and event reducers
* implement system and instance setter APIs
* api getter implementation
* unit test repository package
* command unit tests
* unit test Get queries
* grpc converter unit tests
* migrate the V1 features
* migrate oidc to dynamic features
* projection unit test
* fix instance by host
* fix instance by id data type in sql
* fix linting errors
* add system projection test
* fix behavior inversion
* resolve proto file comments
* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event
* use write models and conditional set events
* system features integration tests
* instance features integration tests
* error on empty request
* documentation entry
* typo in feature.proto
* fix start unit tests
* solve linting error on key case switch
* remove system defaults after discussion with @eliobischof
* fix system feature projection
* resolve comments in defaults.yaml
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat: add events for execution
* feat: add events for execution and command side
* feat: add events for execution and command side
* feat: add api endpoints for set and delete executions with integration tests
* feat: add integration and unit tests and more existence checks
* feat: add integration and unit tests and more existence checks
* feat: unit tests for includes in executions
* feat: integration tests for includes in executions
* fix: linting
* fix: update internal/api/api.go
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: update internal/command/command.go
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: apply suggestions from code review
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: change api return
* fix: change aggregateID with prefix of execution type and add to documentation
* fix: change body in proto for documentation and correct linting
* fix: changed existing check to single query in separate writemodel
* fix: linter changes and list endpoints for conditions in executions
* fix: remove writemodel query on exeuction set as state before is irrelevant
* fix: testing for exists write models and correction
* fix: translations for errors and event types
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
This PR changes the domain / prefix of the user agent cookie from including the subdomain to the domain only and therefore changing the prefix from __Secure to __Host.
Note:
As the cookie is used to determine existing session on the login UI, applying the change will require end-users to start a new session on the next login, since the existing ones cannot be retrieved anymore.
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: apply suggestions from code review
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: review changes to remove resourceowner as parameters
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix(oidc): ignore public key expiry for ID Token hints
This splits the key sets used for access token and ID token hints.
ID Token hints should be able to be verified by with public keys that are already expired.
However, we do not want to change this behavior for Access Tokens,
where an error for an expired public key is still returned.
The public key cache is modified to purge public keys based on last use,
instead of expiry.
The cache is shared between both verifiers.
* resolve review comments
* pin oidc 3.11
Even though this is a feature it's released as fix so that we can back port to earlier revisions.
As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.
The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
* fix: add granted org info to user grants query response
* fix: show user info, tests and add columns to user grant
* fix: add check for org membership
* fix: typo in find logic
---------
Co-authored-by: Max Peintner <max@caos.ch>
* feat: add query endpoints for user v2 api
* fix: correct integration tests
* fix: correct linting
* fix: correct linting
* fix: comment out permission check on user get and list
* fix: permission check on user v2 query
* fix: merge back origin/main
* fix: add search query in user emails
* fix: reset count for SearchUser if users are removed due to permissions
* fix: reset count for SearchUser if users are removed due to permissions
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
* fix: change logic for usergrants projection with no selects
* fix: change logic for usergrants projection with one select
* fix: move resource owner select to single function
* fix: move resource owner select to single function
* fix: changes after merge
* fix: changes after merge
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix(setup): unmarshal of failed step
* fix(cleanup): cleanup all stuck states
* use lastRun for repeatable steps
* typo
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
fix(oidc); refresh token for device authorization
Due to a mis-alignment of OIDC interface and concrete implementations in zitadel, requesting a refresh token for device authorization would fail.
This change adds the possibility to to use the op.IDTokenRequest directly.
Also, the UserAgentID is dropped as required parameter, as devices do not have a user agent.
* feat: user service v2 remove user
* feat: user service v2 add user human
* feat: user service v2 change user human
* feat: user service v2 change user human unit tests
* feat: user service v2 reactivate, deactivate, lock, unlock user
* feat: user service v2 integration tests
* fix: merge back origin/main
* lint: linter corrections
* fix: move permission check for isVerfied and password change
* fix: add deprecated notices and other review comments
* fix: consistent naming in proto
* fix: errors package renaming
* fix: remove / delete user renaming in integration test
* fix: machine user status changes through user v2 api
* fix: linting changes
* fix: linting changes
* fix: changes from review
* fix: changes from review
* fix: changes from review
* fix: changes from review
* fix: changes from review
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix(db): add additional connection pool for projection spooling
* use correct connection pool for projections
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* cleanup todo
* pass id token details to oidc
* feat(oidc): id token for device authorization
This changes updates to the newest oidc version,
so the Device Authorization grant can return ID tokens when
the scope `openid` is set.
There is also some refactoring done, so that the eventstore can be
queried directly when polling for state.
The projection is cleaned up to a minimum with only data required for the login UI.
* try to be explicit wit hthe timezone to fix github
* pin oidc v3.8.0
* remove TBD entry
* start user by id
* ignore debug bin
* use new user by id
* new sql
* fix(sql): replace STRING with text for psql compatabilit
* some changes
* fix: correct user queries
* fix tests
* unify sql statements
* use specific get user methods
* search login name case insensitive
* refactor: optimise user statements
* add index
* fix queries
* fix: correct domain segregation
* return all login names
* fix queries
* improve readability
* query should be correct now
* cleanup statements
* fix username / loginname handling
* fix: psql doesn't support create view if not exists
* fix: create pre-release
* ignore release comments
* add lower fields
* fix: always to lower
* update to latest projection
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(console): add time range events filter
* deprecate creation_date, use oneof filter
* use range or from
* implement api
* fix timestamp format
* translate
* styles
* lint
* integration tests
* fix until date
* rearrange sorting control
* sort creation date
* fix events e2e test
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* lint
* lint
* don't use utc call time
---------
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
