# Which Problems Are Solved
The recently introduced notification queue have potential race conditions.
# How the Problems Are Solved
Current code is refactored to use the queue package, which is safe in
regards of concurrency.
# Additional Changes
- the queue is included in startup
- improved code quality of queue
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/9278
# Which Problems Are Solved
Setup fails to push all role permission events when running Zitadel with
CockroachDB. `TransactionRetryError`s were visible in logs which finally
times out the setup job with `timeout: context deadline exceeded`
# How the Problems Are Solved
As suggested in the [Cockroach documentation](timeout: context deadline
exceeded), _"break down larger transactions"_. The commands to be pushed
for the role permissions are chunked in 50 events per push. This
chunking is only done with CockroachDB.
# Additional Changes
- gci run fixed some unrelated imports
- access to `command.Commands` for the setup job, so we can reuse the
sync logic.
# Additional Context
Closes#9293
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
The `creationDate` property on user search V2 endpoint was missing
# How the Problems Are Solved
Added property in v2 `object.proto` and in the function creating the
details on each call
# Additional Changes
- none
# Additional Context
closes#8552
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Some OAuth2 and OIDC providers require the use of PKCE for all their
clients. While ZITADEL already recommended the same for its clients, it
did not yet support the option on the IdP configuration.
# How the Problems Are Solved
- A new boolean `use_pkce` is added to the add/update generic OAuth/OIDC
endpoints.
- A new checkbox is added to the generic OAuth and OIDC provider
templates.
- The `rp.WithPKCE` option is added to the provider if the use of PKCE
has been set.
- The `rp.WithCodeChallenge` and `rp.WithCodeVerifier` options are added
to the OIDC/Auth BeginAuth and CodeExchange function.
- Store verifier or any other persistent argument in the intent or auth
request.
- Create corresponding session object before creating the intent, to be
able to store the information.
- (refactored session structs to use a constructor for unified creation
and better overview of actual usage)
Here's a screenshot showing the URI including the PKCE params:
# Additional Changes
None.
# Additional Context
- Closes#6449
- This PR replaces the existing PR (#8228) of @doncicuto. The base he
did was cherry picked. Thank you very much for that!
---------
Co-authored-by: Miguel Cabrerizo <doncicuto@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
SortingColumn functionality on system API ListIAMMembers
SortingColumn functionality on admin API ListIAMMembers
# How the Problems Are Solved
I have added enum MemberFieldColumnName in` member.proto `file ,
consists of names of the columns on which the request can be sorted.
MEMBER_FIELD_NAME_UNSPECIFIED = 0;
MEMBER_FIELD_NAME_USER_ID=1;
MEMBER_FIELD_NAME_CREATION_DATE = 2;
MEMBER_FIELD_NAME_CHANGE_DATE=3;
MEMBER_FIELD_NAME_USER_RESOURCE_OWNER=4
I have added field Sorting Column for ListIAMMembersRequest in`
system.proto` file. I have added field Sorting Column for
ListIAMMembersRequest in` admin.proto` file.
I have modified ListIAMMembersRequestToQuery function in file
`internal/api/grpc/system/instance_converter.go `to include sorting
column in the query.SearchRequest{}.
I have modified ListIAMMembersRequestToQuery function in file
`internal/api/grpc/admin/iam_member_converter.go ` to include sorting
column in the query.SearchRequest{}.
# Additional Changes
Replace this example text with a concise list of additional changes that
this PR introduces, that are not directly solving the initial problem
but are related.
For example:
- The docs explicitly describe that the property XY is mandatory
- Adds missing translations for validations.
# Additional Context
Replace this example with links to related issues, discussions, discord
threads, or other sources with more context.
Use the Closing #issue syntax for issues that are resolved with this PR.
- Closes https://github.com/zitadel/zitadel/issues/5063
- Discussion #xxx
- Follow-up for PR #xxx
-
https://discordapp.com/channels/927474939156643850/1329872809488416789/1329872809488416789
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
The OAuth2 Device Authorization Grant could not yet been handled through
the new login UI, resp. using the session API.
This PR adds the ability for the login UI to get the required
information to display the user and handle their decision (approve with
authorization or deny) using the OIDC Service API.
# How the Problems Are Solved
- Added a `GetDeviceAuthorizationRequest` endpoint, which allows getting
the `id`, `client_id`, `scope`, `app_name` and `project_name` of the
device authorization request
- Added a `AuthorizeOrDenyDeviceAuthorization` endpoint, which allows to
approve/authorize with the session information or deny the request. The
identification of the request is done by the `device_authorization_id` /
`id` returned in the previous request.
- To prevent leaking the `device_code` to the UI, but still having an
easy reference, it's encrypted and returned as `id`, resp. decrypted
when used.
- Fixed returned error types for device token responses on token
endpoint:
- Explicitly return `access_denied` (without internal error) when user
denied the request
- Default to `invalid_grant` instead of `access_denied`
- Explicitly check on initial state when approving the reqeust
- Properly handle done case (also relates to initial check)
- Documented the flow and handling in custom UIs (according to OIDC /
SAML)
# Additional Changes
- fixed some typos and punctuation in the corresponding OIDC / SAML
guides.
- added some missing translations for auth and saml request
# Additional Context
- closes#6239
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
Enhancing `v2/ListUsers()` tests by adding code to run all test with and
without `permission_check_v2` flag set
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9356
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
When running `ListUsers()` with no permissions, the calling user shoud
be returned
# How the Problems Are Solved
Added additional clause to SQL search statement
# Additional Changes
n/a
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9355
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
New integration tests can't use command side to simulate successful
intents.
# How the Problems Are Solved
Add endpoints to only in integration tests available sink to create
already successful intents.
# Additional Changes
None
# Additional Context
Closes#8557
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Adding ability to add a root CA to LDAP configs
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/7888
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
Solves #8976
# Additional Changes
I have done some intensive refactorings and we are using the new
@zitadel/client package for GRPC access.
# Additional Context
- Closes#8976
---------
Co-authored-by: Max Peintner <peintnerm@gmail.com>
# Which Problems Are Solved
Performance issue for GRPC call `zitadel.user.v2.UserService.ListUsers`
due to lack of org filtering on `ListUsers`
# Additional Context
Replace this example with links to related issues, discussions, discord
threads, or other sources with more context.
Use the Closing #issue syntax for issues that are resolved with this PR.
- Closes https://github.com/zitadel/zitadel/issues/9191
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
OIDC applications can configure the used login version, which is
currently not possible for SAML applications.
# How the Problems Are Solved
Add the same functionality dependent on the feature-flag for SAML
applications.
# Additional Changes
None
# Additional Context
Closes#9267
Follow up issue for frontend changes #9354
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
When specifying a `user_id` as a human admin in
`zitadel.org.v2.AddOrganizationRequest.AddOrganization()` the `user_id`
specified in the request should have been used, before it was being
ignored, this has been fixed with this PR
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9308
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
To integrate river as a task queue we need to ensure the migrations of
river are executed.
# How the Problems Are Solved
- A new schema was added to the Zitadel database called "queue"
- Added a repeatable setup step to Zitadel which executes the
[migrations of
river](https://riverqueue.com/docs/migrations#go-migration-api).
# Additional Changes
- Added more hooks to the databases to properly set the schema for the
task queue
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9280
# Which Problems Are Solved
Project creation always requires a user as project owner, in case of a
system user creating the project, there is no valid user existing at
that moment.
# How the Problems Are Solved
Remove the initially created project owner membership, as this is
something which was necessary in old versions, and all should work
perfectly without.
The call to add a project automatically designates the calling user as
the project owner, which is irrelevant currently, as this user always
already has higher permissions to be able to even create the project.
# Additional Changes
Cleanup of the existing checks for the project, which can be improved
through the usage of the fields table.
# Additional Context
Closes#9182
# Which Problems Are Solved
Ordering of sessions in v2beta is still relevant in the integration
tests.
# How the Problems Are Solved
Correct the integration tests on session service v2beta like in v2.
# Additional Changes
None
# Additional Context
Failing integration tests in pipeline.
# Which Problems Are Solved
Through configuration on projects, there can be additional permission
checks enabled through an OIDC or SAML flow, which were not included in
the OIDC and SAML services.
# How the Problems Are Solved
Add permission check through the query-side of Zitadel in a singular SQL
query, when an OIDC or SAML flow should be linked to a SSO session. That
way it is eventual consistent, but will not impact the performance on
the eventstore. The permission check is defined in the API, which
provides the necessary function to the command side.
# Additional Changes
Added integration tests for the permission check on OIDC and SAML
service for every combination.
Corrected session list integration test, to content checks without
ordering.
Corrected get auth and saml request integration tests, to check for
timestamp of creation, not start of test.
# Additional Context
Closes#9265
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
When implementing simple stateless event processor, `the
user.grant.changed` bears too little information: just grant id and list
of role keys. This makes it impossible to change a users permissions
solely based on available role keys and requires to either:
- Store a mapping grant id -> user id, making a service stateful
- Make an extra call to zitadel to resolve user id by grant id (And it
doesn't seem that such an endpoint exists)
Same with `user.grant.removed` events.
# How the Problems Are Solved
Added `userId` field to `user.grant.changed` and `user.grant.removed`
events
# Additional Changes
`user.grant.removed` now has `projectId` and `grantId` as well
# Additional Context
- Closes#9113
# Which Problems Are Solved
As reported in #9311, even when providing a `x-zitadel-login-client`
header, the auth request would be created as hosted login UI / V1
request.
This is due to a change introduced with #9071, where the login UI
version can be specified using the app configuration.
The configuration set to V1 was not considering if the header was sent.
# How the Problems Are Solved
- Check presence of `x-zitadel-login-client` before the configuration.
Use later only if no header is set.
# Additional Changes
None
# Additional Context
- closes#9311
- needs back ports to 2.67.x, 2.68.x and 2.69.x
# Which Problems Are Solved
- SCIM tests are flaky due to metadata being set by the tests while
shortly after being read by the application, resulting in a race
condition
# How the Problems Are Solved
- whenever metadata is set, the projection is awaited
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
(cherry picked from commit 4dc7a58a256727169daf3ba3d2f9b1b40262f0ca)
# Which Problems Are Solved
- Microsoft Entra invokes the user patch endpoint with `"active":
"True"` / `"active": "False"` when patching a user. This is a well-known
bug in MS Entra (see
[here](https://learn.microsoft.com/en-us/entra/identity/app-provisioning/application-provisioning-config-problem-scim-compatibility)),
but the bug fix has not landed yet and/or the feature flag does not
work.
# How the Problems Are Solved
- To ensure compatibility with MS Entra, the parsing of the the boolean
active flag of the scim user is relaxed and accepts strings in any
casing that resolve to `true` or `false` as well as raw boolean values.
# Additional Context
Part of https://github.com/zitadel/zitadel/issues/8140
(cherry picked from commit 361f7a2edc0145528107c71753dee6e4d8c8b4c6)
# Which Problems Are Solved
- SCIM tests are flaky due to metadata being set by the tests while
shortly after being read by the application, resulting in a race
condition
# How the Problems Are Solved
- whenever metadata is set, the projection is awaited
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
- Microsoft Entra invokes the user patch endpoint with `"active":
"True"` / `"active": "False"` when patching a user. This is a well-known
bug in MS Entra (see
[here](https://learn.microsoft.com/en-us/entra/identity/app-provisioning/application-provisioning-config-problem-scim-compatibility)),
but the bug fix has not landed yet and/or the feature flag does not
work.
# How the Problems Are Solved
- To ensure compatibility with MS Entra, the parsing of the the boolean
active flag of the scim user is relaxed and accepts strings in any
casing that resolve to `true` or `false` as well as raw boolean values.
# Additional Context
Part of https://github.com/zitadel/zitadel/issues/8140
# Which Problems Are Solved
There were multiple issues in the OpenTelemetry (OTEL) implementation
and usage for tracing and metrics, which lead to high cardinality and
potential memory leaks:
- wrongly initiated tracing interceptors
- high cardinality in traces:
- HTTP/1.1 endpoints containing host names
- HTTP/1.1 endpoints containing object IDs like userID (e.g.
`/management/v1/users/2352839823/`)
- high amount of traces from internal processes (spooler)
- high cardinality in metrics endpoint:
- GRPC entries containing host names
- notification metrics containing instanceIDs and error messages
# How the Problems Are Solved
- Properly initialize the interceptors once and update them to use the
grpc stats handler (unary interceptors were deprecated).
- Remove host names from HTTP/1.1 span names and use path as default.
- Set / overwrite the uri for spans on the grpc-gateway with the uri
pattern (`/management/v1/users/{user_id}`). This is used for spans in
traces and metric entries.
- Created a new sampler which will only sample spans in the following
cases:
- remote was already sampled
- remote was not sampled, root span is of kind `Server` and based on
fraction set in the runtime configuration
- This will prevent having a lot of spans from the spooler back ground
jobs if they were not started by a client call querying an object (e.g.
UserByID).
- Filter out host names and alike from OTEL generated metrics (using a
`view`).
- Removed instance and error messages from notification metrics.
# Additional Changes
Fixed the middleware handling for serving Console. Telemetry and
instance selection are only used for the environment.json, but not on
statically served files.
# Additional Context
- closes#8096
- relates to #9074
- back ports to at least 2.66.x, 2.67.x and 2.68.x
(cherry picked from commit 990e1982c712ba2082f3fc6fc4861f3abf85b0cd)
# Which Problems Are Solved
There were multiple issues in the OpenTelemetry (OTEL) implementation
and usage for tracing and metrics, which lead to high cardinality and
potential memory leaks:
- wrongly initiated tracing interceptors
- high cardinality in traces:
- HTTP/1.1 endpoints containing host names
- HTTP/1.1 endpoints containing object IDs like userID (e.g.
`/management/v1/users/2352839823/`)
- high amount of traces from internal processes (spooler)
- high cardinality in metrics endpoint:
- GRPC entries containing host names
- notification metrics containing instanceIDs and error messages
# How the Problems Are Solved
- Properly initialize the interceptors once and update them to use the
grpc stats handler (unary interceptors were deprecated).
- Remove host names from HTTP/1.1 span names and use path as default.
- Set / overwrite the uri for spans on the grpc-gateway with the uri
pattern (`/management/v1/users/{user_id}`). This is used for spans in
traces and metric entries.
- Created a new sampler which will only sample spans in the following
cases:
- remote was already sampled
- remote was not sampled, root span is of kind `Server` and based on
fraction set in the runtime configuration
- This will prevent having a lot of spans from the spooler back ground
jobs if they were not started by a client call querying an object (e.g.
UserByID).
- Filter out host names and alike from OTEL generated metrics (using a
`view`).
- Removed instance and error messages from notification metrics.
# Additional Changes
Fixed the middleware handling for serving Console. Telemetry and
instance selection are only used for the environment.json, but not on
statically served files.
# Additional Context
- closes#8096
- relates to #9074
- back ports to at least 2.66.x, 2.67.x and 2.68.x
# Which Problems Are Solved
- when a scim user is provisioned, a init email could be sent
# How the Problems Are Solved
- no init email should be sent => hard code false for the email init
param
# Additional Context
Related to https://github.com/zitadel/zitadel/issues/8140
Co-authored-by: Fabienne Bühler <fabienne@zitadel.com>
# Which Problems Are Solved
- Some SCIM clients send "op" of a patch operation in PascalCase
# How the Problems Are Solved
- Well known "op" values of patch operations are matched
case-insensitive.
# Additional Context
Related to #8140
# Which Problems Are Solved
- If a SCIM endpoint is called with an orgID in the URL that is not the
resource owner, no error is returned, and the action is executed.
# How the Problems Are Solved
- The orgID provided in the SCIM URL path must match the resource owner
of the target user. Otherwise, an error will be returned.
# Additional Context
Part of https://github.com/zitadel/zitadel/issues/8140
# Which Problems Are Solved
* Adds support for the service provider configuration SCIM v2 endpoints
# How the Problems Are Solved
* Adds support for the service provider configuration SCIM v2 endpoints
* `GET /scim/v2/{orgId}/ServiceProviderConfig`
* `GET /scim/v2/{orgId}/ResourceTypes`
* `GET /scim/v2/{orgId}/ResourceTypes/{name}`
* `GET /scim/v2/{orgId}/Schemas`
* `GET /scim/v2/{orgId}/Schemas/{id}`
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Add the ability to update the timestamp when MFA initialization was last
skipped.
Get User By ID now also returns the timestamps when MFA setup was last
skipped.
# How the Problems Are Solved
- Add a `HumanMFAInitSkipped` method to the `users/v2` API.
- MFA skipped was already projected in the `auth.users3` table. In this
PR the same column is added to the users projection. Event handling is
kept the same as in the `UserView`:
<details>
62804ca45f/internal/user/repository/view/model/user.go (L243-L377)
</details>
# Additional Changes
- none
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9197
# Which Problems Are Solved
* Adds support for the bulk SCIM v2 endpoint
# How the Problems Are Solved
* Adds support for the bulk SCIM v2 endpoint under `POST
/scim/v2/{orgID}/Bulk`
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
- when listing users via scim v2.0 filters applied to the username are
applied case-sensitive
# How the Problems Are Solved
- when a query filter is appleid on the username it is applied
case-insensitive
# Additional Context
Part of https://github.com/zitadel/zitadel/issues/8140
# Which Problems Are Solved
Adds `failed attempts` field to the grpc response when a user enters
wrong password when logging in
FYI:
this only covers the senario above; other senarios where this is not
applied are:
SetPasswordWithVerifyCode
setPassword
ChangPassword
setPasswordWithPermission
# How the Problems Are Solved
Created new grpc message `CredentialsCheckError` -
`proto/zitadel/message.proto` to include `failed_attempts` field.
Had to create a new package -
`github.com/zitadel/zitadel/internal/command/errors` to resolve cycle
dependency between `github.com/zitadel/zitadel/internal/command` and
`github.com/zitadel/zitadel/internal/command`.
# Additional Changes
- none
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9198
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
- scim v2 only maps the primary phone/email to the zitadel user, this
does not work if no primary is set
# How the Problems Are Solved
- the first phone / email is mapped if no primary is available
# Additional Context
Part of #8140
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
#9185 changed that if a notification channel was not present,
notification workers would no longer retry to send the notification and
would also cancel in case Twilio would return a 4xx error.
However, this would not affect the "legacy" mode.
# How the Problems Are Solved
- Handle `CancelError` in legacy notifier as not failed (event).
# Additional Changes
None
# Additional Context
- relates to #9185
- requires back port to 2.66.x and 2.67.x
(cherry picked from commit 3fc68e5d60bb105c2ac72ca3d9cac7327f240b9b)
# Which Problems Are Solved
#9185 changed that if a notification channel was not present,
notification workers would no longer retry to send the notification and
would also cancel in case Twilio would return a 4xx error.
However, this would not affect the "legacy" mode.
# How the Problems Are Solved
- Handle `CancelError` in legacy notifier as not failed (event).
# Additional Changes
None
# Additional Context
- relates to #9185
- requires back port to 2.66.x and 2.67.x
# Which Problems Are Solved
- scim list users endpoint (`GET /scim/v2/{orgId}/Users`): handle
unsupported `SortBy` columns correctly
# How the Problems Are Solved
- throw an error if sorting by an unsupported column is requested
# Additional Context
Part of #8140
# Which Problems Are Solved
- requests to the scim interface with content type `*/*` are rejected
# How the Problems Are Solved
- `*/*` is accepted as content type
# Additional Context
Part of #8140
# Which Problems Are Solved
- SCIM user metadata mapping keys have differing case styles.
# How the Problems Are Solved
- key casing style is unified to strict camelCase
# Additional Context
Part of #8140
Although this is technically a breaking change, it is considered
acceptable because the SCIM feature is still in the preview stage and
not fully implemented yet.
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
* Adds support for the patch user SCIM v2 endpoint
# How the Problems Are Solved
* Adds support for the patch user SCIM v2 endpoint under `PATCH
/scim/v2/{orgID}/Users/{id}`
# Additional Context
Part of #8140
# Which Problems Are Solved
A customer reached out to support, that the (last) `additional origin`
could not be removed. While testing / implementation it was discovered,
that the same applied to `redirect_uris` and `post_logout_redirect_uris`
# How the Problems Are Solved
- Correctly set the corresponding array to empty in the event so it can
be differentiated to `null` / not set in case of no change.
# Additional Changes
Replaced `reflect.DeepEqual` with `slices.Equal`
# Additional Context
- Reported to support
(cherry picked from commit c9aa5db2a55ef3c4f6fb62292b69b01973b264e7)
