# Which Problems Are Solved
The session API allowed any authenticated user to update sessions by their ID without any further check.
This was unintentionally introduced with version 2.53.0 when the requirement of providing the latest session token on every session update was removed and no other permission check (e.g. session.write) was ensured.
# How the Problems Are Solved
- Granted `session.write` to `IAM_OWNER` and `IAM_LOGIN_CLIENT` in the defaults.yaml
- Granted `session.read` to `IAM_ORG_MANAGER`, `IAM_USER_MANAGER` and `ORG_OWNER` in the defaults.yaml
- Pass the session token to the UpdateSession command.
- Check for `session.write` permission on session creation and update.
- Alternatively, the (latest) sessionToken can be used to update the session.
- Setting an auth request to failed on the OIDC Service `CreateCallback` endpoint now ensures it's either the same user as used to create the auth request (for backwards compatibilty) or requires `session.link` permission.
- Setting an device auth request to failed on the OIDC Service `AuthorizeOrDenyDeviceAuthorization` endpoint now requires `session.link` permission.
- Setting an auth request to failed on the SAML Service `CreateResponse` endpoint now requires `session.link` permission.
# Additional Changes
none
# Additional Context
none
(cherry picked from commit 4c942f3477)
# Which Problems Are Solved
A customer reported that randomly certain login flows, such as automatic
redirect to the only configured IdP would not work. During the
investigation it was discovered that they used that same primary domain
on two different instances. As they used the domain for preselecting the
organization, one would always overwrite the other in the cache. Since
The organization and especially it's policies could not be retrieved on
the other instance, it would fallback to the default organization
settings, where the external login and the corresponding IdP were not
configured.
# How the Problems Are Solved
Include the instance id in the cache key for organizations to prevent
overwrites.
# Additional Changes
None
# Additional Context
- found because of a support request
- requires backport to 2.70.x, 2.71.x and 3.x
(cherry picked from commit 15902f5bc7)
# Which Problems Are Solved
Zitadel encounters a migration error when setting `restricted languages`
and fails to start.
# How the Problems Are Solved
The problem is that there is a check that checks that at least one of
the restricted languages is the same as the `default language`, however,
in the `authz instance` (where the default language is pulled form) is
never set.
I've added code to set the `default language` in the `authz instance`
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9787
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit b46c41e4bf)
if Zitadel was started using `start-from-init` or `start-from-setup`
there were rare cases where a panic occured when
`Notifications.LegacyEnabled` was set to false. The cause was a list
which was not reset before refilling.
The list is now reset before each time it gets filled.
Ensure all contexts are canceled for the init and setup functions for
`start-from-init- or `start-from-setup` commands.
none
# Which Problems Are Solved
List organization integration test fails sometimes due to incorrect
sorting of results.
# How the Problems Are Solved
Add sorting column to request on list organizations endpoint and sort
expected results.
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 6b07e57e5c)
# Which Problems Are Solved
ZITADEL uses the notification triggering requests Forwarded or
X-Forwarded-Proto header to build the button link sent in emails for
confirming a password reset with the emailed code. If this header is
overwritten and a user clicks the link to a malicious site in the email,
the secret code can be retrieved and used to reset the users password
and take over his account.
Accounts with MFA or Passwordless enabled can not be taken over by this
attack.
# How the Problems Are Solved
- The `X-Forwarded-Proto` and `proto` of the Forwarded headers are
validated (http / https).
- Additionally, when exposing ZITADEL through https. An overwrite to
http is no longer possible.
# Additional Changes
None
# Additional Context
None
(cherry picked from commit c097887bc5)
# Which Problems Are Solved
1. The projection handler reported no error if an error happened but
updating the current state was successful. This can lead to skipped
projections during setup as soon as the projection has an error but does
not correctly report if to the caller.
2. Mirror projections skipped as soon as an error occures, this leads to
unprojected projections.
3. Mirror checked position wrongly in some cases
# How the Problems Are Solved
1. the error returned by the `Trigger` method will will only be set to
the error of updating current states if there occured an error.
2. triggering projections checks for the error type returned and retries
if the error had code `23505`
3. Corrected to use the `Equal` method
# Additional Changes
unify logging on mirror projections
# Which Problems Are Solved
In rare cases there was a possibility that multiple users were found by
a loginname. This prevented the corresponding user to sign in.
# How the Problems Are Solved
Fixed the corresponding query (to correctly respect the org domain
policy).
# Additional Changes
None
# Additional Context
Found during the investigation of a support request
(cherry picked from commit 867e9cb15a)
# Which Problems Are Solved
When a user changes their password, Zitadel needs to terminate all of
that user's active sessions. This query can take many seconds on
deployments with large session and user tables. This happens as part of
session projection handling, so doesn't directly impact user experience,
but potentially bogs down the projection handler which isn't great. In
the future, this index could be used to power a "see all of my current
sessions" feature in Zitadel.
# How the Problems Are Solved
Adds new index on `user_id` column on `projections.sessions8` table.
Alternatively, we can index on `(instance_id, user_id)` instead but
opted for keeping the index smaller as we already index on `instance_id`
separately.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
(cherry picked from commit 8cb1d24b36)
# Which Problems Are Solved
New integration tests can't use command side to simulate successful
intents.
# How the Problems Are Solved
Add endpoints to only in integration tests available sink to create
already successful intents.
# Additional Changes
None
# Additional Context
Closes#8557
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 93466055ee)
# Which Problems Are Solved
With the change of #9561, the `mirror` command panics as there's no
metrics provider configured.
# How the Problems Are Solved
Correctly initialize the provider (no-op by default) for the mirror
command.
# Additional Changes
None
# Additional Context
relates to #9561 -> needs backports to 2.66.x - 2.71.x and 3.0.0-rc
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
We found some paths in the login UI, where requests without any
`AuthRequest` were not handled correctly and could potentially panic.
This also includes providing the `AuthRequest` as part of `ctx` object
in actions V1.
# How the Problems Are Solved
- Check for the existance of an `AuthRequest` were needed and return an
error otherwise.
- Provide correct state of the `AuthRequest` for actions V1
# Additional Changes
None
# Additional Context
- Noticed as part of a support request
- requires backport to at least 2.70.x
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
ZITADEL allows the use of JSON Web Token (JWT) Profile OAuth 2.0 for Authorization Grants in machine-to-machine (M2M) authentication. Multiple keys can be managed for a single machine account (service user), each with an individual expiry.
A vulnerability existed where expired keys can be used to retrieve tokens. Specifically, ZITADEL fails to properly check the expiration date of the JWT key when used for Authorization Grants. This allows an attacker with an expired key to obtain valid access tokens.
This vulnerability does not affect the use of JWT Profile for OAuth 2.0 Client Authentication on the Token and Introspection endpoints, which correctly reject expired keys.
# How the Problems Are Solved
Added proper validation of the expiry of the stored public key.
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 315503beab)
# Which Problems Are Solved
The username entered by the user was resp. replaced by the stored user's username. This provided a possibility to enumerate usernames as unknown usernames were not normalized.
# How the Problems Are Solved
- Store and display the username as entered by the user.
- Removed the part where the loginname was always set to the user's loginname when retrieving the `nextSteps`
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 14de8ecac2)
# Which Problems Are Solved
The service name is hardcoded in the metrics code. Making the service
name to be configurable helps when running multiple instances of
Zitadel.
The defaults remain unchanged, the service name will be defaulted to
ZITADEL.
# How the Problems Are Solved
Add a config option to override the name in defaults.yaml and pass it
down to the corresponding metrics or tracing module (google or otel)
# Additional Changes
NA
# Additional Context
NA
(cherry picked from commit dc64e35128)
# Which Problems Are Solved
With current provided telemetry it's difficult to predict when a
projection handler is under increased load until it's too late and
causes downstream issues. Importantly, projection updating is in the
critical path for many login flows and increased latency there can
result in system downtime for users.
# How the Problems Are Solved
This PR adds three new prometheus-style metrics:
1. **projection_events_processed** (_labels: projection, success_) -
This metric gives us a counter of the number of events processed per
projection update run and whether they we're processed without error. A
high number of events being processed can let us know how busy a
particular projection handler is.
2. **projection_handle_timer** _(labels: projection)_ - This is the time
it takes to process a projection update given a batch of events - time
to take the current_states lock, query for new events, reduce,
update_the projection, and update current_states.
3. **projection_state_latency** _(labels: projection)_ - This is the
time from the last event processed in the current_states table for a
given projection. It tells us how old was the last event you processed?
Or, how far behind are you running for this projection? Higher latencies
could mean high load or stalled projection handling.
# Additional Changes
I also had to initialize the global otel metrics provider (`metrics.M`)
in the `setup` step additionally to `start` since projection handlers
are initialized at setup. The initialization checks if a metrics
provider is already set (in case of `start-from-setup` or
`start-from-init` to prevent overwriting, which causes the otel metrics
provider to stop working.
# Additional Context
## Example Dashboards
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit c1535b7b49)
# Which Problems Are Solved
Build and test workflows are currently running on specific GitHub hosted
runners. These is not needed for most worklfows and just costs more.
# How the Problems Are Solved
Moved all the steps apart from integration-tests to public runners.
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 3424204291)
# Which Problems Are Solved
It was not possible to use the `api.v1.appendUserGrant` function in the
`postCreation` trigger action as documented.
# How the Problems Are Solved
- Correctly initialize the javascript / Goja function
- Added `projectID` and `projectGrantID` (as documented), but kept
`projectId` and `projectGrantId` (for backwards compatibility) when
mapping the object in the `appendUserGrant` function
# Additional Changes
None
# Additional Context
- A customer reached out to support regarding this issue.
- requires back port to 2.70.x
(cherry picked from commit 57bfecf7f7)
# Which Problems Are Solved
When we recently changed some permission for the `SYSTEM_OWNER` role on
QA, we noticed that there we multiple `permission.added` even when we
removed specific permissions.
# How the Problems Are Solved
Fixed the event type when removing permissions.
# Additional Changes
None
# Additional Context
Noticed when rolling out some changes on QA
(cherry picked from commit e7c7f9b800)
# Which Problems Are Solved
The milestones query returns multiple results for every milestone for
every instance domain.
# How the Problems Are Solved
Corrected where condition on milestone query.
# Additional Changes
None
# Additional Context
None
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 352fa6aa6f)
# Which Problems Are Solved
SCIM integration test failed sometimes, as ListUsers with usernames-sort
was not reliable if the asserted list is not sorted as well.
# How the Problems Are Solved
Sort the list of results in the sorted integration tests.
# Additional Changes
None
# Additional Context
Relates to
https://github.com/zitadel/zitadel/actions/runs/13922326003/job/38960759621
(cherry picked from commit 5486ef2627)
# Which Problems Are Solved
Quota notification integration test failed sometimes due to eventual
consistency issues, which resulted in calls which should have been
counted to the quota not being added. This resulted in flaky integration
tests as the expected API calls to be limited were executed normally.
# How the Problems Are Solved
As there is no API call to query the currently applied Quota, there was
a sleep added as a last effort, to give some time that the event gets
processed into the projection.
# Additional Changes
None
# Additional Context
Related to
https://github.com/zitadel/zitadel/actions/runs/13922326003/job/38959595055
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 5ca76af779)
# Which Problems Are Solved
Zitadel should not record 404 response counts of unknown paths (check
`/debug/metrics`).
This can lead to high cardinality on metrics endpoint and in traces.
```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
BAD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="404",uri="/junk"} 2000
```
After
```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
```
# How the Problems Are Solved
This PR makes sure, that any unknown path is recorded as `UNKNOWN_PATH`
instead of the actual path.
# Additional Changes
N/A
# Additional Context
On our production instance, when a penetration test was run, it caused
our metric count to blow up to many thousands due to Zitadel recording
404 response counts.
Next nice to have steps, remove 404 timer recordings which serve no
purpose
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio@zitadel.com>
(cherry picked from commit 599850e7e8)
# Which Problems Are Solved
Scripts and other assets for the hosted login UI are served with a
public cache with `max-age` and `s-maxage`. After changing scripts or
assets, old versions might be still used as they might be cached locally
or in a shared cache (CDN, proxy, ...). This can lead to unwanted
behaviour or even errors.
# How the Problems Are Solved
To ensure the correct file is served a query parameter with the build
time is added to the assets filename dynamically. (`?v=2025-03-17...`)
# Additional Changes
None
# Additional Context
- relates to #9485
- requires backport to at least 2.70.x
(cherry picked from commit 19f022e1cf)
# Which Problems Are Solved
When using a custom / new login UI and an OIDC application with
registered BackChannelLogoutUI, no logout requests were sent to the URI
when the user signed out.
Additionally, as described in #9427, an error was logged:
`level=error msg="event of type *session.TerminateEvent doesn't
implement OriginEvent"
caller="/home/runner/work/zitadel/zitadel/internal/notification/handlers/origin.go:24"`
# How the Problems Are Solved
- Properly pass `TriggerOrigin` information to session.TerminateEvent
creation and implement `OriginEvent` interface.
- Implemented `RegisterLogout` in `CreateOIDCSessionFromAuthRequest` and
`CreateOIDCSessionFromDeviceAuth`, both used when interacting with the
OIDC v2 API.
- Both functions now receive the `BackChannelLogoutURI` of the client
from the OIDC layer.
# Additional Changes
None
# Additional Context
- closes#9427
(cherry picked from commit ed697bbd69)
# Which Problems Are Solved
When requesting a JWT (`urn:ietf:params:oauth:token-type:jwt`) to be
returned in a Token Exchange request, ZITADEL would panic if the `actor`
was not granted the necessary permission.
# How the Problems Are Solved
Properly check the error and return it.
# Additional Changes
None
# Additional Context
- closes#9436
(cherry picked from commit e6ce1af003)
# Which Problems Are Solved
When registering passkeys or u2f methods as second factor, some users
pressed the "Enter" key, rather than clicking the submit button. This
method has bypassed the execution of the device registration and
encoding scripts, resulting in the form being submitted without the
necessary encoded values.
# How the Problems Are Solved
This PR ensures that device registration is always executed and the
required information are submitted in the form regardless of pressing
"Enter" or clicking the button.
# Additional Changes
None
# Additional Context
- closes#6592
- closes#2910
(cherry picked from commit 27b319bd98)
# Which Problems Are Solved
ZITADEL's Admin API, intended for managing ZITADEL instances, contains 12 HTTP endpoints that are unexpectedly accessible to authenticated ZITADEL users who are not ZITADEL managers. The most critical vulnerable endpoints relate to LDAP configuration:
- /idps/ldap
- /idps/ldap/{id}
By accessing these endpoints, unauthorized users could:
- Modify ZITADEL's instance LDAP settings, redirecting all LDAP login attempts to a malicious server, effectively taking over user accounts.
- Expose the original LDAP server's password, potentially compromising all user accounts.
The following endpoints are also affected by IDOR vulnerabilities, potentially allowing unauthorized modification of instance settings such as languages, labels, and templates:
- /idps/templates/_search
- /idps/templates/{id}
- /policies/label/_activate
- /policies/label/logo
- /policies/label/logo_dark
- /policies/label/icon
- /policies/label/icon_dark
- /policies/label/font
- /text/message/passwordless_registration/{language}
- /text/login/{language}
Please checkout https://github.com/zitadel/zitadel/security/advisories/GHSA-f3gh-529w-v32x for more information.
# How the Problems Are Solved
- Required permission have been fixed (only instance level allowed)
# Additional Changes
None
# Additional Context
- resolves https://github.com/zitadel/zitadel/security/advisories/GHSA-f3gh-529w-v32x
(cherry picked from commit d9d8339813)
# Which Problems Are Solved
#9110 introduced more possibilities to search for "own" sessions. Due to
this the permission checks for retrieving a session had to be updated
accordingly. Internal calls, such as retrieving them for sending
notifications do not require a permission, but the code was not properly
adjusted and thus could lead to panics.
# How the Problems Are Solved
- Properly handled (do not require) permission check for internal only
calls when retrieving the session by id.
# Additional Changes
None
# Additional Context
- needs backports to 2.68.x, 2.69.x, 2.70.x
- closeszitadel/devops#117
(cherry picked from commit 4e1868e9bb)
# Which Problems Are Solved
Systems running with PostgreSQL before Zitadel v2.39 are likely to have
a wrong type for the `in_tx_order` column in the `eventstore.event2`
table. The migration at the time used the `event_sequence` as default
value without typecast, which results in a `bigint` type for that
column. However, when creating the table from scratch, we explicitly
specify the type to be `integer`.
Starting from Zitadel v2.67 we use a Pl/PgSQL function to push events.
The function requires the types from `eventstore.events2` to the same as
the `select` destinations used in the function. In the function
`in_tx_order` is also expected to by of `integer` type.
CochroachDB systems are not affected because `bigint` is an alias to the
`int` type. In other words, CockroachDB uses `int8` when specifying type
`int`. Therefore the types already match.
# How the Problems Are Solved
Retrieve the actual column type currently in use. A template is used to
assign the type to the `ordinality` column returned as `in_tx_order`.
# Additional Changes
- Detailed logging on migration failure
# Additional Context
- Closes#9180
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
(cherry picked from commit bcc6a689fa)
# Which Problems Are Solved
As reported in #9311, even when providing a `x-zitadel-login-client`
header, the auth request would be created as hosted login UI / V1
request.
This is due to a change introduced with #9071, where the login UI
version can be specified using the app configuration.
The configuration set to V1 was not considering if the header was sent.
# How the Problems Are Solved
- Check presence of `x-zitadel-login-client` before the configuration.
Use later only if no header is set.
# Additional Changes
None
# Additional Context
- closes#9311
- needs back ports to 2.67.x, 2.68.x and 2.69.x
(cherry picked from commit e7a73eb6b1)
# Which Problems Are Solved
OIDC applications can configure the used login version, which is
currently not possible for SAML applications.
# How the Problems Are Solved
Add the same functionality dependent on the feature-flag for SAML
applications.
# Additional Changes
None
# Additional Context
Closes#9267
Follow up issue for frontend changes #9354
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
When specifying a `user_id` as a human admin in
`zitadel.org.v2.AddOrganizationRequest.AddOrganization()` the `user_id`
specified in the request should have been used, before it was being
ignored, this has been fixed with this PR
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9308
---------
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
To integrate river as a task queue we need to ensure the migrations of
river are executed.
# How the Problems Are Solved
- A new schema was added to the Zitadel database called "queue"
- Added a repeatable setup step to Zitadel which executes the
[migrations of
river](https://riverqueue.com/docs/migrations#go-migration-api).
# Additional Changes
- Added more hooks to the databases to properly set the schema for the
task queue
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9280
# Which Problems Are Solved
Project creation always requires a user as project owner, in case of a
system user creating the project, there is no valid user existing at
that moment.
# How the Problems Are Solved
Remove the initially created project owner membership, as this is
something which was necessary in old versions, and all should work
perfectly without.
The call to add a project automatically designates the calling user as
the project owner, which is irrelevant currently, as this user always
already has higher permissions to be able to even create the project.
# Additional Changes
Cleanup of the existing checks for the project, which can be improved
through the usage of the fields table.
# Additional Context
Closes#9182
# Which Problems Are Solved
Systems running with PostgreSQL before Zitadel v2.39 are likely to have
a wrong type for the `in_tx_order` column in the `eventstore.event2`
table. The migration at the time used the `event_sequence` as default
value without typecast, which results in a `bigint` type for that
column. However, when creating the table from scratch, we explicitly
specify the type to be `integer`.
Starting from Zitadel v2.67 we use a Pl/PgSQL function to push events.
The function requires the types from `eventstore.events2` to the same as
the `select` destinations used in the function. In the function
`in_tx_order` is also expected to by of `integer` type.
CochroachDB systems are not affected because `bigint` is an alias to the
`int` type. In other words, CockroachDB uses `int8` when specifying type
`int`. Therefore the types already match.
# How the Problems Are Solved
Retrieve the actual column type currently in use. A template is used to
assign the type to the `ordinality` column returned as `in_tx_order`.
# Additional Changes
- Detailed logging on migration failure
# Additional Context
- Closes#9180
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
Ordering of sessions in v2beta is still relevant in the integration
tests.
# How the Problems Are Solved
Correct the integration tests on session service v2beta like in v2.
# Additional Changes
None
# Additional Context
Failing integration tests in pipeline.
# Which Problems Are Solved
Through configuration on projects, there can be additional permission
checks enabled through an OIDC or SAML flow, which were not included in
the OIDC and SAML services.
# How the Problems Are Solved
Add permission check through the query-side of Zitadel in a singular SQL
query, when an OIDC or SAML flow should be linked to a SSO session. That
way it is eventual consistent, but will not impact the performance on
the eventstore. The permission check is defined in the API, which
provides the necessary function to the command side.
# Additional Changes
Added integration tests for the permission check on OIDC and SAML
service for every combination.
Corrected session list integration test, to content checks without
ordering.
Corrected get auth and saml request integration tests, to check for
timestamp of creation, not start of test.
# Additional Context
Closes#9265
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
