# Which Problems Are Solved
In rare cases there was a possibility that multiple users were found by
a loginname. This prevented the corresponding user to sign in.
# How the Problems Are Solved
Fixed the corresponding query (to correctly respect the org domain
policy).
# Additional Changes
None
# Additional Context
Found during the investigation of a support request
(cherry picked from commit 867e9cb15a)
# Which Problems Are Solved
In rare cases there was a possibility that multiple users were found by
a loginname. This prevented the corresponding user to sign in.
# How the Problems Are Solved
Fixed the corresponding query (to correctly respect the org domain
policy).
# Additional Changes
None
# Additional Context
Found during the investigation of a support request
# Which Problems Are Solved
When a user changes their password, Zitadel needs to terminate all of
that user's active sessions. This query can take many seconds on
deployments with large session and user tables. This happens as part of
session projection handling, so doesn't directly impact user experience,
but potentially bogs down the projection handler which isn't great. In
the future, this index could be used to power a "see all of my current
sessions" feature in Zitadel.
# How the Problems Are Solved
Adds new index on `user_id` column on `projections.sessions8` table.
Alternatively, we can index on `(instance_id, user_id)` instead but
opted for keeping the index smaller as we already index on `instance_id`
separately.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
(cherry picked from commit 8cb1d24b36)
# Which Problems Are Solved
Currently, users can't delete themselves using the V2 RemoveUser API
because of the redunant API middleware permission check.
On main, using a machine user PAT to delete the same machine user:
```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
ERROR:
Code: NotFound
Message: membership not found (AUTHZ-cdgFk)
Details:
1) {
"@type": "type.googleapis.com/zitadel.v1.ErrorDetail",
"id": "AUTHZ-cdgFk",
"message": "membership not found"
}
```
Same on this PRs branch:
```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
{
"details": {
"sequence": "3",
"changeDate": "2025-05-06T13:44:54.349048Z",
"resourceOwner": "318838541083804033"
}
}
```
Repeated call
```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
ERROR:
Code: Unauthenticated
Message: Errors.Token.Invalid (AUTH-7fs1e)
Details:
1) {
"@type": "type.googleapis.com/zitadel.v1.ErrorDetail",
"id": "AUTH-7fs1e",
"message": "Errors.Token.Invalid"
}
```
# How the Problems Are Solved
The middleware permission check is disabled and the
domain.PermissionCheck is used exclusively.
# Additional Changes
A new type command.PermissionCheck allows to optionally accept a
permission check for commands, so APIs with middleware permission checks
can omit redundant permission checks by passing nil while APIs without
middleware permission checks can pass one to the command.
# Additional Context
This is a subtask of #9763
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
When a user changes their password, Zitadel needs to terminate all of
that user's active sessions. This query can take many seconds on
deployments with large session and user tables. This happens as part of
session projection handling, so doesn't directly impact user experience,
but potentially bogs down the projection handler which isn't great. In
the future, this index could be used to power a "see all of my current
sessions" feature in Zitadel.
# How the Problems Are Solved
Adds new index on `user_id` column on `projections.sessions8` table.
Alternatively, we can index on `(instance_id, user_id)` instead but
opted for keeping the index smaller as we already index on `instance_id`
separately.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
The sorting column of action targets and executions defaults to the ID
column instead of the creation date column.
This is only relevant, if the sorting column is explicitly passed as
unspecified.
If the sorting column is not passed, it correctly defaults to the
creation date.
```bash
# ❌ Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": "TARGET_FIELD_NAME_UNSPECIFIED"}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
# ❌ Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": 0}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
# ✅ Sorts by creation date
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
```
# How the Problems Are Solved
`action.TargetFieldName_TARGET_FIELD_NAME_UNSPECIFIED` maps to the
sorting column `query.TargetColumnCreationDate`.
# Additional Context
As IDs are also generated in ascending, like creation dates, the the bug
probably only causes unexpected behavior for cases, where the ID is
specified during target or execution creation. This is currently not
supported, so this bug probably has no impact at all. It doesn't need to
be backported.
Found during implementation of #9763
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 74ace1aec3)
# Which Problems Are Solved
The sorting column of action targets and executions defaults to the ID
column instead of the creation date column.
This is only relevant, if the sorting column is explicitly passed as
unspecified.
If the sorting column is not passed, it correctly defaults to the
creation date.
```bash
# ❌ Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": "TARGET_FIELD_NAME_UNSPECIFIED"}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
# ❌ Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": 0}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
# ✅ Sorts by creation date
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
```
# How the Problems Are Solved
`action.TargetFieldName_TARGET_FIELD_NAME_UNSPECIFIED` maps to the
sorting column `query.TargetColumnCreationDate`.
# Additional Context
As IDs are also generated in ascending, like creation dates, the the bug
probably only causes unexpected behavior for cases, where the ID is
specified during target or execution creation. This is currently not
supported, so this bug probably has no impact at all. It doesn't need to
be backported.
Found during implementation of #9763
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
In Actions v2, if a target is removed, which is still used in an
execution, the target is still listed when list executions.
# How the Problems Are Solved
Removed targets are now also removed from the executions.
# Additional Changes
To be sure the list executions include a check if the target is still
existing.
# Additional Context
None
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit a05f7ce3fc)
# Which Problems Are Solved
Users from Google IDP's are not unmarshalled correctly in intent
endpoints and not returned to callers.
# How the Problems Are Solved
Provided correct type for unmarshalling of the information.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 3953879fe9)
# Which Problems Are Solved
Instance that had improved performance flags set, got event errors when
getting instance features. This is because the improved performance
flags were marshalled using the enumerated integers, but now needed to
be unmashalled using the added UnmarshallText method.
# How the Problems Are Solved
- Remove emnumer generation
# Additional Changes
- none
# Additional Context
- reported on QA
- Backport to next-rc / v3
(cherry picked from commit 0465d5093e)
# Which Problems Are Solved
The `auth.auth_requests` table is not cleaned up so long running Zitadel
installations can contain many rows.
The mirror command can take long because a the data are first copied
into memory (or disk) on cockroach and users do not get any output from
mirror. This is unfortunate because people don't know if Zitadel got
stuck.
# How the Problems Are Solved
Enhance logging throughout the projection processes and introduce a
configuration option for the maximum age of authentication requests.
# Additional Changes
None
# Additional Context
closes https://github.com/zitadel/zitadel/issues/9764
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 181186e477)
# Which Problems Are Solved
Webkeys were not generated with new instances when the webkey feature
flag was enabled for instance defaults. This would cause a redirect loop
with console for new instances on QA / coud.
# How the Problems Are Solved
- uncomment the webkeys section on defaults.yaml
- Fix field naming of webkey config
# Additional Changes
- Add all available features as comments.
- Make the improved performance type enum parsable from the config,
untill now they were just ints.
- Running of the enumer command created missing enum entries for feature
keys.
# Additional Context
- Needs to be back-ported to v3 / next-rc
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 91bc71db74)
# Which Problems Are Solved
In Actions v2, if a target is removed, which is still used in an
execution, the target is still listed when list executions.
# How the Problems Are Solved
Removed targets are now also removed from the executions.
# Additional Changes
To be sure the list executions include a check if the target is still
existing.
# Additional Context
None
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Users from Google IDP's are not unmarshalled correctly in intent
endpoints and not returned to callers.
# How the Problems Are Solved
Provided correct type for unmarshalling of the information.
# Additional Changes
None
# Additional Context
None
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Instance that had improved performance flags set, got event errors when
getting instance features. This is because the improved performance
flags were marshalled using the enumerated integers, but now needed to
be unmashalled using the added UnmarshallText method.
# How the Problems Are Solved
- Remove emnumer generation
# Additional Changes
- none
# Additional Context
- reported on QA
- Backport to next-rc / v3
# Which Problems Are Solved
The `auth.auth_requests` table is not cleaned up so long running Zitadel
installations can contain many rows.
The mirror command can take long because a the data are first copied
into memory (or disk) on cockroach and users do not get any output from
mirror. This is unfortunate because people don't know if Zitadel got
stuck.
# How the Problems Are Solved
Enhance logging throughout the projection processes and introduce a
configuration option for the maximum age of authentication requests.
# Additional Changes
None
# Additional Context
closes https://github.com/zitadel/zitadel/issues/9764
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Webkeys were not generated with new instances when the webkey feature
flag was enabled for instance defaults. This would cause a redirect loop
with console for new instances on QA / coud.
# How the Problems Are Solved
- uncomment the webkeys section on defaults.yaml
- Fix field naming of webkey config
# Additional Changes
- Add all available features as comments.
- Make the improved performance type enum parsable from the config,
untill now they were just ints.
- Running of the enumer command created missing enum entries for feature
keys.
# Additional Context
- Needs to be back-ported to v3 / next-rc
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Actions v2 is not a feature flag anymore, include functionality on
executions is not used and json tags of proto messages are handled
incorrectly.
# How the Problems Are Solved
- Remove actions from the feature flags on system and instance level
- Remove include type on executions, only in the API, later maybe in the
handling logic as well
- Use protojson in request and response handling of actions v2
# Additional Changes
- Correct integration tests for request and response handling
- Use json.RawMessage for events, so that the event payload is not
base64 encoded
- Added separate context for async webhook calls, that executions are
not cancelled when called async
# Additional Context
Related to #9759Closes#9710
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit b8ba7bd5ba)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->
# Which Problems Are Solved
A customer reached out that after an upgrade, actions would always fail
with the error "host is denied" when calling an external API.
This is due to a security fix
(https://github.com/zitadel/zitadel/security/advisories/GHSA-6cf5-w9h3-4rqv),
where a DNS lookup was added to check whether the host name resolves to
a denied IP or subnet.
If the lookup fails due to the internal DNS setup, the action fails as
well. Additionally, the lookup was also performed when the deny list was
empty.
# How the Problems Are Solved
- Prevent DNS lookup when deny list is empty
- Properly initiate deny list and prevent empty entries
# Additional Changes
- Log the reason for blocked address (domain, IP, subnet)
# Additional Context
- reported by a customer
- needs backport to 2.70.x, 2.71.x and 3.0.0 rc
(cherry picked from commit 4ffd4ef381)
# Which Problems Are Solved
Actions v2 is not a feature flag anymore, include functionality on
executions is not used and json tags of proto messages are handled
incorrectly.
# How the Problems Are Solved
- Remove actions from the feature flags on system and instance level
- Remove include type on executions, only in the API, later maybe in the
handling logic as well
- Use protojson in request and response handling of actions v2
# Additional Changes
- Correct integration tests for request and response handling
- Use json.RawMessage for events, so that the event payload is not
base64 encoded
- Added separate context for async webhook calls, that executions are
not cancelled when called async
# Additional Context
Related to #9759Closes#9710
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->
# Which Problems Are Solved
A customer reached out that after an upgrade, actions would always fail
with the error "host is denied" when calling an external API.
This is due to a security fix
(https://github.com/zitadel/zitadel/security/advisories/GHSA-6cf5-w9h3-4rqv),
where a DNS lookup was added to check whether the host name resolves to
a denied IP or subnet.
If the lookup fails due to the internal DNS setup, the action fails as
well. Additionally, the lookup was also performed when the deny list was
empty.
# How the Problems Are Solved
- Prevent DNS lookup when deny list is empty
- Properly initiate deny list and prevent empty entries
# Additional Changes
- Log the reason for blocked address (domain, IP, subnet)
# Additional Context
- reported by a customer
- needs backport to 2.70.x, 2.71.x and 3.0.0 rc
(cherry picked from commit 4ffd4ef381)
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr.
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions.
-->
# Which Problems Are Solved
A customer reached out that after an upgrade, actions would always fail
with the error "host is denied" when calling an external API.
This is due to a security fix
(https://github.com/zitadel/zitadel/security/advisories/GHSA-6cf5-w9h3-4rqv),
where a DNS lookup was added to check whether the host name resolves to
a denied IP or subnet.
If the lookup fails due to the internal DNS setup, the action fails as
well. Additionally, the lookup was also performed when the deny list was
empty.
# How the Problems Are Solved
- Prevent DNS lookup when deny list is empty
- Properly initiate deny list and prevent empty entries
# Additional Changes
- Log the reason for blocked address (domain, IP, subnet)
# Additional Context
- reported by a customer
- needs backport to 2.70.x, 2.71.x and 3.0.0 rc
# Which Problems Are Solved
Add the possibility to filter project resources based on project member
roles.
# How the Problems Are Solved
Extend and refactor existing Pl/PgSQL functions to implement the
following:
- Solve O(n) complexity in returned resources IDs by returning a boolean
filter for instance level permissions.
- Individually permitted orgs are returned only if there was no instance
permission
- Individually permitted projects are returned only if there was no
instance permission
- Because of the multiple filter terms, use `INNER JOIN`s instead of
`WHERE` clauses.
# Additional Changes
- system permission function no longer query the organization view and
therefore can be `immutable`, giving big performance benefits for
frequently reused system users. (like our hosted login in Zitadel cloud)
- The permitted org and project functions are now defined as `stable`
because the don't modify on-disk data. This might give a small
performance gain
- The Pl/PgSQL functions are now tested using Go unit tests.
# Additional Context
- Depends on https://github.com/zitadel/zitadel/pull/9677
- Part of https://github.com/zitadel/zitadel/issues/9188
- Closes https://github.com/zitadel/zitadel/issues/9190
# Which Problems Are Solved
Classic permission checks execute for every returned row on resource
based search APIs. Complete background and problem definition can be
found here: https://github.com/zitadel/zitadel/issues/9188
# How the Problems Are Solved
- PermissionClause function now support dynamic query building, so it
supports multiple cases.
- PermissionClause is applied to all list resources which support org
level permissions.
- Wrap permission logic into wrapper functions so we keep the business
logic clean.
# Additional Changes
- Handle org ID optimization in the query package, so it is reusable for
all resources, instead of extracting the filter in the API.
- Cleanup and test system user conversion in the authz package. (context
middleware)
- Fix: `core_integration_db_up` make recipe was missing the postgres
service.
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/9190
# Which Problems Are Solved
We found some paths in the login UI, where requests without any
`AuthRequest` were not handled correctly and could potentially panic.
This also includes providing the `AuthRequest` as part of `ctx` object
in actions V1.
# How the Problems Are Solved
- Check for the existance of an `AuthRequest` were needed and return an
error otherwise.
- Provide correct state of the `AuthRequest` for actions V1
# Additional Changes
None
# Additional Context
- Noticed as part of a support request
- requires backport to at least 2.70.x
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
We found some paths in the login UI, where requests without any
`AuthRequest` were not handled correctly and could potentially panic.
This also includes providing the `AuthRequest` as part of `ctx` object
in actions V1.
# How the Problems Are Solved
- Check for the existance of an `AuthRequest` were needed and return an
error otherwise.
- Provide correct state of the `AuthRequest` for actions V1
# Additional Changes
None
# Additional Context
- Noticed as part of a support request
- requires backport to at least 2.70.x
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
ZITADEL allows the use of JSON Web Token (JWT) Profile OAuth 2.0 for Authorization Grants in machine-to-machine (M2M) authentication. Multiple keys can be managed for a single machine account (service user), each with an individual expiry.
A vulnerability existed where expired keys can be used to retrieve tokens. Specifically, ZITADEL fails to properly check the expiration date of the JWT key when used for Authorization Grants. This allows an attacker with an expired key to obtain valid access tokens.
This vulnerability does not affect the use of JWT Profile for OAuth 2.0 Client Authentication on the Token and Introspection endpoints, which correctly reject expired keys.
# How the Problems Are Solved
Added proper validation of the expiry of the stored public key.
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 315503beab)
# Which Problems Are Solved
The username entered by the user was resp. replaced by the stored user's username. This provided a possibility to enumerate usernames as unknown usernames were not normalized.
# How the Problems Are Solved
- Store and display the username as entered by the user.
- Removed the part where the loginname was always set to the user's loginname when retrieving the `nextSteps`
# Additional Changes
None
# Additional Context
None
(cherry picked from commit 14de8ecac2)
This pull request improves the scalability of the session API by
enhancing middleware tracing and refining SQL query behavior for user
authentication methods.
# Which Problems Are Solved
- Eventstore subscriptions locked each other during they wrote the
events to the event channels of the subscribers in push.
- `ListUserAuthMethodTypesRequired` query used `Bitmap heap scan` to
join the tables needed.
- The auth and oidc package triggered projections often when data were
read.
- The session API triggered the user projection each time a user was
searched to write the user check command.
# How the Problems Are Solved
- the `sync.Mutex` was replaced with `sync.RWMutex` to allow parallel
read of the map
- The query was refactored to use index scans only
- if the data should already be up-to-date `shouldTriggerBulk` is set to
false
- as the user should already exist for some time the trigger was
removed.
# Additional Changes
- refactoring of `tracing#Span.End` calls
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/9239
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
With current provided telemetry it's difficult to predict when a
projection handler is under increased load until it's too late and
causes downstream issues. Importantly, projection updating is in the
critical path for many login flows and increased latency there can
result in system downtime for users.
# How the Problems Are Solved
This PR adds three new prometheus-style metrics:
1. **projection_events_processed** (_labels: projection, success_) -
This metric gives us a counter of the number of events processed per
projection update run and whether they we're processed without error. A
high number of events being processed can let us know how busy a
particular projection handler is.
2. **projection_handle_timer** _(labels: projection)_ - This is the time
it takes to process a projection update given a batch of events - time
to take the current_states lock, query for new events, reduce,
update_the projection, and update current_states.
3. **projection_state_latency** _(labels: projection)_ - This is the
time from the last event processed in the current_states table for a
given projection. It tells us how old was the last event you processed?
Or, how far behind are you running for this projection? Higher latencies
could mean high load or stalled projection handling.
# Additional Changes
I also had to initialize the global otel metrics provider (`metrics.M`)
in the `setup` step additionally to `start` since projection handlers
are initialized at setup. The initialization checks if a metrics
provider is already set (in case of `start-from-setup` or
`start-from-init` to prevent overwriting, which causes the otel metrics
provider to stop working.
# Additional Context
## Example Dashboards
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit c1535b7b49)
# Which Problems Are Solved
It was not possible to use the `api.v1.appendUserGrant` function in the
`postCreation` trigger action as documented.
# How the Problems Are Solved
- Correctly initialize the javascript / Goja function
- Added `projectID` and `projectGrantID` (as documented), but kept
`projectId` and `projectGrantId` (for backwards compatibility) when
mapping the object in the `appendUserGrant` function
# Additional Changes
None
# Additional Context
- A customer reached out to support regarding this issue.
- requires back port to 2.70.x
(cherry picked from commit 57bfecf7f7)
# Which Problems Are Solved
When we recently changed some permission for the `SYSTEM_OWNER` role on
QA, we noticed that there we multiple `permission.added` even when we
removed specific permissions.
# How the Problems Are Solved
Fixed the event type when removing permissions.
# Additional Changes
None
# Additional Context
Noticed when rolling out some changes on QA
(cherry picked from commit e7c7f9b800)
# Which Problems Are Solved
The service name is hardcoded in the metrics code. Making the service
name to be configurable helps when running multiple instances of
Zitadel.
The defaults remain unchanged, the service name will be defaulted to
ZITADEL.
# How the Problems Are Solved
Add a config option to override the name in defaults.yaml and pass it
down to the corresponding metrics or tracing module (google or otel)
# Additional Changes
NA
# Additional Context
NA
(cherry picked from commit dc64e35128)
# Which Problems Are Solved
The milestones query returns multiple results for every milestone for
every instance domain.
# How the Problems Are Solved
Corrected where condition on milestone query.
# Additional Changes
None
# Additional Context
None
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 352fa6aa6f)
# Which Problems Are Solved
Quota notification integration test failed sometimes due to eventual
consistency issues, which resulted in calls which should have been
counted to the quota not being added. This resulted in flaky integration
tests as the expected API calls to be limited were executed normally.
# How the Problems Are Solved
As there is no API call to query the currently applied Quota, there was
a sleep added as a last effort, to give some time that the event gets
processed into the projection.
# Additional Changes
None
# Additional Context
Related to
https://github.com/zitadel/zitadel/actions/runs/13922326003/job/38959595055
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 5ca76af779)
# Which Problems Are Solved
With current provided telemetry it's difficult to predict when a
projection handler is under increased load until it's too late and
causes downstream issues. Importantly, projection updating is in the
critical path for many login flows and increased latency there can
result in system downtime for users.
# How the Problems Are Solved
This PR adds three new prometheus-style metrics:
1. **projection_events_processed** (_labels: projection, success_) -
This metric gives us a counter of the number of events processed per
projection update run and whether they we're processed without error. A
high number of events being processed can let us know how busy a
particular projection handler is.
2. **projection_handle_timer** _(labels: projection)_ - This is the time
it takes to process a projection update given a batch of events - time
to take the current_states lock, query for new events, reduce,
update_the projection, and update current_states.
3. **projection_state_latency** _(labels: projection)_ - This is the
time from the last event processed in the current_states table for a
given projection. It tells us how old was the last event you processed?
Or, how far behind are you running for this projection? Higher latencies
could mean high load or stalled projection handling.
# Additional Changes
I also had to initialize the global otel metrics provider (`metrics.M`)
in the `setup` step additionally to `start` since projection handlers
are initialized at setup. The initialization checks if a metrics
provider is already set (in case of `start-from-setup` or
`start-from-init` to prevent overwriting, which causes the otel metrics
provider to stop working.
# Additional Context
## Example Dashboards
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
When using implicit flow through the session API and a login UI on a
custom domain (proxy), the tokens were signed by the API domain of the
instance, rather than the public (proxy) domain.
The SAML response had the same issue. Additionally, the saml library had
an issue and lost the issuer context. This prevented also a successful
login through the hosted login UI.
# How the Problems Are Solved
- The issuer of the SAML and Auth request is persisted to provide the
information when signing the responses and tokens.
- The SAML library is updated to the latest version.
# Additional Changes
None
# Additional Context
None
# Which Problems Are Solved
It was not possible to use the `api.v1.appendUserGrant` function in the
`postCreation` trigger action as documented.
# How the Problems Are Solved
- Correctly initialize the javascript / Goja function
- Added `projectID` and `projectGrantID` (as documented), but kept
`projectId` and `projectGrantId` (for backwards compatibility) when
mapping the object in the `appendUserGrant` function
# Additional Changes
None
# Additional Context
- A customer reached out to support regarding this issue.
- requires back port to 2.70.x
