# Which Problems Are Solved
- SCIM user metadata mapping keys have differing case styles.
# How the Problems Are Solved
- key casing style is unified to strict camelCase
# Additional Context
Part of #8140
Although this is technically a breaking change, it is considered
acceptable because the SCIM feature is still in the preview stage and
not fully implemented yet.
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
* Adds support for the patch user SCIM v2 endpoint
# How the Problems Are Solved
* Adds support for the patch user SCIM v2 endpoint under `PATCH
/scim/v2/{orgID}/Users/{id}`
# Additional Context
Part of #8140
# Which Problems Are Solved
The membership fields migration timed out in certain cases. It also
tried to migrate instances which were already removed.
# How the Problems Are Solved
Revert the previous fix that combined the repeatable step for multiple
fill triggers. The membeship migration is now single-run as it might
take a lot of time. It is not worth making it repeatable. Instance IDs
of removed instances are skipped.
# Additional Changes
None
# Additional Context
Introduced in https://github.com/zitadel/zitadel/pull/9199
# Which Problems Are Solved
The subquery of the notification requested and retry requested is
missing the aggregate_type filter that would allow it to utilize the
`es_projection` or `active_instances_events` on the eventstore.events2
table.
# How the Problems Are Solved
Add additional filter on subquery. Final query:
```sql
SELECT <all the fields omitted> FROM eventstore.events2
WHERE
instance_id = $1
AND aggregate_type = $2
AND event_type = $3
AND created_at > $4
AND aggregate_id NOT IN (
SELECT aggregate_id
FROM eventstore.events2
WHERE
aggregate_type = $5 <-- NB: previously missing
AND event_type = ANY ($6)
AND instance_id = $7
AND created_at > $8
)
ORDER BY "position", in_tx_order
LIMIT $9
FOR UPDATE SKIP LOCKED
```
# Additional Changes
# Additional Context
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
A customer reached out to support, that the (last) `additional origin`
could not be removed. While testing / implementation it was discovered,
that the same applied to `redirect_uris` and `post_logout_redirect_uris`
# How the Problems Are Solved
- Correctly set the corresponding array to empty in the event so it can
be differentiated to `null` / not set in case of no change.
# Additional Changes
Replaced `reflect.DeepEqual` with `slices.Equal`
# Additional Context
- Reported to support
# Which Problems Are Solved
- Adds support for the list users SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the list users SCIM v2 endpoints under `GET
/scim/v2/{orgID}/Users` and `POST /scim/v2/{orgID}/Users/.search`
# Additional Changes
- adds a new function `SearchUserMetadataForUsers` to the query layer to
query a metadata keyset for given user ids
- adds a new function `NewUserMetadataExistsQuery` to the query layer to
query a given metadata key value pair exists
- adds a new function `CountUsers` to the query layer to count users
without reading any rows
- handle `ErrorAlreadyExists` as scim errors `uniqueness`
- adds `NumberLessOrEqual` and `NumberGreaterOrEqual` query comparison
methods
- adds `BytesQuery` with `BytesEquals` and `BytesNotEquals` query
comparison methods
# Additional Context
Part of #8140
Supported fields for scim filters:
* `meta.created`
* `meta.lastModified`
* `id`
* `username`
* `name.familyName`
* `name.givenName`
* `emails` and `emails.value`
* `active` only eq and ne
* `externalId` only eq and ne
# Which Problems Are Solved
Small update to docs/docs/concepts/features/selfservice.md to fix issue
in grammar
Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
# Which Problems Are Solved
Memberships did not have a fields table fill migration.
# How the Problems Are Solved
Add filling of membership fields to the repeatable steps.
# Additional Changes
- Use the same repeatable step for multiple fill fields handlers.
- Fix an error for PostgreSQL 15 where a subquery in a `FROM` clause
needs an alias ing the `permitted_orgs` function.
# Additional Context
- Part of https://github.com/zitadel/zitadel/issues/9188
- Introduced in https://github.com/zitadel/zitadel/pull/9152
# Which Problems Are Solved
https://github.com/zitadel/zitadel/pull/9186 introduced the new `push`
sql function for cockroachdb. The function used the wrong database
function to generate the position of the event and would therefore
insert events at a position before events created with an old Zitadel
version.
# How the Problems Are Solved
Instead of `EXTRACT(EPOCH FROM NOW())`, `cluster_logical_timestamp()` is
used to calculate the position of an event.
# Additional Context
- Introduced in https://github.com/zitadel/zitadel/pull/9186
- Affected versions:
https://github.com/zitadel/zitadel/releases/tag/v2.67.3
# Which Problems Are Solved
If a notification channel was not present, notification workers would
retry to the max attempts. This leads to unnecessary load.
Additionally, a client noticed bad actors trying to abuse SMS MFA.
# How the Problems Are Solved
- Directly cancel the notification on:
- a missing channel and stop retries.
- any `4xx` errors from Twilio Verify
# Additional Changes
None
# Additional Context
reported by customer
# Which Problems Are Solved
Organization name change results in domain events even if the domain
itself doesn't change.
# How the Problems Are Solved
Check if the domain itself really changes, and if not, don't create the
events.
# Additional Changes
Unittest for this specific case.
# Additional Context
None
# Which Problems Are Solved
Zitadel currently uses 3 database pool, 1 for queries, 1 for pushing
events and 1 for scheduled projection updates. This defeats the purpose
of a connection pool which already handles multiple connections.
During load tests we found that the current structure of connection
pools consumes a lot of database resources. The resource usage dropped
after we reduced the amount of database pools to 1 because existing
connections can be used more efficiently.
# How the Problems Are Solved
Removed logic to handle multiple connection pools and use a single one.
# Additional Changes
none
# Additional Context
part of https://github.com/zitadel/zitadel/issues/8352
# Which Problems Are Solved
- include tzdata in the binary to correctly validate time zones in the
scim layer if the os doesn't have timezone data available.
# How the Problems Are Solved
- by importing the go pkg `"time/tzdata"`
# Additional Context
Part of https://github.com/zitadel/zitadel/issues/8140
# Which Problems Are Solved
Currently ZITADEL defines organization and instance member roles and
permissions in defaults.yaml. The permission check is done on API call
level. For example: "is this user allowed to make this call on this
org". This makes sense on the V1 API where the API is permission-level
shaped. For example, a search for users always happens in the context of
the organization. (Either the organization the calling user belongs to,
or through member ship and the x-zitadel-orgid header.
However, for resource based APIs we must be able to resolve permissions
by object. For example, an IAM_OWNER listing users should be able to get
all users in an instance based on the query filters. Alternatively a
user may have user.read permissions on one or more orgs. They should be
able to read just those users.
# How the Problems Are Solved
## Role permission mapping
The role permission mappings defined from `defaults.yaml` or local
config override are synchronized to the database on every run of
`zitadel setup`:
- A single query per **aggregate** builds a list of `add` and `remove`
actions needed to reach the desired state or role permission mappings
from the config.
- The required events based on the actions are pushed to the event
store.
- Events define search fields so that permission checking can use the
indices and is strongly consistent for both query and command sides.
The migration is split in the following aggregates:
- System aggregate for for roles prefixed with `SYSTEM`
- Each instance for roles not prefixed with `SYSTEM`. This is in
anticipation of instance level management over the API.
## Membership
Current instance / org / project membership events now have field table
definitions. Like the role permissions this ensures strong consistency
while still being able to use the indices of the fields table. A
migration is provided to fill the membership fields.
## Permission check
I aimed keeping the mental overhead to the developer to a minimal. The
provided implementation only provides a permission check for list
queries for org level resources, for example users. In the `query`
package there is a simple helper function `wherePermittedOrgs` which
makes sure the underlying database function is called as part of the
`SELECT` query and the permitted organizations are part of the `WHERE`
clause. This makes sure results from non-permitted organizations are
omitted. Under the hood:
- A Pg/PlSQL function searches for a list of organization IDs the passed
user has the passed permission.
- When the user has the permission on instance level, it returns early
with all organizations.
- The functions uses a number of views. The views help mapping the
fields entries into relational data and simplify the code use for the
function. The views provide some pre-filters which allow proper index
usage once the final `WHERE` clauses are set by the function.
# Additional Changes
# Additional Context
Closes#9032
Closes https://github.com/zitadel/zitadel/issues/9014https://github.com/zitadel/zitadel/issues/9188 defines follow-ups for
the new permission framework based on this concept.
# Which Problems Are Solved
The performance of the initial push function can further be increased
# How the Problems Are Solved
`eventstore.push`- and `eventstore.commands_to_events`-functions were
rewritten
# Additional Changes
none
# Additional Context
same optimizations as for postgres:
https://github.com/zitadel/zitadel/pull/9092
# Which Problems Are Solved
As we are going into the Beta testing phase of our new typescript login
(login V2), we need to have a documentation about the capabilities, how
to test, and what the current limitations are.
# How the Problems Are Solved
Added new section for the Login V2
---------
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Max Peintner <peintnerm@gmail.com>
# Which Problems Are Solved
The current login will always prefer external authentication (through an
IdP) over local authentication. So as soon as either the user had
connected to an IdP or even when the login policy was just set up to
have an IdP allowed, users would be redirected to that IdP for
(re)authentication.
This could lead to problems, where the IdP was not available or any
other error occurred in the process (such as secret expired for
EntraID).
Even when local authentication (passkeys or password) was allowed for
the corresponding user, they would always be redirected to the IdP
again, preventing any authentication. If admins were affected, they
might not even be able to update the client secret of the IdP.
# How the Problems Are Solved
Errors during the external IdP flow are handled in an
`externalAuthFailed` function, which will check if the organisation
allows local authentication and if the user has set up such.
If either password or passkeys is set up, the corresponding login page
will be presented to the user. As already with local auth passkeys is
preferred over password authentication.
The user is informed that the external login failed and fail back to
local auth as an error on the corresponding page in a focused mode. Any
interaction or after 5 second the focus mode is disabled.
# Additional Changes
None.
# Additional Context
closes#6466
# Which Problems Are Solved
In versions previous to v2.66 it was possible to set a different
resource owner on project grants. This was introduced with the new
resource based API. The resource owner was possible to overwrite using
the x-zitadel-org header.
Because of this issue project grants got the wrong resource owner,
instead of the owner of the project it got the granted org which is
wrong because a resource owner of an aggregate is not allowed to change.
# How the Problems Are Solved
- The wrong owners of the events are set to the original owner of the
project.
- A new event is pushed to these aggregates `project.owner.corrected`
- The projection updates the owners of the user grants if that event was
written
# Additional Changes
The eventstore push function (replaced in version 2.66) writes the
correct resource owner.
# Additional Context
closes https://github.com/zitadel/zitadel/issues/9072
Update the ../proto/zitadel/member.proto to
include the UserResourceOwner as part of member.
Update the queries to include UserResourceOwner
for the following :
zitadel/internal/query/iam_member.go
zitadel/internal/query/org_member.go
zitadel/internal/query/project_member.go
zitadel/internal/query/project_grant_member.go
Non Breaking Changes
# Which Problems Are Solved
https://github.com/zitadel/zitadel/issues/5062
# How the Problems Are Solved
- Updated the member.proto file to include user_resource_owner. I have
compiled using` make compile` command .
- Changed the queries to include the userResourceOwner as part of
Member.
- Then, updated the converter to map the userResourceOwner.
# Additional Changes
Replace this example text with a concise list of additional changes that
this PR introduces, that are not directly solving the initial problem
but are related.
For example:
- The docs explicitly describe that the property XY is mandatory
- Adds missing translations for validations.
# Additional Context
- Closes#5062
-
https://discordapp.com/channels/927474939156643850/1326245856193544232/1326476710752948316
# Which Problems Are Solved
- Adds support for the replace user SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the replace user SCIM v2 endpoint under `PUT
/scim/v2/{orgID}/Users/{id}`
# Additional Changes
- Respect the `Active` field in the SCIM v2 create user endpoint `POST
/scim/v2/{orgID}/Users`
- Eventually consistent read endpoints used in SCIM tests are wrapped in
`assert.EventuallyWithT` to work around race conditions
# Additional Context
Part of #8140
# Which Problems Are Solved
ListSessions only works to list the sessions that you are the creator
of.
# How the Problems Are Solved
Add options to search for sessions created by other users, sessions
belonging to the same useragent and sessions belonging to your user.
Possible through additional search parameters which as default use the
information contained in your session token but can also be filled with
specific IDs.
# Additional Changes
Remodel integration tests, to separate the Create and Get of sessions
correctly.
# Additional Context
Closes#8301
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Wrongly not returned but documented 'nbf' claim in ID token.
# How the Problems Are Solved
Correct documentation to not include 'nbf' in ID token.
# Additional Changes
None
# Additional Context
None
# Which Problems Are Solved
- Adds support for the get user SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the get user SCIM v2 endpoint under `GET
/scim/v2/{orgID}/Users/{id}`
# Additional Context
Part of #8140
Replaces https://github.com/zitadel/zitadel/pull/9154 as requested by
the maintainers, discussions see
https://github.com/zitadel/zitadel/pull/9154.
# Which Problems Are Solved
- Adds support for the user delete SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the user delete SCIM v2 endpoint under `DELETE
/scim/v2/{orgID}/Users/{id}`
# Additional Context
Part of #8140
# Which Problems Are Solved
- Adds infrastructure code (basic implementation, error handling,
middlewares, ...) to implement the SCIM v2 interface
- Adds support for the user create SCIM v2 endpoint
# How the Problems Are Solved
- Adds support for the user create SCIM v2 endpoint under `POST
/scim/v2/{orgID}/Users`
# Additional Context
Part of #8140
# Which Problems Are Solved
On Zitadel cloud we found changing the order of columns in the
`eventstore.events2_current_sequence` index improved CPU usage for the
`SELECT ... FOR UPDATE` query the pusher executes.
# How the Problems Are Solved
`eventstore.events2_current_sequence`-index got replaced
# Additional Context
closes https://github.com/zitadel/zitadel/issues/9082
# Which Problems Are Solved
We were seeing high query costs in a the lateral join executed in the
commands_to_events procedural function in the database. The high cost
resulted in incremental CPU usage as a load test continued and less
req/sec handled, sarting at 836 and ending at 130 req/sec.
# How the Problems Are Solved
1. Set `PARALLEL SAFE`. I noticed that this option defaults to `UNSAFE`.
But it's actually safe if the function doesn't `INSERT`
2. Set the returned `ROWS 10` parameter.
3. Function is re-written in Pl/PgSQL so that we eliminate expensive
joins.
4. Introduced an intermediate state that does `SELECT DISTINCT` for the
aggregate so that we don't have to do an expensive lateral join.
# Additional Changes
Use a `COALESCE` to get the owner from the last event, instead of a
`CASE` switch.
# Additional Context
- Function was introduced in
https://github.com/zitadel/zitadel/pull/8816
- Closes https://github.com/zitadel/zitadel/issues/8352
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
Events like "password check succeeded" store some information about the
caller including their IP.
The `X-Forwarded-For` was not correctly logged, but instead the
RemoteAddress.
# How the Problems Are Solved
- Correctly get the `X-Forwarded-For` in canonical form.
# Additional Changes
None
# Additional Context
closes [#9106](https://github.com/zitadel/zitadel/issues/9106)
# Which Problems Are Solved
It was possible to set a diffent algorithm for the legacy signer. This
is not supported howerver and breaks the token endpoint.
# How the Problems Are Solved
Remove the OIDC.SigningKeyAlgorithm config option and hard-code RS256
for the legacy signer.
# Additional Changes
- none
# Additional Context
Only RS256 is supported by the legacy signer. It was mentioned in the
comment of the config not to use it and use the webkeys resource
instead.
- closes#9121
# Which Problems Are Solved
Typo in RU localization on login page.
# How the Problems Are Solved
Fixed typo by replacing to correct text.
# Additional Changes
n/a
# Additional Context
n/a
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
If a not allowed IDP is selected or now not allowed IDP was selected
before at login, the login will still try to use it as fallback.
The same goes for the linked IDPs which are not necessarily active
anymore, or disallowed through policies.
# How the Problems Are Solved
Check all possible or configured IDPs if they can be used.
# Additional Changes
None
# Additional Context
Addition to #6466
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
get instance by domain cannot provide an instance id because it is not
known at that time. This causes a full table scan on the fields table
because current indexes always include the `instance_id` column.
# How the Problems Are Solved
Added a specific index for this query.
# Additional Context
If a system has many fields and there is no cache hit for the given
domain this query can heaviuly influence database CPU usage, the newly
added resolves this problem.
# Which Problems Are Solved
The console has no information about where and how to send PostHog
events.
# How the Problems Are Solved
A PostHog API URL and token are passed through as plain text from the
Zitadel runtime config to the environment.json. By default, no values
are configured and the keys in the environment.json are omitted.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9070
- Complements https://github.com/zitadel/zitadel/pull/9077
# Which Problems Are Solved
When `LastUseAge` was configured properly, the Redis LUA script uses
manual cleanup for `MaxAge` based expiry. The expiry obtained from Redis
apears to be a string and was compared to an int, resulting in a script
error.
# How the Problems Are Solved
Convert expiry to number.
# Additional Changes
- none
# Additional Context
- Introduced in #8822
- LastUseAge was fixed in #9097
- closes https://github.com/zitadel/zitadel/issues/9140
Added a note on Zitadel support in Dribdat, which explicitly mentions it
in the [install notes](https://dribdat.cc/deploy.html#authentication)
and soon in a blog post or screencast.
---------
Co-authored-by: Swarna Podila <swarnap@users.noreply.github.com>
# Which Problems Are Solved
IdPs using form callback were not always correctly handled with the
newly introduced cache mechanism
(https://github.com/zitadel/zitadel/pull/9097).
# How the Problems Are Solved
Get the data from cache before parsing it.
# Additional Changes
None
# Additional Context
Relates to https://github.com/zitadel/zitadel/pull/9097
# Which Problems Are Solved
Some IdP callbacks use HTTP form POST to return their data on callbacks.
For handling CSRF in the login after such calls, a 302 Found to the
corresponding non form callback (in ZITADEL) is sent. Depending on the
size of the initial form body, this could lead to ZITADEL terminating
the connection, resulting in the user not getting a response or an
intermediate proxy to return them an HTTP 502.
# How the Problems Are Solved
- the form body is parsed and stored into the ZITADEL cache (using the
configured database by default)
- the redirect (302 Found) is performed with the request id
- the callback retrieves the data from the cache instead of the query
parameters (will fallback to latter to handle open uncached requests)
# Additional Changes
- fixed a typo in the default (cache) configuration: `LastUsage` ->
`LastUseAge`
# Additional Context
- reported by a customer
- needs to be backported to current cloud version (2.66.x)
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
Commands for installing compose stacks with reverse proxies don't work.
# How the Problems Are Solved
- The `docker compose up` commands are fixed by specifying all necessary
services to spin up. This is obviously not (or not with all docker
compose versions) resolved by the dependencies declarations.
- The initial postgres admin username is postgres.
- Fix postgres health check to succeed before the init job created the
DB.
- A hint tells the user to install the grpcurl binary.
# Additional Changes
- Passing `--wait` to `docker compose up` doesn't require us to sleep
for exactly three seconds.
- It looks to me like the order of the depends_on declaration for
zitadel matters, but I don't understand why. I changed it so that it's
for sure correct.
- Silenced some command outputs
- Removed the version property from all compose files to avoid the
following warning
```
WARN[0000] /tmp/caddy-example/docker-compose-base.yaml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion
```
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/9115
This is the easiest way to test the updated docs:
```bash
# Use this PR branches files:
export ZITADEL_CONFIG_FILES=https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/fix-reverse-proxy-guides/docs/docs/self-hosting/manage/reverseproxy
```
The rest of the commands as described in
https://docs-git-fix-reverse-proxy-guides-zitadel.vercel.app/docs/self-hosting/manage/reverseproxy/caddy
# Which Problems Are Solved
The v2 api currently has no endpoint the get all second factors of a
user.
# How the Problems Are Solved
Our v1 api has the ListHumanAuthFactors which got added to the v2 api
under the User resource.
# Additional Changes
# Additional Context
Closes#8833
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Added search by phone to user Service V2.
```
curl --request POST \
--url https://<zitadel_domain>/v2/users \
--header 'Accept: application/json' \
--header 'Authorization: Bearer <Token>' \
--header 'Content-Type: application/json' \
--header 'content-type: application/json' \
--data '{
"query": {
"offset": "0",
"limit": 100,
"asc": true
},
"sortingColumn": "USER_FIELD_NAME_UNSPECIFIED",
"queries": [
{
"phoneQuery": {
"number": "+12011223313",
"method": "TEXT_QUERY_METHOD_EQUALS"
}
}
]
}'
```
Why?
Searching for a user by phone was missing from User Service V2 and V2
beta.
# How the Problems Are Solved
* Added to the SearchQuery proto
* Added code to filter users by phone
# Additional Changes
N/A
# Additional Context
Search by phone is present in V3 User Service
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
There is currently no endpoint to send an email code for verification of
the email if you don't change the email itself.
# How the Problems Are Solved
Endpoint HasEmailCode to get the information that an email code is
existing, used by the new login.
Endpoint SendEmailCode, if no code is existing to replace
ResendEmailCode as there is a check that a code has to be there, before
it can be resend.
# Additional Changes
None
# Additional Context
Closes#9096
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
SAML IdPs exposing an `EntitiesDescriptor` using an `xsd:duration` time
format for the `cacheDuration` property (e.g. `PT5H`) failed parsing.
# How the Problems Are Solved
Handle the unmarshalling for `EntitiesDescriptor` specifically.
[crewjam/saml](bbccb7933d/metadata.go (L88-L103))
already did this for `EntitiyDescriptor` the same way.
# Additional Changes
None
# Additional Context
- reported by a customer
- needs to be backported to current cloud version (2.66.x)
# Which Problems Are Solved
It is currently not possible to use SAML with the Session API.
# How the Problems Are Solved
Add SAML service, to get and resolve SAML requests.
Add SAML session and SAML request aggregate, which can be linked to the
Session to get back a SAMLResponse from the API directly.
# Additional Changes
Update of dependency zitadel/saml to provide all functionality for
handling of SAML requests and responses.
# Additional Context
Closes#6053
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
