# Which Problems Are Solved
https://github.com/zitadel/zitadel/pull/8788 accidentally changed the
spelling of milestone types from PascalCase to snake_case. This breaks
systems where `milestone.pushed` events already exist.
# How the Problems Are Solved
- Use PascalCase again
- Prefix event types with v2. (Previous pushed event type was anyway
ignored).
- Create `milstones3` projection
# Additional Changes
None
# Additional Context
relates to #8788
# Which Problems Are Solved
Add a cache implementation using Redis single mode. This does not add
support for Redis Cluster or sentinel.
# How the Problems Are Solved
Added the `internal/cache/redis` package. All operations occur
atomically, including setting of secondary indexes, using LUA scripts
where needed.
The [`miniredis`](https://github.com/alicebob/miniredis) package is used
to run unit tests.
# Additional Changes
- Move connector code to `internal/cache/connector/...` and remove
duplicate code from `query` and `command` packages.
- Fix a missed invalidation on the restrictions projection
# Additional Context
Closes#8130
# Which Problems Are Solved
Currently ZITADEL supports RP-initiated logout for clients. Back-channel
logout ensures that user sessions are terminated across all connected
applications, even if the user closes their browser or loses
connectivity providing a more secure alternative for certain use cases.
# How the Problems Are Solved
If the feature is activated and the client used for the authentication
has a back_channel_logout_uri configured, a
`session_logout.back_channel` will be registered. Once a user terminates
their session, a (notification) handler will send a SET (form POST) to
the registered uri containing a logout_token (with the user's ID and
session ID).
- A new feature "back_channel_logout" is added on system and instance
level
- A `back_channel_logout_uri` can be managed on OIDC applications
- Added a `session_logout` aggregate to register and inform about sent
`back_channel` notifications
- Added a `SecurityEventToken` channel and `Form`message type in the
notification handlers
- Added `TriggeredAtOrigin` fields to `HumanSignedOut` and
`TerminateSession` events for notification handling
- Exported various functions and types in the `oidc` package to be able
to reuse for token signing in the back_channel notifier.
- To prevent that current existing session termination events will be
handled, a setup step is added to set the `current_states` for the
`projections.notifications_back_channel_logout` to the current position
- [x] requires https://github.com/zitadel/oidc/pull/671
# Additional Changes
- Updated all OTEL dependencies to v1.29.0, since OIDC already updated
some of them to that version.
- Single Session Termination feature is correctly checked (fixed feature
mapping)
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8467
- TODO:
- Documentation
- UI to be done: https://github.com/zitadel/zitadel/issues/8469
---------
Co-authored-by: Hidde Wieringa <hidde@hiddewieringa.nl>
# Which Problems Are Solved
Migration of milestones failed on our QA due to the new milestone Type
enum being 0-indexed. The valid range was 0 till 5, inclusive. While on
the previous zitadel version this was 1 till 6, inclusive.
# How the Problems Are Solved
Offset the first constant with `1`.
# Additional Changes
- none
# Additional Context
Introduced in https://github.com/zitadel/zitadel/pull/8788
Closes#8838
This fixes a bug of the `/register/org` page where scripts where not
referenced correctly
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Resource owner can be different than expected if the provided
x-zitadel-orgid header is provided.
# How the Problems Are Solved
Check that the project is only checked with the correct resource owner
to avoid unexpected situations.
# Additional Changes
None
# Additional Context
Closes#8685
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Milestones used existing events from a number of aggregates. OIDC
session is one of them. We noticed in load-tests that the reduction of
the oidc_session.added event into the milestone projection is a costly
business with payload based conditionals. A milestone is reached once,
but even then we remain subscribed to the OIDC events. This requires the
projections.current_states to be updated continuously.
# How the Problems Are Solved
The milestone creation is refactored to use dedicated events instead.
The command side decides when a milestone is reached and creates the
reached event once for each milestone when required.
# Additional Changes
In order to prevent reached milestones being created twice, a migration
script is provided. When the old `projections.milestones` table exist,
the state is read from there and `v2` milestone aggregate events are
created, with the original reached and pushed dates.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8800
# Which Problems Are Solved
Searching orgs by domain currently only looked for the primary domain,
but should be possible with all verified domains (as documented)
# How the Problems Are Solved
- fixed the search query
# Additional Changes
None
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8749
# Which Problems Are Solved
In integration tests there is waiting for the application, but the
project is also included if the token can be created.
# How the Problems Are Solved
Wait for project not only for the application in the integration tests.
# Additional Changes
Some more corrections in integration tests.
# Additional Context
None
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
System administrators can block hosts and IPs for HTTP calls in actions.
Using DNS, blocked IPs could be bypassed.
# How the Problems Are Solved
- Hosts are resolved (DNS lookup) to check whether their corresponding
IP is blocked.
# Additional Changes
- Added complete lookup ip address range and "unspecified" address to
the default `DenyList`
(cherry picked from commit 79fb4cc1cc)
# Which Problems Are Solved
Eventual consistency is handled wrongly in the newly improved
integration tests.
# How the Problems Are Solved
Correct the usage of the require package with the assert package where
necessary, to remove the panics where the EventuallyWithT functions can
rerun.
# Additional Changes
Modify the timeout values for some EventuallyWithT which can vary when a
instance is freshly setup.
# Additional Context
None
(cherry picked from commit fca6b28a97)
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
(cherry picked from commit 8d97363642)
- **docs: s/Secondfactor/Second factor/**
- **docs: s/IDP/IdP/**
- **docs: s/Hardwaretokens/Hardware tokens/**
- **docs: standardize multi-factor vs multi factor vs multifactor**
# Which Problems Are Solved
- English strings are improved
# How the Problems Are Solved
- With better strings
---------
Co-authored-by: Fabi <fabienne@zitadel.com>
# Which Problems Are Solved
System administrators can block hosts and IPs for HTTP calls in actions.
Using DNS, blocked IPs could be bypassed.
# How the Problems Are Solved
- Hosts are resolved (DNS lookup) to check whether their corresponding
IP is blocked.
# Additional Changes
- Added complete lookup ip address range and "unspecified" address to
the default `DenyList`
# Which Problems Are Solved
Eventual consistency is handled wrongly in the newly improved
integration tests.
# How the Problems Are Solved
Correct the usage of the require package with the assert package where
necessary, to remove the panics where the EventuallyWithT functions can
rerun.
# Additional Changes
Modify the timeout values for some EventuallyWithT which can vary when a
instance is freshly setup.
# Additional Context
None
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
# Which Problems Are Solved
There is currently the possibility that you can jump to the register
path, even if register is disallowed through the settings.
# How the Problems Are Solved
Check before handling the HTTP requests if register is allowed.
# Additional Changes
Function to determine the resourceowner for all register related
functionality in the login.
# Additional Context
closes#8123
# Which Problems Are Solved
There is currently the possibility that you can jump to the register
path, even if register is disallowed through the settings.
# How the Problems Are Solved
Check before handling the HTTP requests if register is allowed.
# Additional Changes
Function to determine the resourceowner for all register related
functionality in the login.
# Additional Context
closes#8123
- Fully translated all UI elements, documentation, and error messages
- Added Hungarian as a new supported language option
- Updated language selection menus and related configuration files
- Ensured consistency across all translated content
# Which Problems Are Solved
- ZITADEL was not accessible for Hungarian-speaking users due to lack of
language support
- Hungarian users had to rely on English or other languages to use the
platform
- Potential user base was limited due to language barrier
# How the Problems Are Solved
- Translated all user interface elements, including console and login
interfaces
- Translated all documentation files to Hungarian
- Added Hungarian translations for all error messages and notifications
- Implemented Hungarian as a selectable language option in the system
# Additional Changes
- Updated language selection menus to include Hungarian
- Modified configuration files to support Hungarian language
- Ensured consistent terminology and style across all translated content
- Added Hungarian language option to relevant dropdown menus and
settings
# Additional Context
- Relates to the ongoing internationalization efforts of ZITADEL
- Enhances accessibility for Hungarian-speaking developers and users
- Expands ZITADEL's potential user base in Hungary and
Hungarian-speaking regions
---------
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Adding a SAML IDPs with an empty metadata XML and URL leads to failed
projection events. The main IDP template projection succeeds, but the
subtable projection for SAML template fails, because the metadata field
is not nullable in that table. The SAML IDP shows up in list queries,
because the list method only queries the main template projection.
However, getting a SAML IDP created without metadata by ID misses the
SAML specific IDP data.
# How the Problems Are Solved
- The command for adding a SAML IDP properly ensures that non-empty
metadata is either given by XML or resolved by URL.
- The console doesn't send requests with empty metadata anymore. This
works by explicitly setting a single oneof option for either XML or URL
and emptying the other one.
# Additional Context
Closes#8443
# Which Problems Are Solved
If SAML response validation in crewjam/saml fails, a generic
"Authentication failed" error is thrown. This makes it challenging to
determine the actual cause, since there are a variety of reasons
response validation may fail.
# How the Problems Are Solved
Add a log statement if we receive a response validation error from
crewjam/saml that logs the internal `InvalidResponseError.PrivateErr`
error from crewjam/saml to stdout. We continue to return a generic error
message to the client to prevent leaking data.
Verified by running `go test -v ./internal/idp/providers/saml` in
verbose mode, which output the following line for the "response_invalid"
test case:
```
time="2024-10-03T14:53:10+01:00" level=info msg="invalid SAML response details" caller="/Users/sdouglas/Documents/thirdparty-repos/zitadel/internal/idp/providers/saml/session.go:72" error="cannot parse base64: illegal base64 data at input byte 2"
```
# Additional Changes
None
# Additional Context
- closes#8717
---------
Co-authored-by: Stuart Douglas <sdouglas@hopper.com>
# Which Problems Are Solved
There is no option to only query auth methods related to specific
domains.
# How the Problems Are Solved
Add domain as attribute to the ListAuthenticationMethodTypes request.
# Additional Changes
OwnerRemoved column removed from the projection.
# Additional Context
Closes#8615
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
If an organization was preselected using an orgID or primaryDomain
scope, users could still switch to another organization, if the latter
allowed domain discovery and the entered username / or login_hint
included the corresponding domain suffix.
# How the Problems Are Solved
Domain discovery will only be done in case no org was preselected.
# Additional Changes
None
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8464
- closes https://github.com/zitadel/zitadel/issues/8588
# Which Problems Are Solved
OTP Email links currently could not use / include the sessionID they
belong to. This prevents an easy use for redirecting and handling OTP
via email through the session API.
# How the Problems Are Solved
Added the sessionID as placeholder for the OTP Email link template.
# Additional Changes
List all available placeholders in the url_templates of V2 endpoints.
# Additional Context
- discussed in a customer meeting
# Which Problems Are Solved
Admins can set the branding to be used from the project's organization.
Until now, only the branding (colors) were respected, but texts were
still loaded from the user's organization.
# How the Problems Are Solved
Respect the setting when loading the texts for the login pages.
# Additional Changes
None
# Additional Context
- closes#8502
# Which Problems Are Solved
Since the optiimzation of the token endpoint, we longer push the
`user.token.added` event. However, the milestone projection keeps
quering for it, including a payload query.
This incured a static waste of DB resources.
# How the Problems Are Solved
Remove the `user.token.added` event reducer from the milestone
projection
# Additional Changes
- none
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/8742. Other
changes ommitted so this PR can be backported to stable.
# Which Problems Are Solved
There is no check that the creation of a project also has a resource
owner.
# How the Problems Are Solved
Check if resource owner and the ID of the owner are provided on command
side.
# Additional Changes
None
# Additional Context
Closes#7794
# Which Problems Are Solved
A customer reported, that after a created user (in initial state) got
(manually) locked and a new initial code would be created, the user
could not be locked again.
# How the Problems Are Solved
Query for the initial code added event
# Additional Changes
None
# Additional Context
- reported by a customer
# Which Problems Are Solved
Optimize the query that checks for terminated sessions in the access
token verifier. The verifier is used in auth middleware, userinfo and
introspection.
# How the Problems Are Solved
The previous implementation built a query for certain events and then
appended a single `PositionAfter` clause. This caused the postgreSQL
planner to use indexes only for the instance ID, aggregate IDs,
aggregate types and event types. Followed by an expensive sequential
scan for the position. This resulting in internal over-fetching of rows
before the final filter was applied.
Furthermore, the query was searching for events which are not always
applicable. For example, there was always a session ID search and if
there was a user ID, we would also search for a browser fingerprint in
event payload (expensive). Even if those argument string would be empty.
This PR changes:
1. Nest the position query, so that a full `instance_id, aggregate_id,
aggregate_type, event_type, "position"` index can be matched.
2. Redefine the `es_wm` index to include the `position` column.
3. Only search for events for the IDs that actually have a value. Do not
search (noop) if none of session ID, user ID or fingerpint ID are set.
New query plan:
# Additional Changes
- cleanup how we load multi-statement migrations and make that a bit
more reusable.
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/7639
# Which Problems Are Solved
To get the default organization we have no direct solution.
# How the Problems Are Solved
Add default organization as query to the org v2 List.
# Additional Changes
None
# Additional Context
Closes#8616
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
The recently added possibility to generate and verify codes through
Twilio verification service did failed on checking OTP SMS code through
the session API. Additionally, password codes generated by the V2 API
and sent through phone would always use the internal generator and
verification mechanism rather than the configured.
# How the Problems Are Solved
- Correctly set the verifier for OTP SMS for the session API
- Always use the internal verifier for OTP Email (for now)
- Select the generator / verifier based on the configuration for
password codes with notification type SMS for V2 APIs
# Additional Changes
None
# Additional Context
- relates to #8678
- reported by customer
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
(cherry picked from commit f653589609)
# Which Problems Are Solved
The recently added possibility to generate and verify codes through
Twilio verification service did failed on checking OTP SMS code through
the session API. Additionally, password codes generated by the V2 API
and sent through phone would always use the internal generator and
verification mechanism rather than the configured.
# How the Problems Are Solved
- Correctly set the verifier for OTP SMS for the session API
- Always use the internal verifier for OTP Email (for now)
- Select the generator / verifier based on the configuration for
password codes with notification type SMS for V2 APIs
# Additional Changes
None
# Additional Context
- relates to #8678
- reported by customer
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Cache implementation using a PGX connection pool.
# How the Problems Are Solved
Defines a new schema `cache` in the zitadel database.
A table for string keys and a table for objects is defined.
For postgreSQL, tables are unlogged and partitioned by cache name for
performance.
Cockroach does not have unlogged tables and partitioning is an
enterprise feature that uses alternative syntax combined with sharding.
Regular tables are used here.
# Additional Changes
- `postgres.Config` can return a pxg pool. See following discussion
# Additional Context
- Part of https://github.com/zitadel/zitadel/issues/8648
- Closes https://github.com/zitadel/zitadel/issues/8647
---------
Co-authored-by: Silvan <silvan.reusser@gmail.com>
# Which Problems Are Solved
https://github.com/zitadel/zitadel/pull/8545 incorrectly created the
list of current smtp providers, if an SMTP provider was changed, that
was created before https://github.com/zitadel/zitadel/pull/6932 /
[v2.50.0](https://github.com/zitadel/zitadel/releases/tag/v2.50.0)).
This led to problems when trying to send emails to users (email
verification and OTP email).
# How the Problems Are Solved
Correctly handle events of old SMTP configurations, which do not have an
id set.
# Additional Changes
None
# Additional Context
- relates to #8545
- support requests from cloud customers
(cherry picked from commit 0bcf136f6f)
# Which Problems Are Solved
https://github.com/zitadel/zitadel/pull/8545 incorrectly created the
list of current smtp providers, if an SMTP provider was changed, that
was created before https://github.com/zitadel/zitadel/pull/6932 /
[v2.50.0](https://github.com/zitadel/zitadel/releases/tag/v2.50.0)).
This led to problems when trying to send emails to users (email
verification and OTP email).
# How the Problems Are Solved
Correctly handle events of old SMTP configurations, which do not have an
id set.
# Additional Changes
None
# Additional Context
- relates to #8545
- support requests from cloud customers
# Which Problems Are Solved
Errors messages are mixed up for some policies
# How the Problems Are Solved
Define new error IDs and correct the messages.
# Additional Changes
None
# Additional Context
None
# Which Problems Are Solved
If a SAML IdP did not send a `NameID` (even though required by the
specification), ZITADEL would crash.
# How the Problems Are Solved
- Check specifically if the `Subject` and its `NameID` is passed
# Additional Changes
None
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8654
(cherry picked from commit 18499274dd)
# Which Problems Are Solved
If a SAML IdP did not send a `NameID` (even though required by the
specification), ZITADEL would crash.
# How the Problems Are Solved
- Check specifically if the `Subject` and its `NameID` is passed
# Additional Changes
None
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8654
# Which Problems Are Solved
There are cases where not all statements of multiExec are succeed. This
leads to inconsistent states. One example is [LDAP
IDPs](https://github.com/zitadel/zitadel/issues/7959).
If statements get executed only partially this can lead to inconsistent
states or even break projections for objects which might not were
correctly created in a sub table.
This behaviour is possible because we use
[`SAVEPOINTS`](https://www.postgresql.org/docs/current/sql-savepoint.html)
during each statement of a multiExec.
# How the Problems Are Solved
SAVEPOINTS are only created at the beginning of an exec function not
during every execution like before. Additionally `RELEASE` or `ROLLBACK`
of `SAVEPOINTS` are only used when needed.
# Additional Changes
- refactor some unused parameters
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/7959
(cherry picked from commit ddeeeed303)
# Which Problems Are Solved
V2 and V3 APIs allow setting the organization context by providing the
organization domain in the request. Users currently experience the
following error: "rpc error: code = Unauthenticated desc = context
missing (AUTH-rKLWEH)"
# How the Problems Are Solved
Correctly check the org domain when set.
# Additional Changes
None
# Additional Context
- support request
(cherry picked from commit dc7330f251)
# Which Problems Are Solved
There are cases where not all statements of multiExec are succeed. This
leads to inconsistent states. One example is [LDAP
IDPs](https://github.com/zitadel/zitadel/issues/7959).
If statements get executed only partially this can lead to inconsistent
states or even break projections for objects which might not were
correctly created in a sub table.
This behaviour is possible because we use
[`SAVEPOINTS`](https://www.postgresql.org/docs/current/sql-savepoint.html)
during each statement of a multiExec.
# How the Problems Are Solved
SAVEPOINTS are only created at the beginning of an exec function not
during every execution like before. Additionally `RELEASE` or `ROLLBACK`
of `SAVEPOINTS` are only used when needed.
# Additional Changes
- refactor some unused parameters
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/7959
# Which Problems Are Solved
V2 and V3 APIs allow setting the organization context by providing the
organization domain in the request. Users currently experience the
following error: "rpc error: code = Unauthenticated desc = context
missing (AUTH-rKLWEH)"
# How the Problems Are Solved
Correctly check the org domain when set.
# Additional Changes
None
# Additional Context
- support request
# Which Problems Are Solved
Calls with tokens issued through JWT Profile or Client Credentials
Grants were no longer possible and threw a "could not read projectid by
clientid (AUTH-GHpw2)" error.
ZITADEL checks the allowed origins of an application and load its
projectID into the context on any API call.
Tokens from service accounts did not contain any clientID and therefore
never did that check.
But due to a change in https://github.com/zitadel/zitadel/pull/8580,
were the service user id was set as client_id in the OIDC session to fix
the introspection response
(https://github.com/zitadel/zitadel/issues/8590).
# How the Problems Are Solved
- Check if the project and origin were retrieved and only then check the
origins
# Additional Changes
None.
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8676
- relates to https://github.com/zitadel/zitadel/pull/8580 (released on
2.62.0)
- relates to https://github.com/zitadel/zitadel/issues/8590
(cherry picked from commit c347e75485)
# Which Problems Are Solved
Calls with tokens issued through JWT Profile or Client Credentials
Grants were no longer possible and threw a "could not read projectid by
clientid (AUTH-GHpw2)" error.
ZITADEL checks the allowed origins of an application and load its
projectID into the context on any API call.
Tokens from service accounts did not contain any clientID and therefore
never did that check.
But due to a change in https://github.com/zitadel/zitadel/pull/8580,
were the service user id was set as client_id in the OIDC session to fix
the introspection response
(https://github.com/zitadel/zitadel/issues/8590).
# How the Problems Are Solved
- Check if the project and origin were retrieved and only then check the
origins
# Additional Changes
None.
# Additional Context
- closes https://github.com/zitadel/zitadel/issues/8676
- relates to https://github.com/zitadel/zitadel/pull/8580 (released on
2.62.0)
- relates to https://github.com/zitadel/zitadel/issues/8590
# Which Problems Are Solved
When executing many concurrent authentication requests on a single
machine user, there were performance issues. As the same aggregate is
being searched and written to concurrently, we traced it down to a
locking issue on the used index.
We already optimized the token endpoint by creating a separate OIDC
aggregate.
At the time we decided to push a single event to the user aggregate, for
the user audit log. See [technical advisory
10010](https://zitadel.com/docs/support/advisory/a10010) for more
details.
However, a recent security fix introduced an additional search query on
the user aggregate, causing the locking issue we found.
# How the Problems Are Solved
Add a feature flag which disables pushing of the `user.token.v2.added`.
The event has no importance and was only added for informational
purposes on the user objects. The `oidc_session.access_token.added` is
the actual payload event and is pushed on the OIDC session aggregate and
can still be used for audit trail.
# Additional Changes
- Fix an event mapper type for
`SystemOIDCSingleV1SessionTerminationEventType`
# Additional Context
- Reported by support request
- https://github.com/zitadel/zitadel/pull/7822 changed the token
aggregate
- https://github.com/zitadel/zitadel/pull/8631 introduced user state
check
Load test trace graph with `user.token.v2.added` **enabled**. Query
times are steadily increasing:
Load test trace graph with `user.token.v2.added` **disabled**. Query
times constant:
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
(cherry picked from commit 63d733b3a2)
