# Which Problems Are Solved
Sometimes integrations tests are failing with an error `http: no
location header in response`. The underlying cause was hidden, as in
some tests we assumed a 3xx range response but got a 4xx response
instead. No assertion on the status code was made, resulting in the
above error message on calling `resp.Location()`.
The underlying issue, the application not found in the projection, is
also fixed.
# How the Problems Are Solved
This change adds a check for the status code and returns the response
body if the response is not in the 3xx status code range.
Helper function that create applications now do an additional
`GetAppByID` in a retry loop to ensure consitency in the projection
before proceeding with tests.
# Additional Changes
- none
# Additional Context
- Pipline failures were observed, no issue was created
- Cherry-picked form WIP #8407
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
To have more insight on the performance, CPU and memory usage of
ZITADEL, we want to enable profiling.
# How the Problems Are Solved
- Allow profiling by configuration.
- Provide Google Cloud Profiler as first implementation
# Additional Changes
None.
# Additional Context
There were possible memory leaks reported:
https://discord.com/channels/927474939156643850/1273210227918897152
Co-authored-by: Silvan <silvan.reusser@gmail.com>
# Which Problems Are Solved
When Web keys with a large RSA Bitsize were generated, the new web key
did not get projected.
Zitadel logs printed the following errors:
```
time="2024-08-15T09:18:47Z" level=debug msg="execution of statements failed" caller="/home/tim/Repositories/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:518" error="statement failed: ID=CRDB-pKtsr Message=exec failed Parent=(ERROR: index row size 3400 exceeds btree version 4 maximum 2704 for index \"web_keys_web_key_state_idx\" (SQLSTATE 54000))" lastProcessedIndex=-1 projection=projections.web_keys
```
# How the Problems Are Solved
Remove the the `WithInclude` option with the Private key from the
`web_keys` projection index and bump the projection version.
# Additional Changes
- Correct RPC summaries for documentation
- Fix docs slug in sidebar.js
# Additional Context
- Found during release review.
# Which Problems Are Solved
Execution responses with HTTP StatusCode not equal to 200 interrupt the
client request silently.
# How the Problems Are Solved
Adds information about the recieved StatusCode and Body into the error
if StatusCode not 200.
# Additional Context
Closes#8177
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
The filter option was not displayed on the user list page for users who
only have `user.read` permission, e.g. an IAM_OWNER_VIEWER or
ORG_OWNER_VIEWER
# How the Problems Are Solved
- Filter is correctly displayed.
# Additional Changes
None.
# Additional Context
- noticed by a customer
- needs backports
# Which Problems Are Solved
In case a user was deleted and recreated with the same id, they would
never be able to authenticate through the login UI, since it would
return an error "User not active".
This was due to the check in the auth request / session handling for the
login UI, where the user removed event would terminate an further event
check and ignore the newly added user.
# How the Problems Are Solved
- The user removed event no longer returns an error, but is handled as a
session termination event.
(A user removed event will already delete the user and the preceding
`activeUserById` function will deny the authentication.)
# Additional Changes
Updated tests to be able to handle multiple events in the mocks.
# Additional Context
closes https://github.com/zitadel/zitadel/issues/8201
Co-authored-by: Silvan <silvan.reusser@gmail.com>
# Which Problems Are Solved
Org v2 service does not have a ListOrganizations endpoint.
# How the Problems Are Solved
Implement ListOrganizations endpoint.
# Additional Changes
- moved descriptions in the protos to comments
- corrected the RemoveNoPermissions for the ListUsers, to get the
correct TotalResults
# Additional Context
For new typescript login
# Which Problems Are Solved
GetIDPByID as endpoint in the API v2 so that it can be available for the
new login.
# How the Problems Are Solved
Create GetIDPByID endpoint with IDP v2 API, throught the GetProviderByID
implementation from admin and management API.
# Additional Changes
- Remove the OwnerType attribute from the response, as the information
is available through the resourceOwner.
- correct refs to messages in proto which are used for doc generation
- renaming of elements for API v3
# Additional Context
Closes#8337
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Implement a new API service that allows management of OIDC signing web
keys.
This allows users to manage rotation of the instance level keys. which
are currently managed based on expiry.
The API accepts the generation of the following key types and
parameters:
- RSA keys with 2048, 3072 or 4096 bit in size and:
- Signing with SHA-256 (RS256)
- Signing with SHA-384 (RS384)
- Signing with SHA-512 (RS512)
- ECDSA keys with
- P256 curve
- P384 curve
- P512 curve
- ED25519 keys
# How the Problems Are Solved
Keys are serialized for storage using the JSON web key format from the
`jose` library. This is the format that will be used by OIDC for
signing, verification and publication.
Each instance can have a number of key pairs. All existing public keys
are meant to be used for token verification and publication the keys
endpoint. Keys can be activated and the active private key is meant to
sign new tokens. There is always exactly 1 active signing key:
1. When the first key for an instance is generated, it is automatically
activated.
2. Activation of the next key automatically deactivates the previously
active key.
3. Keys cannot be manually deactivated from the API
4. Active keys cannot be deleted
# Additional Changes
- Query methods that later will be used by the OIDC package are already
implemented. Preparation for #8031
- Fix indentation in french translation for instance event
- Move user_schema translations to consistent positions in all
translation files
# Additional Context
- Closes#8030
- Part of #7809
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
# Which Problems Are Solved
As of now, **automatic creation** and **automatic linking options** were
only considered if the corresponding **allowed option** (account
creation / linking allowed) was enabled.
With this PR, this is no longer needed and allows administrators to
address cases, where only an **automatic creation** is allowed, but
users themselves should not be allowed to **manually** create new
accounts using an identity provider or edit the information during the
process.
Also, allowing users to only link to the proposed existing account is
now possible with an enabled **automatic linking option**, while
disabling **account linking allowed**.
# How the Problems Are Solved
- Check for **automatic** options without the corresponding **allowed**
option.
- added technical advisory to notify about the possible behavior change
# Additional Changes
- display the error message on the IdP linking step in the login UI (in
case there is one)
- display an error in case no option is possible
- exchanged deprecated `eventstoreExpect` with `expectEventstore` in
touched test files
# Additional Context
closes https://github.com/zitadel/zitadel/issues/7393
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Log some details when a secret generator is not found.
This should help us debugging such issue.
# How the Problems Are Solved
When a secret generator by type query fails,
we log the generator type and instance id for which
the generator was requested.
# Additional Changes
- none
# Additional Context
- Related to https://github.com/zitadel/zitadel/issues/8379
- Also encountered in https://github.com/zitadel/zitadel/pull/8407
# Which Problems Are Solved
The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).
# How the Problems Are Solved
- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details
# Additional Changes
Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.
# Additional Context
- Closes#8169
- Depends on https://github.com/zitadel/zitadel/pull/8225
---------
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
^Since publishing the new V2 GA APi, we have a lot of broken links in
our docs
# How the Problems Are Solved
replace api links with v2 links
# Which Problems Are Solved
The mirror command used the wrong position to filter for events if
different database technologies for source and destination were used.
# How the Problems Are Solved
The statements which diverge are stored on the client so that different
technologies can use different statements.
# Additional Context
- https://discord.com/channels/927474939156643850/1256396896243552347
# Which Problems Are Solved
There was no default configuration for `DeviceAuth`, which makes it
impossible to override by environment variables.
Additionally, a custom `CharAmount` value would overwrite also the
`DashInterval`.
# How the Problems Are Solved
- added to defaults.yaml
- fixed customization
# Additional Changes
None.
# Additional Context
- noticed during a customer request
# Which Problems Are Solved
In the PR template we have added some ideas about additional context,
but we link to existing prs and issues as an example.
So everytime someone doesn't change the description when creating the
issue, its a mention to that issue or pr.
# How the Problems Are Solved
replace with non existing values
![Uploading image.png…]()
# Which Problems Are Solved
If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.
# How the Problems Are Solved
Changed the isolation level of the current states query to Read Commited
# Which Problems Are Solved
If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.
# How the Problems Are Solved
Changed the isolation level of the current states query to Read Commited
# Which Problems Are Solved
The e2e tests fail because the organization selection is too fast.
# How the Problems Are Solved
Wait until console has loaded properly.
# Additional Context
- The tests still use the wrong browser, #8404 describes the problem
- closes https://github.com/zitadel/zitadel/issues/8378
# Which Problems Are Solved
OIDC redirects have wrong headers
# How the Problems Are Solved
This is fixed with https://github.com/zitadel/oidc/pull/632. This change
updates the OIDC lib to a fixed version.
# Which Problems Are Solved
OIDC redirects have wrong headers
# How the Problems Are Solved
This is fixed with https://github.com/zitadel/oidc/pull/632. This change
updates the OIDC lib to a fixed version.
# Which Problems Are Solved
During triggering of the fields table WriteTooOld errors can occure when
using cockroachdb.
# How the Problems Are Solved
The statements exclusively lock the projection before they start to
insert data by using `FOR UPDATE`.
# Which Problems Are Solved
We noticed logging where 500: Internal Server errors were returned from
the token endpoint, mostly for the `refresh_token` grant. The error was
thrown by the database as it received non-UTF8 strings for token IDs
Zitadel uses symmetric encryption for opaque tokens, including refresh
tokens. Encrypted values are base64 encoded. It appeared to be possible
to send garbage base64 to the token endpoint, which will pass decryption
and string-splitting. In those cases the resulting ID is not a valid
UTF-8 string.
Invalid non-UTF8 strings are now rejected during token decryption.
# How the Problems Are Solved
- `AESCrypto.DecryptString()` checks if the decrypted bytes only contain
valid UTF-8 characters before converting them into a string.
- `AESCrypto.Decrypt()` is unmodified and still allows decryption on
non-UTF8 byte strings.
- `FromRefreshToken` now uses `DecryptString` instead of `Decrypt`
# Additional Changes
- Unit tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`.
- Fuzz tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`. This was to pinpoint the problem
- Testdata with values that resulted in invalid strings are committed.
In the pipeline this results in the Fuzz tests to execute as regular
unit-test cases. As we don't use the `-fuzz` flag in the pipeline no
further fuzzing is performed.
# Additional Context
- Closes#7765
- https://go.dev/doc/tutorial/fuzz
# Which Problems Are Solved
During triggering of the fields table WriteTooOld errors can occure when
using cockroachdb.
# How the Problems Are Solved
The statements exclusively lock the projection before they start to
insert data by using `FOR UPDATE`.
# Which Problems Are Solved
The text appears to contradict the statement in the page on oauth.net.
# How the Problems Are Solved
The text has been updated to reflect the statement in the page on
oauth.net.
# Additional Changes
None
# Additional Context
The page [OAUTH2.0 for mobile and native
apps](https://oauth.net/2/native-apps/) linked just above the text that
was changed states:
> It describes things like not allowing the third-party application to
open an embedded web view which is more susceptible to phishing attacks,
as well as platform-specific recommendations on how to do so.
Co-authored-by: Max Peintner <max@caos.ch>
# Which Problems Are Solved
We noticed logging where 500: Internal Server errors were returned from
the token endpoint, mostly for the `refresh_token` grant. The error was
thrown by the database as it received non-UTF8 strings for token IDs
Zitadel uses symmetric encryption for opaque tokens, including refresh
tokens. Encrypted values are base64 encoded. It appeared to be possible
to send garbage base64 to the token endpoint, which will pass decryption
and string-splitting. In those cases the resulting ID is not a valid
UTF-8 string.
Invalid non-UTF8 strings are now rejected during token decryption.
# How the Problems Are Solved
- `AESCrypto.DecryptString()` checks if the decrypted bytes only contain
valid UTF-8 characters before converting them into a string.
- `AESCrypto.Decrypt()` is unmodified and still allows decryption on
non-UTF8 byte strings.
- `FromRefreshToken` now uses `DecryptString` instead of `Decrypt`
# Additional Changes
- Unit tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`.
- Fuzz tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`. This was to pinpoint the problem
- Testdata with values that resulted in invalid strings are committed.
In the pipeline this results in the Fuzz tests to execute as regular
unit-test cases. As we don't use the `-fuzz` flag in the pipeline no
further fuzzing is performed.
# Additional Context
- Closes#7765
- https://go.dev/doc/tutorial/fuzz
# Which Problems Are Solved
ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.
# How the Problems Are Solved
- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain
# Additional Changes
None
# Additional Context
- part of #8279
- open topics:
- "single-instance" mode
- Console UI
# Which Problems Are Solved
The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).
# How the Problems Are Solved
- **Breaking**: The current v3alpha actions APIs are removed. This is
breaking.
- **Resource Namespace**: New v3alpha actions APIs for targets and
executions are added under the namespace /resources.
- **Feature Flag**: New v3alpha actions APIs still have to be activated
using the actions feature flag
- **Reduced Executions Overhead**: Executions are managed similar to
settings according to the new API design: an empty list of targets
basically makes an execution a Noop. So a single method, SetExecution is
enough to cover all use cases. Noop executions are not returned in
future search requests.
- **Compatibility**: The executions created with previous v3alpha APIs
are still available to be managed with the new executions API.
# Additional Changes
- Removed integration tests which test executions but rely on readable
targets. They are added again with #8169
# Additional Context
Closes#8168
# Which Problems Are Solved
ZITADEL administrators can enable a setting called "Ignoring unknown
usernames" which helps mitigate attacks that try to guess/enumerate
usernames. If enabled, ZITADEL will show the password prompt even if the
user doesn't exist and report "Username or Password invalid".
Due to a implementation change to prevent deadlocks calling the
database, the flag would not be correctly respected in all cases and an
attacker would gain information if an account exist within ZITADEL,
since the error message shows "object not found" instead of the generic
error message.
# How the Problems Are Solved
- Proper check of the error using an error function / type and
`errors.Is`
# Additional Changes
None.
# Additional Context
- raised in a support request
Co-authored-by: Silvan <silvan.reusser@gmail.com>
(cherry picked from commit a1d24353db)
# Which Problems Are Solved
ZITADEL uses HTML for emails and renders certain information such as
usernames dynamically. That information can be entered by users or
administrators. Due to a missing output sanitization, these emails could
include malicious code.
This may potentially lead to a threat where an attacker, without
privileges, could send out altered notifications that are part of the
registration processes. An attacker could create a malicious link, where
the injected code would be rendered as part of the email.
During investigation of this issue a related issue was found and
mitigated, where on the user's detail page the username was not
sanitized and would also render HTML, giving an attacker the same
vulnerability.
While it was possible to inject HTML including javascript, the execution
of such scripts would be prevented by most email clients and the Content
Security Policy in Console UI.
# How the Problems Are Solved
- All arguments used for email are sanitized (`html.EscapeString`)
- The email text no longer `html.UnescapeString` (HTML in custom text is
still possible)
- Console no longer uses `[innerHtml]` to render the username
# Additional Changes
None
# Additional Context
- raised via email
---------
Co-authored-by: peintnermax <max@caos.ch>
(cherry picked from commit 189505c80f)
# Which Problems Are Solved
ZITADEL administrators can enable a setting called "Ignoring unknown
usernames" which helps mitigate attacks that try to guess/enumerate
usernames. If enabled, ZITADEL will show the password prompt even if the
user doesn't exist and report "Username or Password invalid".
Due to a implementation change to prevent deadlocks calling the
database, the flag would not be correctly respected in all cases and an
attacker would gain information if an account exist within ZITADEL,
since the error message shows "object not found" instead of the generic
error message.
# How the Problems Are Solved
- Proper check of the error using an error function / type and
`errors.Is`
# Additional Changes
None.
# Additional Context
- raised in a support request
Co-authored-by: Silvan <silvan.reusser@gmail.com>
# Which Problems Are Solved
ZITADEL uses HTML for emails and renders certain information such as
usernames dynamically. That information can be entered by users or
administrators. Due to a missing output sanitization, these emails could
include malicious code.
This may potentially lead to a threat where an attacker, without
privileges, could send out altered notifications that are part of the
registration processes. An attacker could create a malicious link, where
the injected code would be rendered as part of the email.
During investigation of this issue a related issue was found and
mitigated, where on the user's detail page the username was not
sanitized and would also render HTML, giving an attacker the same
vulnerability.
While it was possible to inject HTML including javascript, the execution
of such scripts would be prevented by most email clients and the Content
Security Policy in Console UI.
# How the Problems Are Solved
- All arguments used for email are sanitized (`html.EscapeString`)
- The email text no longer `html.UnescapeString` (HTML in custom text is
still possible)
- Console no longer uses `[innerHtml]` to render the username
# Additional Changes
None
# Additional Context
- raised via email
---------
Co-authored-by: peintnermax <max@caos.ch>
This fixes a problem where the org settings were hidden.
The console reads the context from either a query param or the local
storage. When one context was found, it executed a single request with
orgId filter. This let to a single org and then to a hidden org setting,
as we hide org settings for instances with a single result.
(cherry picked from commit 51210c8e34)
This fixes a problem where the org settings were hidden.
The console reads the context from either a query param or the local
storage. When one context was found, it executed a single request with
orgId filter. This let to a single org and then to a hidden org setting,
as we hide org settings for instances with a single result.
# Which Problems Are Solved
The v2beta services are stable but not GA.
# How the Problems Are Solved
The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.
# Additional Context
Closes#7236
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
(cherry picked from commit 7d2d85f57c)
# Which Problems Are Solved
The v2beta services are stable but not GA.
# How the Problems Are Solved
The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.
# Additional Context
Closes#7236
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
# Which Problems Are Solved
I fixed more spelling and grammar misstakes in the German language
files.
# Additional Context
- Follow-up for PR #8240
Co-authored-by: Fabi <fabienne@zitadel.com>
# Which Problems Are Solved
The initial load of the login UI with dark mode preference
(prefers-color-scheme: dark) first rendered the logo configured for
light mode. Also switching from dark to light or vice versa would result
in the same behavior.
This was due to a mixed logic of server (based on cookie) and client
(prefers-color-scheme and cookie) deciding which mode to render.
# How the Problems Are Solved
- Since the main logic of which mode to use (`prefers-color-scheme`) can
only be achieve client side, both logos will be served in the HTML and
either will be rendered based on CSS.
# Additional Changes
None
# Additional Context
- closes#2085
# Which Problems Are Solved
The default terms of service and privacy policy links are applied to all
new ZITADEL instances, also for self hosters. However, the links
contents don't apply to self-hosters.
# How the Problems Are Solved
The links are removed from the DefaultInstance section in the
*defaults.yaml* file.
By default, the links are not shown anymore in the hosted login pages.
They can still be configured using the privacy policy.
# Additional Context
- Found because of a support request
