Commit Graph

1672 Commits

Author SHA1 Message Date
Livio Spring
c667ab7047
Merge branch 'main' into next 2024-09-02 14:46:00 +02:00
Stefan Benz
a9eb50321c
fix: update userschema api to v3alpha definition (#8511)
# Which Problems Are Solved

UserSchema API is currently not completely as defined for v3alpha.

# How the Problems Are Solved

Update the protos and integration tests.

# Additional Changes

None

# Additional Context

None
2024-09-02 11:24:15 +00:00
Stefan Benz
41ae35f2ef
feat: add schema user create and remove (#8494)
# Which Problems Are Solved

Added functionality that user with a userschema can be created and
removed.

# How the Problems Are Solved

Added logic and moved APIs so that everything is API v3 conform.

# Additional Changes

- move of user and userschema API to resources folder
- changed testing and parameters
- some renaming

# Additional Context

closes #7308

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-08-28 19:46:45 +00:00
Tim Möhlmann
90b908c361
fix(oidc): don't push introspection client events (#8481)
# Which Problems Are Solved

Do not push secret succeeded and failed events for API and OIDC clients
on the introspection endpoint.
On instances where introspection was fequently called, the pushed events
created issues on duplicate primary keys, due to collisions on the
`sequence` column in the eventstore. As the event pusher retries on this
collision and we pushed above mentioned events async, it would create a
backpressure of concurrent pushers and effectively cripple an instance.

We considered that pushing these events have little value with regards
to the audit trail, as we do not push similar events when client
assertion is used. Also, before #7657 the events were defined, but not
pushed.

# How the Problems Are Solved

- Removed API secret check succeeded and faild event definitions
- Removed OIDC secret check succeeded and faild event definitions
- Push only Hash Updated event when needed

# Additional Changes

- None

# Additional Context

- Fixes https://github.com/zitadel/zitadel/issues/8479
- Closes https://github.com/zitadel/zitadel/issues/8430
- Intoduced in https://github.com/zitadel/zitadel/pull/7657
2024-08-28 18:19:50 +00:00
Livio Spring
ca8f82423a
fix: automatically link user without prompt (#8487)
# Which Problems Are Solved

There were UX issue with the autolinking prompt page and users were not
able to link their account or would not understand what to do. Since the
trust to the IdP is already bound by the configuration, the user can
directly be linked without any user input.

# How the Problems Are Solved

- remove the prompt page and directly link the user if possible
- remove corresponding customization texts from the API and Console

# Additional Changes

None

# Additional Context

- relates to https://github.com/zitadel/zitadel/issues/7977
- discussed with customers
- created as a `fix` to be able to backport

---------

Co-authored-by: Max Peintner <max@caos.ch>
2024-08-28 05:33:20 +00:00
Livio Spring
cbbd44c303
fix: return 401 instead of 403 on expired tokens (#8476)
# Which Problems Are Solved

The access token verifier returned a permission denied (HTTP 403 / GRPC
7) instead of a unauthenticated (HTTP 401 / GRPC 16) error.

# How the Problems Are Solved

Return the correct error type.

# Additional Changes

None

# Additional Context

close #8392
2024-08-26 10:15:40 +00:00
Livio Spring
862d141171
fix: internal check of JWT access tokens (#8486)
# Which Problems Are Solved

When using a JWT access_token on a ZITADEL API, the token was considered
invalid If the `WebKey` feature flag is enabled.

# How the Problems Are Solved

- Merge the new and old web keys if flag is enabled (as already done for
the keys endpoint).

# Additional Changes

None

# Additional Context

relates to #8449
2024-08-26 09:26:13 +00:00
Tim Möhlmann
fd0c15dd4f
feat(oidc): use web keys for token signing and verification (#8449)
# Which Problems Are Solved

Use web keys, managed by the `resources/v3alpha/web_keys` API, for OIDC
token signing and verification,
as well as serving the public web keys on the jwks / keys endpoint.
Response header on the keys endpoint now allows caching of the response.
This is now "safe" to do since keys can be created ahead of time and
caches have sufficient time to pickup the change before keys get
enabled.

# How the Problems Are Solved

- The web key format is used in the `getSignerOnce` function in the
`api/oidc` package.
- The public key cache is changed to get and store web keys.
- The jwks / keys endpoint returns the combined set of valid "legacy"
public keys and all available web keys.
- Cache-Control max-age default to 5 minutes and is configured in
`defaults.yaml`.

When the web keys feature is enabled, fallback mechanisms are in place
to obtain and convert "legacy" `query.PublicKey` as web keys when
needed. This allows transitioning to the feature without invalidating
existing tokens. A small performance overhead may be noticed on the keys
endpoint, because 2 queries need to be run sequentially. This will
disappear once the feature is stable and the legacy code gets cleaned
up.

# Additional Changes

- Extend legacy key lifetimes so that tests can be run on an existing
database with more than 6 hours apart.
- Discovery endpoint returns all supported algorithms when the Web Key
feature is enabled.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8031
- Part of https://github.com/zitadel/zitadel/issues/7809
- After https://github.com/zitadel/oidc/pull/637
- After https://github.com/zitadel/oidc/pull/638
2024-08-23 14:43:46 +02:00
Stefan Benz
2847806531
fix: generalise permission check for query user information (#8458)
# Which Problems Are Solved

IDPLinks list and other list endpoints can provide you with empty
results if the used user has no permission for the information.

# How the Problems Are Solved

List endpoints with subelements to users, and provided userIDQuery, will
return a PermissionDenied error if no permission for the user exsists.

# Additional Changes

Function to check for permission is re-used from the GetUserByID.

# Additional Context

Closes #8451
2024-08-23 06:44:18 +00:00
Livio Spring
1cefa7cb40
fix: respect x-zitadel-forwarded header again (#8473)
# Which Problems Are Solved

#8369 added the possibility to handle trusted domains for public hosts
as response. Additionally, the OIDC issuer is extracted from the
`DomainContext` and not from headers anymore.
This accidentally dropped support for the `x-zitadel-forwarded`.

# How the Problems Are Solved

Added `x-zitadel-forwarded` in the list of additionally handled headers.

# Additional Changes

None

# Additional Context

- relates to #8369
- reported in Discord:
https://discord.com/channels/927474939156643850/1275484169626980403

(cherry picked from commit b1f5b1979c)
2024-08-22 06:37:09 +02:00
Livio Spring
b1f5b1979c
fix: respect x-zitadel-forwarded header again (#8473)
# Which Problems Are Solved

#8369 added the possibility to handle trusted domains for public hosts
as response. Additionally, the OIDC issuer is extracted from the
`DomainContext` and not from headers anymore.
This accidentally dropped support for the `x-zitadel-forwarded`.

# How the Problems Are Solved

Added `x-zitadel-forwarded` in the list of additionally handled headers.

# Additional Changes

None

# Additional Context

- relates to #8369
- reported in Discord:
https://discord.com/channels/927474939156643850/1275484169626980403
2024-08-21 16:36:16 +00:00
Tim Möhlmann
5fd2061770
feat(oidc): allow returning of parent errors to client (#8376)
# Which Problems Are Solved

Currently the OIDC API of ZITADEL only prints parent errors to the logs.
Where 4xx status are typically warn level and 5xx error level. This
makes it hard to debug certain errors for client in multi-instance
environments like ZITADEL cloud, where there is no direct access to
logs. In case of support requests we often can't correlate past
log-lines to the error that was reported.

This change adds the possibility to return the parent error in the
response to the OIDC client. For the moment this only applies to JSON
body responses, not error redirects to the RP.

# How the Problems Are Solved

- New instance-level feature flag: `debug_oidc_parent_error`
- Use the new `WithReturnParentToClient()` function from the oidc lib
introduced in https://github.com/zitadel/oidc/pull/629 for all cases
where `WithParent` was already used and the request context is
available.

# Additional Changes

none

# Additional Context

- Depends on: https://github.com/zitadel/oidc/pull/629
- Related to: https://github.com/zitadel/zitadel/issues/8362

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-20 06:45:24 +00:00
Livio Spring
bdae824e8c
Merge branch 'refs/heads/main' into next
# Conflicts:
#	cmd/start/start.go
#	docs/sidebars.js
#	internal/api/grpc/action/v3alpha/execution_integration_test.go
#	internal/api/grpc/action/v3alpha/query_integration_test.go
#	internal/api/grpc/action/v3alpha/target_integration_test.go
#	internal/api/grpc/feature/v2beta/converter.go
#	internal/api/grpc/feature/v2beta/converter_test.go
#	internal/api/grpc/oidc/v2beta/oidc.go
#	internal/api/grpc/resources/action/v3alpha/server_integration_test.go
#	internal/api/grpc/settings/v2beta/server.go
#	internal/api/grpc/user/v2/query_integration_test.go
#	internal/api/grpc/user/v2beta/query.go
#	internal/api/grpc/user/v2beta/query_integration_test.go
#	internal/auth/repository/eventsourcing/eventstore/auth_request_test.go
#	internal/command/user_idp_link_test.go
#	internal/crypto/crypto.go
#	internal/integration/assert.go
#	internal/integration/client.go
#	proto/zitadel/action/v3alpha/target.proto
#	proto/zitadel/feature/v2/instance.proto
#	proto/zitadel/org/v2/org_service.proto
#	proto/zitadel/resources/action/v3alpha/action_service.proto
#	proto/zitadel/resources/action/v3alpha/execution.proto
#	proto/zitadel/resources/action/v3alpha/query.proto
#	proto/zitadel/user/v2/user_service.proto
2024-08-19 16:55:55 +02:00
Tim Möhlmann
10d5fc6184
test: no location header in response (#8441)
# Which Problems Are Solved

Sometimes integrations tests are failing with an error `http: no
location header in response`. The underlying cause was hidden, as in
some tests we assumed a 3xx range response but got a 4xx response
instead. No assertion on the status code was made, resulting in the
above error message on calling `resp.Location()`.

The underlying issue, the application not found in the projection, is
also fixed.

# How the Problems Are Solved

This change adds a check for the status code and returns the response
body if the response is not in the 3xx status code range.

Helper function that create applications now do an additional
`GetAppByID` in a retry loop to ensure consitency in the projection
before proceeding with tests.

# Additional Changes

- none

# Additional Context

- Pipline failures were observed, no issue was created
- Cherry-picked form WIP #8407

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-17 14:48:06 +00:00
Livio Spring
c8e2a3bd49
feat: enable application performance profiling (#8442)
# Which Problems Are Solved

To have more insight on the performance, CPU and memory usage of
ZITADEL, we want to enable profiling.

# How the Problems Are Solved

- Allow profiling by configuration.
- Provide Google Cloud Profiler as first implementation

# Additional Changes

None.

# Additional Context

There were possible memory leaks reported:
https://discord.com/channels/927474939156643850/1273210227918897152

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-08-16 13:26:53 +00:00
Tim Möhlmann
ab4daa0ebb
fix(webkeys): remove include private key from projection index (#8436)
# Which Problems Are Solved

When Web keys with a large RSA Bitsize were generated, the new web key
did not get projected.
Zitadel logs printed the following errors:

```
time="2024-08-15T09:18:47Z" level=debug msg="execution of statements failed" caller="/home/tim/Repositories/zitadel/zitadel/internal/eventstore/handler/v2/handler.go:518" error="statement failed: ID=CRDB-pKtsr Message=exec failed Parent=(ERROR: index row size 3400 exceeds btree version 4 maximum 2704 for index \"web_keys_web_key_state_idx\" (SQLSTATE 54000))" lastProcessedIndex=-1 projection=projections.web_keys
```


# How the Problems Are Solved

Remove the the `WithInclude` option with the Private key from the
`web_keys` projection index and bump the projection version.

# Additional Changes

- Correct RPC summaries for documentation
- Fix docs slug in sidebar.js

# Additional Context

- Found during release review.
2024-08-16 11:41:09 +00:00
Stefan Benz
83c78a470c
fix: add information about target response into error message if inte… (#8281)
# Which Problems Are Solved

Execution responses with HTTP StatusCode not equal to 200 interrupt the
client request silently.

# How the Problems Are Solved

Adds information about the recieved StatusCode and Body into the error
if StatusCode not 200.

# Additional Context

Closes #8177

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-16 11:26:15 +02:00
Livio Spring
0af37d45e9
fix: handle user remove correctly in v1 sessions for login (#8432)
# Which Problems Are Solved

In case a user was deleted and recreated with the same id, they would
never be able to authenticate through the login UI, since it would
return an error "User not active".
This was due to the check in the auth request / session handling for the
login UI, where the user removed event would terminate an further event
check and ignore the newly added user.

# How the Problems Are Solved

- The user removed event no longer returns an error, but is handled as a
session termination event.
(A user removed event will already delete the user and the preceding
`activeUserById` function will deny the authentication.)

# Additional Changes

Updated tests to be able to handle multiple events in the mocks.

# Additional Context

closes https://github.com/zitadel/zitadel/issues/8201

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-08-15 05:39:54 +00:00
Stefan Benz
5fab533e37
feat: org v2 ListOrganizations (#8411)
# Which Problems Are Solved

Org v2 service does not have a ListOrganizations endpoint.

# How the Problems Are Solved

Implement ListOrganizations endpoint.

# Additional Changes

- moved descriptions in the protos to comments
- corrected the RemoveNoPermissions for the ListUsers, to get the
correct TotalResults

# Additional Context

For new typescript login
2024-08-15 06:37:06 +02:00
Stefan Benz
3e3d46ac0d
feat: idp v2 api GetIDPByID (#8425)
# Which Problems Are Solved

GetIDPByID as endpoint in the API v2 so that it can be available for the
new login.

# How the Problems Are Solved

Create GetIDPByID endpoint with IDP v2 API, throught the GetProviderByID
implementation from admin and management API.

# Additional Changes

- Remove the OwnerType attribute from the response, as the information
is available through the resourceOwner.
- correct refs to messages in proto which are used for doc generation
- renaming of elements for API v3

# Additional Context

Closes #8337

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-08-14 18:18:29 +00:00
Tim Möhlmann
64a3bb3149
feat(v3alpha): web key resource (#8262)
# Which Problems Are Solved

Implement a new API service that allows management of OIDC signing web
keys.
This allows users to manage rotation of the instance level keys. which
are currently managed based on expiry.

The API accepts the generation of the following key types and
parameters:

- RSA keys with 2048, 3072 or 4096 bit in size and:
  - Signing with SHA-256 (RS256)
  - Signing with SHA-384 (RS384)
  - Signing with SHA-512 (RS512)
- ECDSA keys with
  - P256 curve
  - P384 curve
  - P512 curve
- ED25519 keys

# How the Problems Are Solved

Keys are serialized for storage using the JSON web key format from the
`jose` library. This is the format that will be used by OIDC for
signing, verification and publication.

Each instance can have a number of key pairs. All existing public keys
are meant to be used for token verification and publication the keys
endpoint. Keys can be activated and the active private key is meant to
sign new tokens. There is always exactly 1 active signing key:

1. When the first key for an instance is generated, it is automatically
activated.
2. Activation of the next key automatically deactivates the previously
active key.
3. Keys cannot be manually deactivated from the API
4. Active keys cannot be deleted

# Additional Changes

- Query methods that later will be used by the OIDC package are already
implemented. Preparation for #8031
- Fix indentation in french translation for instance event
- Move user_schema translations to consistent positions in all
translation files

# Additional Context

- Closes #8030
- Part of #7809

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-08-14 14:18:14 +00:00
Livio Spring
e2e1100124
feat(idp): provide auto only options (#8420)
# Which Problems Are Solved

As of now, **automatic creation** and **automatic linking options** were
only considered if the corresponding **allowed option** (account
creation / linking allowed) was enabled.

With this PR, this is no longer needed and allows administrators to
address cases, where only an **automatic creation** is allowed, but
users themselves should not be allowed to **manually** create new
accounts using an identity provider or edit the information during the
process.
Also, allowing users to only link to the proposed existing account is
now possible with an enabled **automatic linking option**, while
disabling **account linking allowed**.

# How the Problems Are Solved

- Check for **automatic** options without the corresponding **allowed**
option.
- added technical advisory to notify about the possible behavior change

# Additional Changes

- display the error message on the IdP linking step in the login UI (in
case there is one)
- display an error in case no option is possible
- exchanged deprecated `eventstoreExpect` with `expectEventstore` in
touched test files

# Additional Context

closes https://github.com/zitadel/zitadel/issues/7393

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-08-14 13:04:26 +00:00
Tim Möhlmann
fcda6580ff
fix(query): print log line on secret generator error (#8424)
# Which Problems Are Solved

Log some details when a secret generator is not found.
This should help us debugging such issue.

# How the Problems Are Solved

When a secret generator by type query fails,
we log the generator type and instance id for which
the generator was requested.

# Additional Changes

- none

# Additional Context

- Related to https://github.com/zitadel/zitadel/issues/8379
- Also encountered in https://github.com/zitadel/zitadel/pull/8407
2024-08-13 14:52:43 +02:00
Elio Bischof
042c438813
feat(v3alpha): read actions (#8357)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details

# Additional Changes

Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.

# Additional Context

- Closes #8169 
- Depends on https://github.com/zitadel/zitadel/pull/8225

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2024-08-12 22:32:01 +02:00
Silvan
cd3ffbd3eb
fix(mirror): use correct statements on push (#8414)
# Which Problems Are Solved

The mirror command used the wrong position to filter for events if
different database technologies for source and destination were used.

# How the Problems Are Solved

The statements which diverge are stored on the client so that different
technologies can use different statements.

# Additional Context

- https://discord.com/channels/927474939156643850/1256396896243552347
2024-08-12 10:33:45 +00:00
Livio Spring
3f25e36fbd
fix: provide device auth config (#8419)
# Which Problems Are Solved

There was no default configuration for `DeviceAuth`, which makes it
impossible to override by environment variables.
Additionally, a custom `CharAmount` value would overwrite also the
`DashInterval`.

# How the Problems Are Solved

- added to defaults.yaml
- fixed customization

# Additional Changes

None.

# Additional Context

- noticed during a customer request
2024-08-12 12:55:07 +03:00
Silvan
d35a0dce7d fix(fields): use read commit isolation level in trigger (#8410)
# Which Problems Are Solved

If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.

# How the Problems Are Solved

Changed the isolation level of the current states query to Read Commited
2024-08-09 11:28:30 +02:00
Silvan
523d73f674
fix(fields): use read commit isolation level in trigger (#8410)
# Which Problems Are Solved

If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.

# How the Problems Are Solved

Changed the isolation level of the current states query to Read Commited
2024-08-09 09:24:28 +00:00
Silvan
b15857821b fix(fields): await running queries during trigger (#8391)
# Which Problems Are Solved

During triggering of the fields table WriteTooOld errors can occure when
using cockroachdb.

# How the Problems Are Solved

The statements exclusively lock the projection before they start to
insert data by using `FOR UPDATE`.
2024-08-06 13:59:06 +02:00
Tim Möhlmann
3759ed9f08 fix(crypto): reject decrypted strings with non-UTF8 characters. (#8374)
# Which Problems Are Solved

We noticed logging where 500: Internal Server errors were returned from
the token endpoint, mostly for the `refresh_token` grant. The error was
thrown by the database as it received non-UTF8 strings for token IDs

Zitadel uses symmetric encryption for opaque tokens, including refresh
tokens. Encrypted values are base64 encoded. It appeared to be possible
to send garbage base64 to the token endpoint, which will pass decryption
and string-splitting. In those cases the resulting ID is not a valid
UTF-8 string.

Invalid non-UTF8 strings are now rejected during token decryption.

# How the Problems Are Solved

- `AESCrypto.DecryptString()` checks if the decrypted bytes only contain
valid UTF-8 characters before converting them into a string.
- `AESCrypto.Decrypt()` is unmodified and still allows decryption on
non-UTF8 byte strings.
- `FromRefreshToken` now uses `DecryptString` instead of `Decrypt`

# Additional Changes

- Unit tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`.
- Fuzz tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`. This was to pinpoint the problem
- Testdata with values that resulted in invalid strings are committed.
In the pipeline this results in the Fuzz tests to execute as regular
unit-test cases. As we don't use the `-fuzz` flag in the pipeline no
further fuzzing is performed.

# Additional Context

- Closes #7765
- https://go.dev/doc/tutorial/fuzz
2024-08-06 13:58:53 +02:00
Silvan
646ffe7a26
fix(fields): await running queries during trigger (#8391)
# Which Problems Are Solved

During triggering of the fields table WriteTooOld errors can occure when
using cockroachdb.

# How the Problems Are Solved

The statements exclusively lock the projection before they start to
insert data by using `FOR UPDATE`.
2024-08-06 11:27:28 +00:00
Tim Möhlmann
4e3fd305ab
fix(crypto): reject decrypted strings with non-UTF8 characters. (#8374)
# Which Problems Are Solved

We noticed logging where 500: Internal Server errors were returned from
the token endpoint, mostly for the `refresh_token` grant. The error was
thrown by the database as it received non-UTF8 strings for token IDs

Zitadel uses symmetric encryption for opaque tokens, including refresh
tokens. Encrypted values are base64 encoded. It appeared to be possible
to send garbage base64 to the token endpoint, which will pass decryption
and string-splitting. In those cases the resulting ID is not a valid
UTF-8 string.

Invalid non-UTF8 strings are now rejected during token decryption.

# How the Problems Are Solved

- `AESCrypto.DecryptString()` checks if the decrypted bytes only contain
valid UTF-8 characters before converting them into a string.
- `AESCrypto.Decrypt()` is unmodified and still allows decryption on
non-UTF8 byte strings.
- `FromRefreshToken` now uses `DecryptString` instead of `Decrypt`

# Additional Changes

- Unit tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`.
- Fuzz tests added for `FromRefreshToken` and
`AESCrypto.DecryptString()`. This was to pinpoint the problem
- Testdata with values that resulted in invalid strings are committed.
In the pipeline this results in the Fuzz tests to execute as regular
unit-test cases. As we don't use the `-fuzz` flag in the pipeline no
further fuzzing is performed.

# Additional Context

- Closes #7765
- https://go.dev/doc/tutorial/fuzz
2024-08-02 08:38:37 +00:00
Livio Spring
3d071fc505
feat: trusted (instance) domains (#8369)
# Which Problems Are Solved

ZITADEL currently selects the instance context based on a HTTP header
(see https://github.com/zitadel/zitadel/issues/8279#issue-2399959845 and
checks it against the list of instance domains. Let's call it instance
or API domain.
For any context based URL (e.g. OAuth, OIDC, SAML endpoints, links in
emails, ...) the requested domain (instance domain) will be used. Let's
call it the public domain.
In cases of proxied setups, all exposed domains (public domains) require
the domain to be managed as instance domain.
This can either be done using the "ExternalDomain" in the runtime config
or via system API, which requires a validation through CustomerPortal on
zitadel.cloud.

# How the Problems Are Solved

- Two new headers / header list are added:
- `InstanceHostHeaders`: an ordered list (first sent wins), which will
be used to match the instance.
(For backward compatibility: the `HTTP1HostHeader`, `HTTP2HostHeader`
and `forwarded`, `x-forwarded-for`, `x-forwarded-host` are checked
afterwards as well)
- `PublicHostHeaders`: an ordered list (first sent wins), which will be
used as public host / domain. This will be checked against a list of
trusted domains on the instance.
- The middleware intercepts all requests to the API and passes a
`DomainCtx` object with the hosts and protocol into the context
(previously only a computed `origin` was passed)
- HTTP / GRPC server do not longer try to match the headers to instances
themself, but use the passed `http.DomainContext` in their interceptors.
- The `RequestedHost` and `RequestedDomain` from authz.Instance are
removed in favor of the `http.DomainContext`
- When authenticating to or signing out from Console UI, the current
`http.DomainContext(ctx).Origin` (already checked by instance
interceptor for validity) is used to compute and dynamically add a
`redirect_uri` and `post_logout_redirect_uri`.
- Gateway passes all configured host headers (previously only did
`x-zitadel-*`)
- Admin API allows to manage trusted domain

# Additional Changes

None

# Additional Context

- part of #8279 
- open topics: 
  - "single-instance" mode
  - Console UI
2024-07-31 18:00:38 +03:00
Elio Bischof
cc3ec1e2a7
feat(v3alpha): write actions (#8225)
# Which Problems Are Solved

The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).

# How the Problems Are Solved

- **Breaking**: The current v3alpha actions APIs are removed. This is
breaking.
- **Resource Namespace**: New v3alpha actions APIs for targets and
executions are added under the namespace /resources.
- **Feature Flag**: New v3alpha actions APIs still have to be activated
using the actions feature flag
- **Reduced Executions Overhead**: Executions are managed similar to
settings according to the new API design: an empty list of targets
basically makes an execution a Noop. So a single method, SetExecution is
enough to cover all use cases. Noop executions are not returned in
future search requests.
- **Compatibility**: The executions created with previous v3alpha APIs
are still available to be managed with the new executions API.

# Additional Changes

- Removed integration tests which test executions but rely on readable
targets. They are added again with #8169

# Additional Context

Closes #8168
2024-07-31 14:42:12 +02:00
Livio Spring
b0e71a81ef
fix: prevent error reason leakage in case of IgnoreUnknownUsernames (#8372)
# Which Problems Are Solved

ZITADEL administrators can enable a setting called "Ignoring unknown
usernames" which helps mitigate attacks that try to guess/enumerate
usernames. If enabled, ZITADEL will show the password prompt even if the
user doesn't exist and report "Username or Password invalid".
Due to a implementation change to prevent deadlocks calling the
database, the flag would not be correctly respected in all cases and an
attacker would gain information if an account exist within ZITADEL,
since the error message shows "object not found" instead of the generic
error message.

# How the Problems Are Solved

- Proper check of the error using an error function / type and
`errors.Is`

# Additional Changes

None.

# Additional Context

- raised in a support request

Co-authored-by: Silvan <silvan.reusser@gmail.com>
(cherry picked from commit a1d24353db)
2024-07-31 14:34:10 +02:00
Livio Spring
f846616a3f
fix: sanitize output for email (#8373)
# Which Problems Are Solved

ZITADEL uses HTML for emails and renders certain information such as
usernames dynamically. That information can be entered by users or
administrators. Due to a missing output sanitization, these emails could
include malicious code.
This may potentially lead to a threat where an attacker, without
privileges, could send out altered notifications that are part of the
registration processes. An attacker could create a malicious link, where
the injected code would be rendered as part of the email.

During investigation of this issue a related issue was found and
mitigated, where on the user's detail page the username was not
sanitized and would also render HTML, giving an attacker the same
vulnerability.

While it was possible to inject HTML including javascript, the execution
of such scripts would be prevented by most email clients and the Content
Security Policy in Console UI.

# How the Problems Are Solved

- All arguments used for email are sanitized (`html.EscapeString`)
- The email text no longer `html.UnescapeString` (HTML in custom text is
still possible)
- Console no longer uses `[innerHtml]` to render the username

# Additional Changes

None

# Additional Context

- raised via email

---------

Co-authored-by: peintnermax <max@caos.ch>
(cherry picked from commit 189505c80f)
2024-07-31 14:34:07 +02:00
Livio Spring
a1d24353db
fix: prevent error reason leakage in case of IgnoreUnknownUsernames (#8372)
# Which Problems Are Solved

ZITADEL administrators can enable a setting called "Ignoring unknown
usernames" which helps mitigate attacks that try to guess/enumerate
usernames. If enabled, ZITADEL will show the password prompt even if the
user doesn't exist and report "Username or Password invalid".
Due to a implementation change to prevent deadlocks calling the
database, the flag would not be correctly respected in all cases and an
attacker would gain information if an account exist within ZITADEL,
since the error message shows "object not found" instead of the generic
error message.

# How the Problems Are Solved

- Proper check of the error using an error function / type and
`errors.Is`

# Additional Changes

None.

# Additional Context

- raised in a support request

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-07-31 14:23:57 +02:00
Livio Spring
189505c80f
fix: sanitize output for email (#8373)
# Which Problems Are Solved

ZITADEL uses HTML for emails and renders certain information such as
usernames dynamically. That information can be entered by users or
administrators. Due to a missing output sanitization, these emails could
include malicious code.
This may potentially lead to a threat where an attacker, without
privileges, could send out altered notifications that are part of the
registration processes. An attacker could create a malicious link, where
the injected code would be rendered as part of the email.

During investigation of this issue a related issue was found and
mitigated, where on the user's detail page the username was not
sanitized and would also render HTML, giving an attacker the same
vulnerability.

While it was possible to inject HTML including javascript, the execution
of such scripts would be prevented by most email clients and the Content
Security Policy in Console UI.

# How the Problems Are Solved

- All arguments used for email are sanitized (`html.EscapeString`)
- The email text no longer `html.UnescapeString` (HTML in custom text is
still possible)
- Console no longer uses `[innerHtml]` to render the username

# Additional Changes

None

# Additional Context

- raised via email

---------

Co-authored-by: peintnermax <max@caos.ch>
2024-07-31 14:21:10 +02:00
Stefan Benz
ce29a78d1b
feat: api v2beta to api v2 (#8283)
# Which Problems Are Solved

The v2beta services are stable but not GA.

# How the Problems Are Solved

The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.

# Additional Context

Closes #7236

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
(cherry picked from commit 7d2d85f57c)
2024-07-29 12:50:41 +02:00
Stefan Benz
7d2d85f57c
feat: api v2beta to api v2 (#8283)
# Which Problems Are Solved

The v2beta services are stable but not GA.

# How the Problems Are Solved

The v2beta services are copied to v2. The corresponding v1 and v2beta
services are deprecated.

# Additional Context

Closes #7236

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2024-07-26 22:39:55 +02:00
RedstonePfalz
57428a1281
fix: Fixed more spelling and grammar misstakes (#8359)
# Which Problems Are Solved

I fixed more spelling and grammar misstakes in the German language
files.

# Additional Context

- Follow-up for PR #8240

Co-authored-by: Fabi <fabienne@zitadel.com>
2024-07-26 11:33:01 +02:00
Livio Spring
3b59b5cb1a
fix(login): correctly render logo based on theme mode (#8355)
# Which Problems Are Solved

The initial load of the login UI with dark mode preference
(prefers-color-scheme: dark) first rendered the logo configured for
light mode. Also switching from dark to light or vice versa would result
in the same behavior.
This was due to a mixed logic of server (based on cookie) and client
(prefers-color-scheme and cookie) deciding which mode to render.

# How the Problems Are Solved

- Since the main logic of which mode to use (`prefers-color-scheme`) can
only be achieve client side, both logos will be served in the HTML and
either will be rendered based on CSS.

# Additional Changes

None

# Additional Context

- closes #2085
2024-07-25 07:38:36 +00:00
Elio Bischof
693e27b906
fix: remove default TOS and privacy links (#8122)
# Which Problems Are Solved

The default terms of service and privacy policy links are applied to all
new ZITADEL instances, also for self hosters. However, the links
contents don't apply to self-hosters.

# How the Problems Are Solved

The links are removed from the DefaultInstance section in the
*defaults.yaml* file.
By default, the links are not shown anymore in the hosted login pages.
They can still be configured using the privacy policy.

# Additional Context

- Found because of a support request
2024-07-25 08:39:10 +02:00
Livio Spring
24ab096e13
Merge branch 'refs/heads/main' into next 2024-07-24 15:47:02 +02:00
Joey Biscoglia
f34897a8c8
feat: add delete phone endpoint to v2 api (#8321)
# Which Problems Are Solved

- Adds delete phone endpoint to v2 api

# How the Problems Are Solved

- Adds new endpoint with DELETE method to /v2beta/users/:userId/phone
which removes currently set phone number

# Additional Changes

- Added integration test for new endpoint.

# Additional Context

- Solves
https://discord.com/channels/927474939156643850/1255557862286032996
2024-07-24 13:46:07 +00:00
Livio Spring
8d13247413
revert: "feat(IDP): use single callback endpoint"
This reverts commit e126ccc9aa.

# Which Problems Are Solved

#8295 introduced the possibility to handle idps on a single callback,
but broke current setups.

# How the Problems Are Solved

- Revert the change until a proper solution is found. Revert is needed
as docs were also changed.

# Additional Changes

None.

# Additional Context

- relates to #8295
2024-07-24 12:29:05 +00:00
Livio Spring
0e99c8356a
fix: correctly check user auth methods and enable button (#8342)
# Which Problems Are Solved

#8291 added backwards compatibilty for users who were created through
the user V2 API and want to sign in to the login UI.
There were however to issues, where users might be prompted to set a
password even if they already had one set or they would not be able to
submit the email verification code.

# How the Problems Are Solved

- Replaced `SearchUserAuthMethods `with `ListUserAuthMethodTypes` to
check for set up auth methods.
- Fixed page / javascript to disable submit button.

# Additional Changes

- Changed `ListActiveUserAuthMethodTypes ` to `ListUserAuthMethodTypes`
and a `activeOnly` boolean parameter

# Additional Context

- relates to #8291
- noticed internally on QA
2024-07-22 14:46:27 +02:00
Zach H
6a97be7bde
fix: always use pgxpool config if MaxOpenConns isn't set (#8328)
# Which Problems Are Solved

- `pgxpool -> pgx` dependency throws "MaxSize must be >= 1" on init if
`postgres.MaxOpenConns` isn't set in the ZItadel config

# How the Problems Are Solved

Only override the `MaxConns` with the Zitadel configured `MaxOpenConns`
if greater than 0 (default value). The default `MaxConns` [is derived by
`pgxpool`](ea9610f672/pgxpool/pool.go (L309-L324))
itself in a sensible way, but somewhat undocumented: checks for explicit
config in connection url or config and falls back on max(num_cpus, 4).

# Additional Changes

Applied same check in cockroach config

# Additional Context

This is likely a regression from the changes in
https://github.com/zitadel/zitadel/pull/8325

(cherry picked from commit e009ed9fe4)
2024-07-22 13:29:09 +02:00
Silvan
93be062d30
refactor(database): exchange connection pool (#8325)
# Which Problems Are Solved

The connection pool of go uses a high amount of database connections.

# How the Problems Are Solved

The standard lib connection pool was replaced by `pgxpool.Pool`

# Additional Changes

The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/7639

(cherry picked from commit 99c645cc60)
2024-07-22 13:28:57 +02:00
Livio Spring
d85af6ad9f
fix: correctly differ between grpc and grpc-web (#8292)
# Which Problems Are Solved

While #8285 also checked for `+proto` and `+json` grpc content types, it
accidentally matched all grpc-web requests to grpc.

# How the Problems Are Solved

- fixed the regex by checking for an exact match (added start `^` and
end `$` anchors)

# Additional Changes

None

# Additional Context

- relates to #8285

(cherry picked from commit d7c0ec282a)
2024-07-22 13:27:07 +02:00