Commit Graph

3776 Commits

Author SHA1 Message Date
Livio Spring
d71795c433 fix: remove index es_instance_position (#9862)
# Which Problems Are Solved

#9837 added a new index `es_instance_position` on the events table with
the idea to improve performance for some projections. Unfortunately, it
makes it worse for almost all projections and would only improve the
situation for the events handler of the actions V2 subscriptions.

# How the Problems Are Solved

Remove the index again.

# Additional Changes

None

# Additional Context

relates to #9837
relates to #9863
2025-05-08 08:35:34 +02:00
Maximilian
577bf9c710 docs(legal): Update to DPA and privacy policy documents (May 2025) (#9566)
We are bringing our DPA and privacy policy document in line with our
changes to the corporate structure, changes to subprocessors, and new
cookie technologies.

This PR replaces #3055 which included more changes to terms of service.
The changes to terms of service will follow in a second step.

---------

Co-authored-by: Florian Forster <florian@zitadel.com>
2025-05-07 15:58:21 +00:00
Stefan Benz
21167a4bba fix: add current state for execution handler into setup (#9863)
# Which Problems Are Solved

The execution handler projection handles all events to check if an
execution has to be provided to the worker to execute.
In this logic all events would be processed from the beginning which is
not necessary.

# How the Problems Are Solved

Add the current state to the execution handler projection, to avoid
processing all existing events.

# Additional Changes

Add custom configuration to the default, so that the transactions are
limited to some events.

# Additional Context

None
2025-05-07 14:26:53 +00:00
Stefan Benz
c6aa6385b6 docs: add invalid information to member requests (#9858)
# Which Problems Are Solved

Misleading information on member endpoint requests.

# How the Problems Are Solved

Add comment to member endpoint requests that the request is invalid if
no roles are provided.

# Additional Changes

None

# Additional Context

Closes #9415

Co-authored-by: Fabienne Bühler <fabienne@zitadel.com>
2025-05-07 13:59:02 +00:00
Elio Bischof
898366c537 fix: allow user self deletion (#9828)
# Which Problems Are Solved

Currently, users can't delete themselves using the V2 RemoveUser API
because of the redunant API middleware permission check.

On main, using a machine user PAT to delete the same machine user:

```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
ERROR:
  Code: NotFound
  Message: membership not found (AUTHZ-cdgFk)
  Details:
  1)	{
    	  "@type": "type.googleapis.com/zitadel.v1.ErrorDetail",
    	  "id": "AUTHZ-cdgFk",
    	  "message": "membership not found"
    	}
```

Same on this PRs branch:

```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
{
  "details": {
    "sequence": "3",
    "changeDate": "2025-05-06T13:44:54.349048Z",
    "resourceOwner": "318838541083804033"
  }
}
```

Repeated call
```bash
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"userId": "318838604669387137"}' localhost:8080 zitadel.user.v2.UserService.DeleteUser
ERROR:
  Code: Unauthenticated
  Message: Errors.Token.Invalid (AUTH-7fs1e)
  Details:
  1)	{
    	  "@type": "type.googleapis.com/zitadel.v1.ErrorDetail",
    	  "id": "AUTH-7fs1e",
    	  "message": "Errors.Token.Invalid"
    	}
```

# How the Problems Are Solved

The middleware permission check is disabled and the
domain.PermissionCheck is used exclusively.

# Additional Changes

A new type command.PermissionCheck allows to optionally accept a
permission check for commands, so APIs with middleware permission checks
can omit redundant permission checks by passing nil while APIs without
middleware permission checks can pass one to the command.

# Additional Context

This is a subtask of #9763

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-05-07 15:24:24 +02:00
Iraq Jaber
34892286bf fixup! fixup! fixup! fixup! Merge branch 'main' into org_api 2025-05-07 14:04:51 +02:00
Iraq Jaber
06bc789b99 fixup! fixup! fixup! Merge branch 'main' into org_api 2025-05-07 13:57:08 +02:00
Iraq Jaber
73eb43bb87 fixup! fixup! Merge branch 'main' into org_api 2025-05-07 13:46:17 +02:00
Iraq Jaber
25d4f1af78 fixup! Merge branch 'main' into org_api 2025-05-07 13:36:50 +02:00
Stefan Benz
0d7d4e6af0 docs: extend api design with additional information and examples (#9856)
# Which Problems Are Solved

There were some misunderstandings on how different points would be
needed to be applied into existing API definitions.

# How the Problems Are Solved

- Added structure to the API design
- Added points to context information in requests and responses
- Added examples to responses with context information
- Corrected available pagination messages
- Added pagination and filter examples

# Additional Changes

None

# Additional Context

None
2025-05-07 10:14:01 +02:00
Iraq Jaber
7383c24931 Merge branch 'main' into org_api 2025-05-07 09:26:32 +02:00
Iraq Jaber
4eb67cebcc fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-07 09:02:56 +02:00
Iraq Jaber
d5602e59a1 fixup! fixup! fixup! fixup! fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-07 06:43:47 +02:00
Iraq Jaber
f0ebb3aa59 fixup! fixup! fixup! fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-06 18:23:29 +02:00
Iraq Jaber
1a05c41614 fixup! fixup! fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-06 17:29:26 +02:00
Iraq Jaber
50e0536408 fixup! fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-06 15:56:20 +02:00
Iraq Jaber
6fa53113c5 fixup! fixup! fixup! adding more tests for ListOrganization() 2025-05-06 14:58:01 +02:00
Iraq Jaber
686079ba66 fixup! fixup! adding more tests for ListOrganization() 2025-05-06 14:04:17 +02:00
Iraq Jaber
5422593910 fixup! fixup! adding more tests for ListOrganization() 2025-05-06 09:57:01 +02:00
Iraq Jaber
1d7c414f87 fixup! adding more tests for ListOrganization() 2025-05-06 09:09:30 +02:00
Zach Hirschtritt
8cb1d24b36 fix: add user id index on sessions8 (#9834)
# Which Problems Are Solved

When a user changes their password, Zitadel needs to terminate all of
that user's active sessions. This query can take many seconds on
deployments with large session and user tables. This happens as part of
session projection handling, so doesn't directly impact user experience,
but potentially bogs down the projection handler which isn't great. In
the future, this index could be used to power a "see all of my current
sessions" feature in Zitadel.

# How the Problems Are Solved

Adds new index on `user_id` column on `projections.sessions8` table.
Alternatively, we can index on `(instance_id, user_id)` instead but
opted for keeping the index smaller as we already index on `instance_id`
separately.

# Additional Changes

None

# Additional Context

None

---------

Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
2025-05-06 06:38:19 +00:00
Silvan
a626678004 fix(setup): execute s54 (#9849)
# Which Problems Are Solved

Step 54 was not executed during setup.

# How the Problems Are Solved

Added the step to setup jobs

# Additional Changes

none

# Additional Context

- the step was added in https://github.com/zitadel/zitadel/pull/9837
- thanks to @zhirschtritt for raising this.
2025-05-06 06:15:45 +00:00
Iraq Jaber
4f340dc973 adding more tests for ListOrganization() 2025-05-05 15:23:28 +02:00
Iraq Jaber
932cebcd54 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-05 11:17:54 +02:00
Iraq Jaber
76cc92e5d7 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-05 11:03:25 +02:00
Iraq Jaber
c7b6a2f4df fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-05 10:46:41 +02:00
Iraq Jaber
2105dd232a fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-05 09:33:40 +02:00
Iraq Jaber
b8498e89ee fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-04 18:00:38 +02:00
Iraq Jaber
492ac01af4 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-04 16:58:25 +02:00
Iraq Jaber
e7d1939012 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-04 15:53:33 +02:00
Iraq Jaber
6878e2f6b5 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-03 12:43:51 +02:00
Iraq Jaber
80f61aefc9 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-03 12:42:05 +02:00
Iraq Jaber
3e482e501e fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based 2025-05-03 12:38:45 +02:00
Iraq Jaber
9dbcfc255d fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based
added all Domain related endpoinds, testing not complete
2025-05-02 19:00:09 +02:00
Livio Spring
b1e60e7398 Merge commit from fork
* fix: prevent intent token reuse and add expiry

* fix duplicate

* fix expiration
2025-05-02 13:44:24 +02:00
Tim Möhlmann
bb56b362a7 perf(eventstore): add instance position index (#9837)
# Which Problems Are Solved

Some projection queries took a long time to run. It seems that 1 or more
queries couldn't make proper use of the `es_projection` index. This
might be because of a specific complexity aggregate_type and event_type
arguments, making the index unfeasible for postgres.

# How the Problems Are Solved

Following the index recommendation, add and index that covers just
instance_id and position.

# Additional Changes

- none

# Additional Context

- Related to https://github.com/zitadel/zitadel/issues/9832
2025-05-02 13:40:22 +02:00
Iraq Jaber
4d5cbc94e2 fixup! fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based
added AddOrganizationDomain & ListOrganizationDomain()
2025-05-02 12:10:30 +02:00
Iraq Jaber
96831fb5f5 fixup! fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based
added Deactivate/ActivateOrg
2025-05-01 12:37:14 +02:00
Elio Bischof
74ace1aec3 fix(actions): default sorting column to creation date (#9795)
# Which Problems Are Solved

The sorting column of action targets and executions defaults to the ID
column instead of the creation date column.
This is only relevant, if the sorting column is explicitly passed as
unspecified.
If the sorting column is not passed, it correctly defaults to the
creation date.

```bash
#  Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": "TARGET_FIELD_NAME_UNSPECIFIED"}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
#  Sorts by ID
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" -d '{"sortingColumn": 0}' localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
#  Sorts by creation date
grpcurl -plaintext -H "Authorization: Bearer ${ZITADEL_ACCESS_TOKEN}" localhost:8080 zitadel.action.v2beta.ActionService.ListTargets
``` 

# How the Problems Are Solved

`action.TargetFieldName_TARGET_FIELD_NAME_UNSPECIFIED` maps to the
sorting column `query.TargetColumnCreationDate`.

# Additional Context

As IDs are also generated in ascending, like creation dates, the the bug
probably only causes unexpected behavior for cases, where the ID is
specified during target or execution creation. This is currently not
supported, so this bug probably has no impact at all. It doesn't need to
be backported.

Found during implementation of #9763

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-05-01 05:41:57 +00:00
Ramon
02acc93242 fix: Improve Actions V2 translations (#9826)
# Which Problems Are Solved
The translation for event was not loaded correctly.

![grafik](https://github.com/user-attachments/assets/3fa8d72f-f55a-44b7-997d-0f0976f66b85)

# How the Problems Are Solved
Correct translations to have the correct key.

# Additional Changes
Improved the translation for all events.
2025-04-30 13:20:39 +00:00
Stefan Benz
a05f7ce3fc fix: correct handling of removed targets (#9824)
# Which Problems Are Solved

In Actions v2, if a target is removed, which is still used in an
execution, the target is still listed when list executions.

# How the Problems Are Solved

Removed targets are now also removed from the executions.

# Additional Changes

To be sure the list executions include a check if the target is still
existing.

# Additional Context

None

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-04-30 12:58:10 +00:00
Ramon
48c1f7e49f fix: Actions V2 improve deleted target handling in executions (#9822)
# Which Problems Are Solved
Previously, if a target was deleted but still referenced by an
execution, it became impossible to load the executions.

# How the Problems Are Solved
Missing targets in the execution table are now gracefully ignored,
allowing executions to load without errors.

# Additional Changes
Enhanced permission handling in the settings sidenav to ensure users
have the correct access rights.
2025-04-30 12:22:27 +00:00
Ramon
002c3eb025 fix: Use ID ordering for the executions in Actions v2 (#9820)
# Which Problems Are Solved

Sort Executions by ID in the Actions V2 view. This way All is the first
element in the table.

# How the Problems Are Solved
Pass ID sorting to the Backend.

# Additional Changes
Cleaned up some imports.

# Additional Context
- Part of Make actions sortable by hirarchie #9688
2025-04-30 11:16:44 +00:00
Stefan Benz
3953879fe9 fix: correct unmarshalling of IdP user when using Google (#9799)
# Which Problems Are Solved

Users from Google IDP's are not unmarshalled correctly in intent
endpoints and not returned to callers.

# How the Problems Are Solved

Provided correct type for unmarshalling of the information.

# Additional Changes

None

# Additional Context

None

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-04-30 09:12:48 +00:00
Tim Möhlmann
0465d5093e fix(features): remove the improved performance enumer (#9819)
# Which Problems Are Solved

Instance that had improved performance flags set, got event errors when
getting instance features. This is because the improved performance
flags were marshalled using the enumerated integers, but now needed to
be unmashalled using the added UnmarshallText method.

# How the Problems Are Solved

- Remove emnumer generation

# Additional Changes

- none

# Additional Context

- reported on QA
- Backport to next-rc / v3
2025-04-30 08:26:04 +00:00
Iraq Jaber
40de3d462a fixup! fixup! fixup! fixup! fixup! fixup! refactor(api): moving organization API resourced based
added DeleteOrganization()
2025-04-30 10:16:22 +02:00
Silvan
181186e477 fix(mirror): add max auth request age configuration (#9812)
# Which Problems Are Solved

The `auth.auth_requests` table is not cleaned up so long running Zitadel
installations can contain many rows.
 
The mirror command can take long because a the data are first copied
into memory (or disk) on cockroach and users do not get any output from
mirror. This is unfortunate because people don't know if Zitadel got
stuck.

# How the Problems Are Solved

Enhance logging throughout the projection processes and introduce a
configuration option for the maximum age of authentication requests.

# Additional Changes

None

# Additional Context

closes https://github.com/zitadel/zitadel/issues/9764

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-04-29 15:29:16 +00:00
Tim Möhlmann
91bc71db74 fix(instance): add web key generation to instance defaults (#9815)
# Which Problems Are Solved

Webkeys were not generated with new instances when the webkey feature
flag was enabled for instance defaults. This would cause a redirect loop
with console for new instances on QA / coud.

# How the Problems Are Solved

- uncomment the webkeys section on defaults.yaml
- Fix field naming of webkey config

# Additional Changes

- Add all available features as comments.
- Make the improved performance type enum parsable from the config,
untill now they were just ints.
- Running of the enumer command created missing enum entries for feature
keys.

# Additional Context

- Needs to be back-ported to v3 / next-rc

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-04-29 14:54:53 +00:00
Elio Bischof
fa3efd9da3 docs: fix Illegal byte sequence (#9750)
# Which Problems Are Solved

In some docs pages, we propose to generate a Zitadel masterkey using the
command `tr -dc A-Za-z0-9 </dev/urandom | head -c 32`. However, this
fails on some systems/locations with the error message `tr: Illegal byte
sequence`.

# How the Problems Are Solved

We replace the command by this more portable variant: `LC_ALL=C tr -dc
'[:graph:]' </dev/urandom | head -c 32`

# Additional Changes

None

# Additional Context

Found by @fcoppede while testing #9496. The new command works for him.
2025-04-29 14:33:23 +00:00
Elio Bischof
c36b0ab2e2 docs(self-hosting): add login to lb example (#9496)
# Which Problems Are Solved

We have no docs for self-hosting the login using the standard login as a
standalone docker container.

# How the Problems Are Solved

A common self-hosting case is to publish the login at the same domain as
Zitadel behind a reverse proxy.
That's why we extend the load balancing example.
We refocus the example from *making TLS work* to *running multiple
services behind the proxy and connect them using an internal network and
DNS*. I decided this together with @fforootd.

For authenticating with the login application, we have to set up a
service user and give it the role IAM_LOGIN_CLIENT. We do so in the
use-new-login "job" container as `zitadel setup` only supports Zitadel
users with the role IAM_ADMIN AFAIR.

The login application relies on a healthy Zitadel API on startup, which
is why we fix the containers readiness reports.

# Additional Changes

- We deploy the init and setup jobs independently, because this better
reflects our production recommendatinons.
It gives more control over the upgrade process.
- We use the ExternalDomain *127.0.0.1.sslip.io* instead of *my.domain*,
because this doesn't require changing the local DNS resolution by
changing */etc/hosts* for local tests.

# Testing

The commands in the preview docs use to the configuration files on main.
This is fine when the PR is merged but not for testing the PR.
Replace the used links to make them point to the PRs changed files.
Instead of the commands in the preview docs, use these: 

```bash
# Download the docker compose example configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/docs-compose-login/docs/docs/self-hosting/deploy/loadbalancing-example/docker-compose.yaml

# Download the Traefik example configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/docs-compose-login/docs/docs/self-hosting/deploy/loadbalancing-example/example-traefik.yaml

# Download and adjust the example configuration file containing standard configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/docs-compose-login/docs/docs/self-hosting/deploy/loadbalancing-example/example-zitadel-config.yaml

# Download and adjust the example configuration file containing secret configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/docs-compose-login/docs/docs/self-hosting/deploy/loadbalancing-example/example-zitadel-secrets.yaml

# Download and adjust the example configuration file containing database initialization configuration.
wget https://raw.githubusercontent.com/zitadel/zitadel/refs/heads/docs-compose-login/docs/docs/self-hosting/deploy/loadbalancing-example/example-zitadel-init-steps.yaml

# A single ZITADEL instance always needs the same 32 bytes long masterkey
# Generate one to a file if you haven't done so already and pass it as environment variable
LC_ALL=C tr -dc '[:graph:]' </dev/urandom | head -c 32 > ./zitadel-masterkey
export ZITADEL_MASTERKEY="$(cat ./zitadel-masterkey)"

# Run the database and application containers
docker compose up --detach --wait
```

# Additional Context

- Closes https://github.com/zitadel/DevOps/issues/111
- Depends on https://github.com/zitadel/typescript/pull/412
- Contributes to road map item
https://github.com/zitadel/zitadel/issues/9481
2025-04-29 14:12:34 +00:00