# Which Problems Are Solved
1. Adding users to user groups and removing users from user groups.
2. Searching for users in user groups by group IDs or user IDs
# How the Problems Are Solved
By adding:
1. The API definitions to manage users in users groups
3. The command-layer implementation of adding users/removing users
to/from user groups.
4. The projection table group_users1
5. Query-side implementation to search for users in user groups
# Additional Changes
1. Remove debug statements from unit tests.
2. Fix removal of groups when orgs are removed
3. Add unit tests for groups projection
# Additional Context
* Related to #9702
* Follow-up for PRs
* https://github.com/zitadel/zitadel/pull/10455
* https://github.com/zitadel/zitadel/pull/10758
* https://github.com/zitadel/zitadel/pull/10853
# Which Problems Are Solved
As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.
# How the Problems Are Solved
- This PR moves the authorization v2beta service and its endpoints to a
corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The docs are moved to the new GA service and its endpoints. The v2beta
is not displayed anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing.
- The `organization_id` to create an authorization is now required to be
always passed. There's no implicit fallback to the project's
organization anymore.
- The `user_id` filter has been removed in favor of the recently added
`in_user_ids` filter.
- The returned `Authorization` object has been reworked to return
`project`, `organization` and `roles` as objects like the granted `user`
already was.
- Additionally the `roles` now not only contain the granted `role_keys`,
but also the `display_name` and `group`. To implement this the query has
been updated internally. Existing APIs are unchanged and still return
just the keys.
# Additional Changes
None
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/10772
- closes#10746
- requires backport to v4.x
# Which Problems Are Solved
During normal operations, when listing authentication method types,
activated IDPs are added to the `userAuthMethodTypes` slice but there's
an error in the logs, causing false alarms in our alerting system which
has been configured to trigger on error-level logs:
```
{
"logDate": "2025-09-29T07:47:49.524794306Z",
"protocol": 0,
"requestUrl": "/zitadel.user.v2.UserService/ListAuthenticationMethodTypes",
"responseStatus": 0,
"requestHeaders": {
"authorization": [
"[REDACTED]"
],
"content-type": [
"application/grpc+proto"
],
"grpc-accept-encoding": [
"gzip,br"
],
"host": [
"zitadel.example.com"
],
"te": [
"trailers"
],
"user-agent": [
"connect-es/2.0.4"
]
},
"responseHeaders": {},
"instanceId": "329151080840616127",
"projectId": "329851980840157809",
"requestedDomain": "zitadel.example.com",
"requestedHost": "zitadel.example.com"
}
{
"caller": "/home/runner/work/zitadel/zitadel/internal/query/user_auth_method.go:478",
"level": "error",
"msg": "IDP1",
"time": "2025-09-29T07:47:49Z"
}
```
# How the Problems Are Solved
The log was changed to debug level and the log text was updated to
better describe what is happening.
Another possible solution would be to remove the log line completely.
Reviewers can decide if this is a better solution.
# Additional Changes
None.
# Additional Context
None.
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Relational table's primary keys are part of the SQL table definition.
However, by abstraction through the repository interfaces callers
(service layer) have no clear understanding of the primary key definion.
This might lead to confusion during implementation:
- The repository makes a sanity check on required conditions before
query execution and returns an error if the condition was not met.
- When a Get method is executed, an error is returned if not exactly one
row is found. Get replaces the legacy GetXxxByID methods we have on
projections. However, a list of flexible conditions can be passed.
- Operations like UPDATE and DELETE should often only affect a single
row, identified by the primary key. If the primary key conditions are
missing an error is returned.
Because there is no "type safety" enforced here, the errors are returned
during runtime. When all possible combinations of API arguments are not
properly tested such errors are actually bugs in the experience of the
end-user. (The API should do it's own input validation, but a
programmer's error is easily made).
# How the Problems Are Solved
Add `PrimaryKeyColumns` and `PrimaryKeyCondition` to existing repository
intefaces.
# Additional Changes
- Where reducers already used repositories, the PrimaryKeyConditions are
used where possible.
# Additional Context
- Part of
https://github.com/zitadel/zitadel/wiki/Decision-Log#expose-primary-key-definition-in-repository-interfaces
---------
Co-authored-by: Marco A. <marco@zitadel.com>
# Which Problems Are Solved
Ensuring the user group resource is managed with appropriate
permissions.
# How the Problems Are Solved
By configuring and checking for the relevant permissions needed to
create, read, update, and delete the user groups resource.
# Additional Changes
N/A
# Additional Context
- Related to #9702
- Follow-up for PRs #10455, #10758
# Which Problems Are Solved
As part of our efforts to simplify the structure and versions of our
APIs, were moving all existing v2beta endpoints to v2 and deprecate
them. They will be removed in Zitadel V5.
# How the Problems Are Solved
- This PR moves the internal permission v2beta service and its endpoints
to a corresponding v2 version. The v2beta service and endpoints are
deprecated.
- The docs are moved to the new GA service and its endpoints. The v2beta
is not displayed anymore.
- The comments and have been improved and, where not already done, moved
from swagger annotations to proto.
- All required fields have been marked with (google.api.field_behavior)
= REQUIRED and validation rules have been added where missing.
- Listing administrators of a project grant can now be done with the
`ProjectGrant` (`project_id` and `organization_id`) instead of a
`project_id`, which corresponds to creation of the administrator ship of
such grant.
- formatted using `buf`
# Additional Changes
None
# Additional Context
- part of https://github.com/zitadel/zitadel/issues/10772
- requires backport to v4.x
---------
Co-authored-by: Gayathri Vijayan <66356931+grvijayan@users.noreply.github.com>
# Which Problems Are Solved
When listing / searching users, each user got multiplied by the amount
of metadata entries they have, towards the `total_results` count. In
PostgreSQL the `COUNT(*) OVER()` window function does not support
`DISTINCT`. Even tho the query did a distinct select, the count would
still include duplicates.
# How the Problems Are Solved
Wrap the original query in a sub-select, so that the `DISTINCT` gets
handled before the count window function is executed in the outer
function. Filters, permission and solting is applied to the inner query.
Offset, limit and count are applied to the outer query.
# Additional Changes
- none
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/10825
- Backport to 4v
This pull request introduces a new feature that allows adding, updating,
and querying metadata for organizations. The changes are primarily in
the backend and include new database tables, repositories, and domain
logic to support organization metadata.
## Changes
* New `org_metadata` table: A new table `zitadel.org_metadata` is
introduced to store metadata for organizations. It includes columns for
`instance_id`, `org_id`, `key`, and `value`. The `value` is stored as a
`BYTEA` type to allow for flexible data storage.
* New `OrganizationMetadataRepository`: A new repository
`OrganizationMetadataRepository` is created to handle all database
operations for organization metadata. It provides methods to `Get`,
`List`, `Set`, and `Remove` metadata.
* New `org_metadata_relational_projection`: A new projection
`org_metadata_relational_projection` is added to update the
`zitadel.org_metadata` table based on events. It handles `MetadataSet`,
`MetadataRemoved`, and `MetadataRemovedAll` events.
* Updated `OrganizationRepository`: The `OrganizationRepository` is
updated to support loading organization metadata. A new method
`LoadMetadata` is added to enable joining the `org_metadata` table when
querying for organizations.
* Updated Organization domain: The Organization domain model is updated
to include a new field `Metadata` of type `[]*OrganizationMetadata`.
## Additional Info
* Extensible Design: The new metadata feature is designed to be
extensible, allowing for future enhancements such as indexing on
specific JSON fields within the `value` column.
* closes https://github.com/zitadel/zitadel/issues/10206
* closes https://github.com/zitadel/zitadel/issues/10214
# Which Problems Are Solved
This is the second PR related to the backend implementation of
GroupService to manage user groups.
The first [PR](https://github.com/zitadel/zitadel/pull/10455) implements
the Command-side.
This PR implements the query side.
# How the Problems Are Solved
* Query-side implementation to search/list groups by
* a list of Group IDs
* by the Group name
* by the Organization ID
# Additional Changes
N/A
# Additional Context
- Follow-up for PR #10455
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Currently there is only the option to either filter for all usergrants
of an organization or the usergrants of a singluar user.
# How the Problems Are Solved
Add the option to provide a list of userIDs to query user grants.
# Additional Changes
Fixed internal typo for function.
# Additional Context
Closes#9675
# Which Problems Are Solved
This PR adds API definition and backend implementation for GroupService
to manage user groups.
# How the Problems Are Solved
* API definition to create, update, retrieve, and delete groups is added
* Command-side implementation to create, update, and delete user groups
as part of the GroupV2 API is added
# Additional Changes
N/A
# Additional Context
- Related to #10089, #9702 (parent ticket)
- User contribution: https://github.com/zitadel/zitadel/pull/9428/files
- Additional functionalities to list/search user groups, add
permissions, manage users in groups, group scopes will be added in
subsequent PRs.
- Also needs documentation, which will be added once the entire feature
is available
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
Add projects to the relational tables
# How the Problems Are Solved
- Define table migrations
- Define and implement Project and Project Role repositories.
- Provide projection handlers to populate the relational tables.
# Additional Changes
- Statement Builder now has a constructor which allows setting of a base
query with arguments.
- Certain operations, like Get, Update and Delete require the Primary
Key to be set as conditions. However, this requires knowledge of the
implementation and table definition. This PR proposes an additional
condition for repositories: `PrimaryKeyCondition`. This gives clarity on
the required IDs for these operations.
- Added couple of helpers to the repository package, to allow more DRY
code.
- getOne / getMany: generic functions for query execution and scanning.
- checkRestrictingColumns, checkPkCondition: simplify condition
checking, instead of using ladders of conditionals.
- Added a couple of helpers to the repository test package:
- Transaction, savepoint and rollback helpers.
- Create instance and organization helpers for objects that depend on
them (like projects).
# Additional Context
- after https://github.com/zitadel/zitadel/pull/10809
- closes#10765
This pull request introduces a significant refactoring of the database
interaction layer, focusing on improving explicitness, transactional
control, and error handling. The core change is the removal of the
stateful `QueryExecutor` from repository instances. Instead, it is now
passed as an argument to each method that interacts with the database.
This change makes transaction management more explicit and flexible, as
the same repository instance can be used with a database pool or a
specific transaction without needing to be re-instantiated.
### Key Changes
- **Explicit `QueryExecutor` Passing:**
- All repository methods (`Get`, `List`, `Create`, `Update`, `Delete`,
etc.) in `InstanceRepository`, `OrganizationRepository`,
`UserRepository`, and their sub-repositories now require a
`database.QueryExecutor` (e.g., a `*pgxpool.Pool` or `pgx.Tx`) as the
first argument.
- Repository constructors no longer accept a `QueryExecutor`. For
example, `repository.InstanceRepository(pool)` is now
`repository.InstanceRepository()`.
- **Enhanced Error Handling:**
- A new `database.MissingConditionError` is introduced to enforce
required query conditions, such as ensuring an `instance_id` is always
present in `UPDATE` and `DELETE` operations.
- The database error wrapper in the `postgres` package now correctly
identifies and wraps `pgx.ErrTooManyRows` and similar errors from the
`scany` library into a `database.MultipleRowsFoundError`.
- **Improved Database Conditions:**
- The `database.Condition` interface now includes a
`ContainsColumn(Column) bool` method. This allows for runtime checks to
ensure that critical filters (like `instance_id`) are included in a
query, preventing accidental cross-tenant data modification.
- A new `database.Exists()` condition has been added to support `EXISTS`
subqueries, enabling more complex filtering logic, such as finding an
organization that has a specific domain.
- **Repository and Interface Refactoring:**
- The method for loading related entities (e.g., domains for an
organization) has been changed from a boolean flag (`Domains(true)`) to
a more explicit, chainable method (`LoadDomains()`). This returns a new
repository instance configured to load the sub-resource, promoting
immutability.
- The custom `OrgIdentifierCondition` has been removed in favor of using
the standard `database.Condition` interface, simplifying the API.
- **Code Cleanup and Test Updates:**
- Unnecessary struct embeddings and metadata have been removed.
- All integration and repository tests have been updated to reflect the
new method signatures, passing the database pool or transaction object
explicitly.
- New tests have been added to cover the new `ExistsDomain`
functionality and other enhancements.
These changes make the data access layer more robust, predictable, and
easier to work with, especially in the context of database transactions.
# Which Problems Are Solved
When exporting users, an error `QUERY-AG4gs` was returned. This was due
to #10750, where the orderBy column was added to the query to prevent
the exact same error. In case there was no sorting column specified,
such as the export, the query would fail.
# How the Problems Are Solved
- Added a default sorting on `id` as we already have for the other APIs.
# Additional Changes
None
# Additional Context
- reported through support
- relates to #10750, #10415
- backport to v4.x
# Which Problems Are Solved
#10415 added the possibility to filter users based on metadata. To
prevent duplicate results an sql `DISTINCT` was added. This resulted in
issues if the list was sorted on string columns like `username` or
`displayname`, since they are sorted using `lower`. Using `DISTINCT`
requires the `order by` column to be part of the `SELECT` statement.
# How the Problems Are Solved
Added the order by column to the statement.
# Additional Changes
None
# Additional Context
- relates to #10415
- backport to v4.x
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
Depending on the metadata values (already existing), the newly created
index (#10415) cannot be created or error in the future.
# How the Problems Are Solved
- Create the index using `sha256` and change the query to use sha256 as
well when comparing bytes values such as user_metadata.
- Added a setup step to cleanup potentially created index on
`projections.user_metadata5`
# Additional Changes
None
# Additional Context
- relates to #10415
- requires backport to v4.x
…of project grant
# Which Problems Are Solved
On Management API the fields for `GrantedOrgId`, `GrantedOrgName` and
`GrantedOrgDomain` were only filled if it was a usergrant for a granted
project.
# How the Problems Are Solved
Correctly query the Organization of the User again to the Organization
the Project is granted to.
Then fill in the information about the Organization of the User in the
fields `GrantedOrgId`, `GrantedOrgName` and `GrantedOrgDomain`.
# Additional Changes
Additionally query the information about the Organization the Project is
granted to, to have it available for the Authorization v2beta API.
# Additional Context
Closes#10723
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
Cached object may have a different schema between Zitadel versions.
# How the Problems Are Solved
Use the curent build version in DB based cache connectors PostgreSQL and
Redis.
# Additional Changes
- Cleanup the ZitadelVersion field from the authz Instance
- Solve potential race condition on global variables in build package.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/10648
- Obsoletes https://github.com/zitadel/zitadel/pull/10646
- Needs to be back-ported to v4 over
https://github.com/zitadel/zitadel/pull/10645
Closes#10498
The registration form's legal checkboxes had incorrect validation logic
that prevented users from completing registration when only one legal
document (ToS or Privacy Policy) was configured, or when no legal
documents were required.
additionally removes a duplicate description for "or use Identity
Provider"
# Which Problems Are Solved
Having only partial legal documents was blocking users to register. The
logic now conditionally renders checkboxes and checks if all provided
documents are accepted.
# How the Problems Are Solved
- Fixed checkbox validation: Now properly validates based on which legal
documents are actually available
- acceptance logic: Only requires acceptance of checkboxes that are
shown
- No legal docs support: Users can proceed when no legal documents are
configured
- Proper state management: Fixed checkbox state tracking and mixed-up
test IDs
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
HTTP Request to HTTP providers for Email or SMS are not signed.
# How the Problems Are Solved
Add a Signing Key to the HTTP Provider resources, which is then used to
generate a header to sign the payload.
# Additional Changes
Additional tests for query side of the SMTP provider.
# Additional Context
Closes#10067
---------
Co-authored-by: Marco A. <marco@zitadel.com>
This PR initiates the rework of Zitadel's backend to state-persisted
objects. This change is a step towards a more scalable and maintainable
architecture.
## Changes
* **New `/backend/v3` package**: A new package structure has been
introduced to house the reworked backend logic. This includes:
* `domain`: Contains the core business logic, commands, and repository
interfaces.
* `storage`: Implements the repository interfaces for database
interactions with new transactional tables.
* `telemetry`: Provides logging and tracing capabilities.
* **Transactional Tables**: New database tables have been defined for
`instances`, `instance_domains`, `organizations`, and `org_domains`.
* **Projections**: New projections have been created to populate the new
relational tables from the existing event store, ensuring data
consistency during the migration.
* **Repositories**: New repositories provide an abstraction layer for
accessing and manipulating the data in the new tables.
* **Setup**: A new setup step for `TransactionalTables` has been added
to manage the database migrations for the new tables.
This PR lays the foundation for future work to fully transition to
state-persisted objects for these components, which will improve
performance and simplify data access patterns.
This PR initiates the rework of ZITADEL's backend to state-persisted
objects. This is a foundational step towards a new architecture that
will improve performance and maintainability.
The following objects are migrated from event-sourced aggregates to
state-persisted objects:
* Instances
* incl. Domains
* Orgs
* incl. Domains
The structure of the new backend implementation follows the software
architecture defined in this [wiki
page](https://github.com/zitadel/zitadel/wiki/Software-Architecturel).
This PR includes:
* The initial implementation of the new transactional repositories for
the objects listed above.
* Projections to populate the new relational tables from the existing
event store.
* Adjustments to the build and test process to accommodate the new
backend structure.
This is a work in progress and further changes will be made to complete
the migration.
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Iraq Jaber <iraq+github@zitadel.com>
Co-authored-by: Iraq <66622793+kkrime@users.noreply.github.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
This PR overhauls our event projection system to make it more robust and
prevent skipped events under high load. The core change replaces our
custom, transaction-based locking with standard PostgreSQL advisory
locks. We also introduce a worker pool to manage concurrency and prevent
database connection exhaustion.
### Key Changes
* **Advisory Locks for Projections:** Replaces exclusive row locks and
inspection of `pg_stat_activity` with PostgreSQL advisory locks for
managing projection state. This is a more reliable and standard approach
to distributed locking.
* **Simplified Await Logic:** Removes the complex logic for awaiting
open transactions, simplifying it to a more straightforward time-based
filtering of events.
* **Projection Worker Pool:** Implements a worker pool to limit
concurrent projection triggers, preventing connection exhaustion and
improving stability under load. A new `MaxParallelTriggers`
configuration option is introduced.
### Problem Solved
Under high throughput, a race condition could cause projections to miss
events from the eventstore. This led to inconsistent data in projection
tables (e.g., a user grant might be missing). This PR fixes the
underlying locking and concurrency issues to ensure all events are
processed reliably.
### How it Works
1. **Event Writing:** When writing events, a *shared* advisory lock is
taken. This signals that a write is in progress.
2. **Event Handling (Projections):**
* A projection worker attempts to acquire an *exclusive* advisory lock
for that specific projection. If the lock is already held, it means
another worker is on the job, so the current one backs off.
* Once the lock is acquired, the worker briefly acquires and releases
the same *shared* lock used by event writers. This acts as a barrier,
ensuring it waits for any in-flight writes to complete.
* Finally, it processes all events that occurred before its transaction
began.
### Additional Information
* ZITADEL no longer modifies the `application_name` PostgreSQL variable
during event writes.
* The lock on the `current_states` table is now `FOR NO KEY UPDATE`.
* Fixes https://github.com/zitadel/zitadel/issues/8509
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
This PR introduces a new feature flag `EnableRelationalTables` that will
be used in following implementations to decide whether Zitadel should
use the relational model or the event sourcing one.
# TODO
- [x] Implement flag at system level
- [x] Display the flag on console:
https://github.com/zitadel/zitadel/pull/10615
# How the Problems Are Solved
- Implement loading the flag from config
- Add persistence of the flag through gRPC endpoint
(SetInstanceFeatures)
- Implement reading of the flag through gRPC endpoint
(GetInstanceFeatures)
# Additional Changes
Some minor refactoring to remove un-needed generics annotations
# Additional Context
- Closes#10574
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
Some users have reported the need of retrieving users given a metadata
key, metadata value or both. This change introduces metadata search
filter on the `ListUsers()` endpoint to allow Zitadel users to search
for user records by metadata.
The changes affect only v2 APIs.
# How the Problems Are Solved
- Add new search filter to `ListUserRequest`: `MetaKey` and `MetaValue`
- Add SQL indices on metadata key and metadata value
- Update query to left join `user_metadata` table
# Additional Context
- Closes#9053
- Depends on https://github.com/zitadel/zitadel/pull/10567
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
The event execution system currently uses a projection handler that
subscribes to and processes all events for all instances. This creates a
high static cost because the system over-fetches event data, handling
many events that are not needed by most instances. This inefficiency is
also reflected in high "rows returned" metrics in the database.
# How the Problems Are Solved
Eliminate the use of a project handler. Instead, events for which
"execution targets" are defined, are directly pushed to the queue by the
eventstore. A Router is populated in the Instance object in the authz
middleware.
- By joining the execution targets to the instance, no additional
queries are needed anymore.
- As part of the instance object, execution targets are now cached as
well.
- Events are queued within the same transaction, giving transactional
guarantees on delivery.
- Uses the "insert many fast` variant of River. Multiple jobs are queued
in a single round-trip to the database.
- Fix compatibility with PostgreSQL 15
# Additional Changes
- The signing key was stored as plain-text in the river job payload in
the DB. This violated our [Secrets
Storage](https://zitadel.com/docs/concepts/architecture/secrets#secrets-storage)
principle. This change removed the field and only uses the encrypted
version of the signing key.
- Fixed the target ordering from descending to ascending.
- Some minor linter warnings on the use of `io.WriteString()`.
# Additional Context
- Introduced in https://github.com/zitadel/zitadel/pull/9249
- Closes https://github.com/zitadel/zitadel/issues/10553
- Closes https://github.com/zitadel/zitadel/issues/9832
- Closes https://github.com/zitadel/zitadel/issues/10372
- Closes https://github.com/zitadel/zitadel/issues/10492
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
- Most inputs have hardcoded label
# How the Problems Are Solved
- add usage of i18n library for every label
- add labels to i18n translation files
# Additional Changes
- fixed key used in `device-code-form.tsx` by submit button
- `v2-default.json` was update and contains all values from login app
not only newly added key for labels.
# Additional Context
N.A
---------
Co-authored-by: David Skewis <david@zitadel.com>
Co-authored-by: Max Peintner <max@caos.ch>
# Which Problems Are Solved
When a SAML IDP is created, the signing algorithm defaults to
`RSA-SHA1`.
This PR adds the functionality to configure the signing algorithm while
creating or updating a SAML IDP. When nothing is specified, `RSA-SHA1`
is the default.
Available options:
* RSA_SHA1
* RSA_SHA256
* RSA_SHA512
# How the Problems Are Solved
By introducing a new optional config to specify the Signing Algorithm.
# Additional Changes
N/A
# Additional Context
- Closes#9842
An existing bug in the UpdateSAMLProvider API will be fixed as a
followup in a different
[PR](https://github.com/zitadel/zitadel/pull/10557).
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
While investigating a support ticket, it was discovered that some
queries using equals or not equals without case matching were not
correctly escaping the value to compare. If a value contained a
backslash (`\`) the row would not match.
# How the Problems Are Solved
- Fixed the escaping for backslash for `like` operations.
- Changed equals and not equals comparison without case matching to `=`
instead of `like`.
# Additional Changes
None
# Additional Context
- related to a support request
- requires backport to v.3 and v4.x
The `/introspect` endpoint showed poor performance during v4 load
testing due to an inefficient database query in
`internal/query/introspection_client_by_id.sql`. This PR optimizes the
query structure to significantly improve performance.
## Query Optimizations
**UNION → UNION ALL**: Changed expensive `UNION` to `UNION ALL` since
`client_id` is unique across both API and OIDC config tables,
eliminating unnecessary deduplication overhead (30-50% improvement
expected).
**Simplified Keys CTE**: Optimized the keys lookup logic by using
`$2::text as client_id` instead of `identifier as client_id` with `group
by`, and added explicit `$3 = true` condition to the LEFT JOIN for
better query planning.
**Enhanced Readability**: Added consistent table aliases (c, a, p, o, k)
for better maintainability.
## Benefits
- **Zero-downtime deployment**: Uses existing database indexes, no
schema changes required
- **Secondary performance gains**: Other similar queries
(`oidc_client_by_id.sql`, `userinfo_client_by_id.sql`) will also benefit
from the optimizations
- **Minimal code changes**: Only 13 lines added, 9 lines removed in the
SQL query
- **Backward compatible**: Same result set and API behavior
The optimized query maintains the same functionality while providing
significant performance improvements for the introspection endpoint
under high concurrent load.
Fixes#10389.
<!-- START COPILOT CODING AGENT TIPS -->
---
💬 Share your feedback on Copilot coding agent for the chance to win a
$200 gift card! Click
[here](https://survey.alchemer.com/s3/8343779/Copilot-Coding-agent) to
start the survey.
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: muhlemmer <5411563+muhlemmer@users.noreply.github.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
# Which Problems Are Solved
Some events that are now unused are clogging the event queue from time
to time.
# How the Problems Are Solved
Remove the events described in #10458
# Additional Changes
- Updated `stringer` and `enumer` in Makefile target `core_generate_all`
to resolve generated files compilation issues
# Notes
It looks like there are a lot of changes, but most of it is fixing
translation files. I suggest doing a review per-commit
# Additional Context
- Closes#10458
- Depends on https://github.com/zitadel/zitadel/pull/10513
# Which Problems Are Solved
`ListAppKeys()` does not work properly, in that it does not return any
app keys.
# How the Problems Are Solved
The issue stems from a mistake SQL query not joining the
`projections.authn_keys2` table to `projections.projects4` instead of
joining to `projections.apps7`
# Additional Changes
`ListAppKeys()` returns the app key IDs in order of their creation
- Closes https://github.com/zitadel/zitadel/issues/10420
- backport to v4.x
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
It should not be possible to start 2 projections with the same name.
If this happens, it can cause issues with the event store such as events
being skipped/unprocessed and can be very hard/time-consuming to
diagnose.
# How the Problems Are Solved
A check was added to make sure no 2 projections have the same table
Closes https://github.com/zitadel/zitadel/issues/10453
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
When searching for an existing external userID from an IdP response, the
comparison is case sensitive. This can lead to issues esp. when using
SAML, since the `NameID`'s value case could change. The existing user
would not be found and the login would try to create a new one, but fail
since the uniqueness check of IdP ID and external userID is not case
insensitive.
# How the Problems Are Solved
Search case insensitive for external useriDs.
# Additional Changes
None
# Additional Context
- closes#10457, #10387
- backport to v3.x
# Which Problems Are Solved
When `Permission Check V2` is enabled, calls to the `ListPasskeys` and
`ListAuthenticationFactors` APIs fail with the following error:
```
ERROR: missing FROM-clause entry for table "users14"
```
# How the Problems Are Solved
By using the right UserID column
(`projections.user_auth_methods5.user_id`) in the permission clause in
the `userAuthMethod` query
# Additional Changes
N/A
# Additional Context
- Closes#10386
Querying an organization by id allowed to trigger the org projection.
This could lead to performance impacts if the projection gets triggered
too often.
Instead of executing the trigger the organization by id query is now
always executed on the eventstore and reduces all event types required
of the organization requested.
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
The current handling of event subscriptions for actions is bad, esp. on
instances with a lot of events
(https://github.com/zitadel/zitadel/issues/9832#issuecomment-2866236414).
This led to severe problems on zitadel.cloud for such instances.
# How the Problems Are Solved
As a workaround until the handling can be improved, we introduce an
option for projections to be disabled completely for specific instances:
`SkipInstanceIDs`
# Additional Changes
None
# Additional Context
- relates to https://github.com/zitadel/zitadel/issues/9832
# Which Problems Are Solved
The deletion of expired sessions does not go through even though a
success response is returned to the user. These expired and supposedly
deleted (to the user) sessions are then returned when the `ListSessions`
API is called.
This PR fixes this issue by:
1. Allowing deletion of expired sessions
2. Providing an `expiration_date` filter in `ListSession` API to filter
sessions by expiration date
# How the Problems Are Solved
1. Remove expired session check during deletion
2. Add an `expiration_date` filter to the `ListSession` API
# Additional Changes
N/A
# Additional Context
- Closes#10045
---------
Co-authored-by: Marco A. <marco@zitadel.com>
# Which Problems Are Solved
Currently the username uniqueness is on instance level, we want to
achieve a way to set it at organization level.
# How the Problems Are Solved
Addition of endpoints and a resource on organization level, where this
setting can be managed. If nothing it set, the uniqueness is expected to
be at instance level, where only users with instance permissions should
be able to change this setting.
# Additional Changes
None
# Additional Context
Includes #10086Closes#9964
---------
Co-authored-by: Marco A. <marco@zitadel.com>
# Which Problems Are Solved
[Permissions v2](https://github.com/zitadel/zitadel/issues/9972) is not
possible in the current implementation.
# How the Problems Are Solved
We remove Permissions v2 from project grants related API calls, to
alleviate this problems.
Resulting in some removals of testing, implementations and performance
impact
# Additional Changes
None
# Additional Context
None
## Which problems are solved
The execution of statements of projections did not have the context
present.
## How the problems were solved
Pass the context to the execute function
## Additional info
This change is required to use the repositories of the relational tables
in projections.
- Turkish language support is added.
- Updated other language files to add Turkish selection.
# Which Problems Are Solved
- Zitadel was not supporting Turkish language. Now supporting.
# How the Problems Are Solved
- Turkish language files are added and updated other language files in
below paths to add Turkish support;
- /console/src/assets/i18n/
- /internal/api/ui/login/static/i18n
- /internal/notification/static/i18n
- /internal/static/i18n
# Additional Changes
- Made changes below files for codes/docs changes;
- /console/src/app/utils/language.ts
- /console/src/app/app.module.ts
- /docs/docs/guides/manage/customize/texts.md
- /internal/api/ui/login/static/templates/external_not_found_option.html
- /internal/query/v2-default.json
- /login/apps/login/src/lib/i18n.ts
---------
Co-authored-by: Marco A. <marco@zitadel.com>
# Which Problems Are Solved
When an organization domain is verified, e.g. also when creating a new
organization (incl. generated domain), existing usernames are checked if
the domain has been claimed.
The query was not optimized for instances with many users and
organizations.
# How the Problems Are Solved
- Replace the query, which was searching over the users projection with
(computed loginnames) with a dedicated query checking the loginnames
projection directly.
- All occurrences have been updated to use the new query.
# Additional Changes
None
# Additional Context
- reported through support
- requires backport to v3.x
# Which Problems Are Solved
The commands for the resource based v2beta AuthorizationService API are
added.
Authorizations, previously knows as user grants, give a user in a
specific organization and project context roles.
The project can be owned or granted.
The given roles can be used to restrict access within the projects
applications.
The commands for the resource based v2beta InteralPermissionService API
are added.
Administrators, previously knows as memberships, give a user in a
specific organization and project context roles.
The project can be owned or granted.
The give roles give the user permissions to manage different resources
in Zitadel.
API definitions from https://github.com/zitadel/zitadel/issues/9165 are
implemented.
Contains endpoints for user metadata.
# How the Problems Are Solved
### New Methods
- CreateAuthorization
- UpdateAuthorization
- DeleteAuthorization
- ActivateAuthorization
- DeactivateAuthorization
- ListAuthorizations
- CreateAdministrator
- UpdateAdministrator
- DeleteAdministrator
- ListAdministrators
- SetUserMetadata to set metadata on a user
- DeleteUserMetadata to delete metadata on a user
- ListUserMetadata to query for metadata of a user
## Deprecated Methods
### v1.ManagementService
- GetUserGrantByID
- ListUserGrants
- AddUserGrant
- UpdateUserGrant
- DeactivateUserGrant
- ReactivateUserGrant
- RemoveUserGrant
- BulkRemoveUserGrant
### v1.AuthService
- ListMyUserGrants
- ListMyProjectPermissions
# Additional Changes
- Permission checks for metadata functionality on query and command side
- correct existence checks for resources, for example you can only be an
administrator on an existing project
- combined all member tables to singular query for the administrators
- add permission checks for command an query side functionality
- combined functions on command side where necessary for easier
maintainability
# Additional Context
Closes#9165
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
This PR *partially* addresses #9450 . Specifically, it implements the
resource based API for app keys.
This PR, together with https://github.com/zitadel/zitadel/pull/10077
completes #9450 .
# How the Problems Are Solved
- Implementation of the following endpoints: `CreateApplicationKey`,
`DeleteApplicationKey`, `GetApplicationKey`, `ListApplicationKeys`
- `ListApplicationKeys` can filter by project, app or organization ID.
Sorting is also possible according to some criteria.
- All endpoints use permissions V2
# TODO
- [x] Deprecate old endpoints
# Additional Context
Closes#9450
# Which Problems Are Solved
Remove the feature flag that allowed triggers in introspection. This
option was a fallback in case introspection would not function properly
without triggers. The API documentation asked for anyone using this flag
to raise an issue. No such issue was received, hence we concluded it is
safe to remove it.
# How the Problems Are Solved
- Remove flags from the system and instance level feature APIs.
- Remove trigger functions that are no longer used
- Adjust tests that used the flag.
# Additional Changes
- none
# Additional Context
- Closes#10026
- Flag was introduced in #7356
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
This PR *partially* addresses #9450 . Specifically, it implements the
resource based API for the apps. APIs for app keys ARE not part of this
PR.
# How the Problems Are Solved
- `CreateApplication`, `PatchApplication` (update) and
`RegenerateClientSecret` endpoints are now unique for all app types:
API, SAML and OIDC apps.
- All new endpoints have integration tests
- All new endpoints are using permission checks V2
# Additional Changes
- The `ListApplications` endpoint allows to do sorting (see protobuf for
details) and filtering by app type (see protobuf).
- SAML and OIDC update endpoint can now receive requests for partial
updates
# Additional Context
Partially addresses #9450
# Which Problems Are Solved
Stabilize the usage of webkeys.
# How the Problems Are Solved
- Remove all legacy signing key code from the OIDC API
- Remove the webkey feature flag from proto
- Remove the webkey feature flag from console
- Cleanup documentation
# Additional Changes
- Resolved some canonical header linter errors in OIDC
- Use the constant for `projections.lock` in the saml package.
# Additional Context
- Closes#10029
- After #10105
- After #10061
