# Which Problems Are Solved
Currently the username uniqueness is on instance level, we want to
achieve a way to set it at organization level.
# How the Problems Are Solved
Addition of endpoints and a resource on organization level, where this
setting can be managed. If nothing it set, the uniqueness is expected to
be at instance level, where only users with instance permissions should
be able to change this setting.
# Additional Changes
None
# Additional Context
Includes #10086Closes#9964
---------
Co-authored-by: Marco A. <marco@zitadel.com>
# Which Problems Are Solved
As an outcome of [this
issue](https://github.com/zitadel/zitadel/issues/9599) we want to
implement relational tables in Zitadel. For that we use new tables as a
successor of the current tables used by Zitadel in `projections`, `auth`
and `admin` schemas. The new logic is based on [this
proposal](https://github.com/zitadel/zitadel/pull/9870). This issue does
not contain the switch from CQRS to the new tables. This is change will
be implemented in a later stage.
We focus on the most critical tables which is user authentication.
We need a table to manage organizations.
### organization fields
The following fields must be managed in this table:
- `id`
- `instance_id`
- `name`
- `state` enum (active, inactive)
- `created_at`
- `updated_at`
- `deleted_at`
DISCUSS: should we add a `primary_domain` to this table so that we do
not have to join on domains to return a simple org?
We must ensure the unique constraints for this table matches the current
commands.
### organization repository
The repository must provide the following functions:
Manipulations:
- create
- `instance_id`
- `name`
- update
- `name`
- delete
Queries:
- get returns single organization matching the criteria and pagination,
should return error if multiple were found
- list returns list of organizations matching the criteria, pagination
Criteria are the following:
- by id
- by name
pagination:
- by created_at
- by updated_at
- by name
### organization events
The following events must be applied on the table using a projection
(`internal/query/projection`)
- `org.added` results in create
- `org.changed` sets the `name` field
- `org.deactivated` sets the `state` field
- `org.reactivated` sets the `state` field
- `org.removed` sets the `deleted_at` field
- if answer is yes to discussion: `org.domain.primary.set` sets the
`primary_domain` field
- `instance.removed` sets the the `deleted_at` field if not already set
### acceptance criteria
- [x] migration is implemented and gets executed
- [x] domain interfaces are implemented and documented for service layer
- [x] repository is implemented and implements domain interface
- [x] testing
- [x] the repository methods
- [x] events get reduced correctly
- [x] unique constraints
# Additional Context
Replace this example with links to related issues, discussions, discord
threads, or other sources with more context.
Use the Closing #issue syntax for issues that are resolved with this PR.
- Closes #https://github.com/zitadel/zitadel/issues/9936
---------
Co-authored-by: adlerhurst <27845747+adlerhurst@users.noreply.github.com>
# Which Problems Are Solved
The commands for the resource based v2beta AuthorizationService API are
added.
Authorizations, previously knows as user grants, give a user in a
specific organization and project context roles.
The project can be owned or granted.
The given roles can be used to restrict access within the projects
applications.
The commands for the resource based v2beta InteralPermissionService API
are added.
Administrators, previously knows as memberships, give a user in a
specific organization and project context roles.
The project can be owned or granted.
The give roles give the user permissions to manage different resources
in Zitadel.
API definitions from https://github.com/zitadel/zitadel/issues/9165 are
implemented.
Contains endpoints for user metadata.
# How the Problems Are Solved
### New Methods
- CreateAuthorization
- UpdateAuthorization
- DeleteAuthorization
- ActivateAuthorization
- DeactivateAuthorization
- ListAuthorizations
- CreateAdministrator
- UpdateAdministrator
- DeleteAdministrator
- ListAdministrators
- SetUserMetadata to set metadata on a user
- DeleteUserMetadata to delete metadata on a user
- ListUserMetadata to query for metadata of a user
## Deprecated Methods
### v1.ManagementService
- GetUserGrantByID
- ListUserGrants
- AddUserGrant
- UpdateUserGrant
- DeactivateUserGrant
- ReactivateUserGrant
- RemoveUserGrant
- BulkRemoveUserGrant
### v1.AuthService
- ListMyUserGrants
- ListMyProjectPermissions
# Additional Changes
- Permission checks for metadata functionality on query and command side
- correct existence checks for resources, for example you can only be an
administrator on an existing project
- combined all member tables to singular query for the administrators
- add permission checks for command an query side functionality
- combined functions on command side where necessary for easier
maintainability
# Additional Context
Closes#9165
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
# Which Problems Are Solved
This PR implements https://github.com/zitadel/zitadel/issues/9850
# How the Problems Are Solved
- New protobuf definition
- Implementation of retrieval of system translations
- Implementation of retrieval and persistence of organization and
instance level translations
# Additional Context
- Closes#9850
# TODO
- [x] Integration tests for Get and Set hosted login translation
endpoints
- [x] DB migration test
- [x] Command function tests
- [x] Command util functions tests
- [x] Query function test
- [x] Query util functions tests
# Which Problems Are Solved
Currently if a user signs in using an IdP, once they sign out of
Zitadel, the corresponding IdP session is not terminated. This can be
the desired behavior. In some cases, e.g. when using a shared computer
it results in a potential security risk, since a follower user might be
able to sign in as the previous using the still open IdP session.
# How the Problems Are Solved
- Admins can enabled a federated logout option on SAML IdPs through the
Admin and Management APIs.
- During the termination of a login V1 session using OIDC end_session
endpoint, Zitadel will check if an IdP was used to authenticate that
session.
- In case there was a SAML IdP used with Federated Logout enabled, it
will intercept the logout process, store the information into the shared
cache and redirect to the federated logout endpoint in the V1 login.
- The V1 login federated logout endpoint checks every request on an
existing cache entry. On success it will create a SAML logout request
for the used IdP and either redirect or POST to the configured SLO
endpoint. The cache entry is updated with a `redirected` state.
- A SLO endpoint is added to the `/idp` handlers, which will handle the
SAML logout responses. At the moment it will check again for an existing
federated logout entry (with state `redirected`) in the cache. On
success, the user is redirected to the initially provided
`post_logout_redirect_uri` from the end_session request.
# Additional Changes
None
# Additional Context
- This PR merges the https://github.com/zitadel/zitadel/pull/9841 and
https://github.com/zitadel/zitadel/pull/9854 to main, additionally
updating the docs on Entra ID SAML.
- closes#9228
- backport to 3.x
---------
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Zach Hirschtritt <zachary.hirschtritt@klaviyo.com>
# Which Problems Are Solved
Some OAuth2 and OIDC providers require the use of PKCE for all their
clients. While ZITADEL already recommended the same for its clients, it
did not yet support the option on the IdP configuration.
# How the Problems Are Solved
- A new boolean `use_pkce` is added to the add/update generic OAuth/OIDC
endpoints.
- A new checkbox is added to the generic OAuth and OIDC provider
templates.
- The `rp.WithPKCE` option is added to the provider if the use of PKCE
has been set.
- The `rp.WithCodeChallenge` and `rp.WithCodeVerifier` options are added
to the OIDC/Auth BeginAuth and CodeExchange function.
- Store verifier or any other persistent argument in the intent or auth
request.
- Create corresponding session object before creating the intent, to be
able to store the information.
- (refactored session structs to use a constructor for unified creation
and better overview of actual usage)
Here's a screenshot showing the URI including the PKCE params:
# Additional Changes
None.
# Additional Context
- Closes#6449
- This PR replaces the existing PR (#8228) of @doncicuto. The base he
did was cherry picked. Thank you very much for that!
---------
Co-authored-by: Miguel Cabrerizo <doncicuto@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
Currently ZITADEL defines organization and instance member roles and
permissions in defaults.yaml. The permission check is done on API call
level. For example: "is this user allowed to make this call on this
org". This makes sense on the V1 API where the API is permission-level
shaped. For example, a search for users always happens in the context of
the organization. (Either the organization the calling user belongs to,
or through member ship and the x-zitadel-orgid header.
However, for resource based APIs we must be able to resolve permissions
by object. For example, an IAM_OWNER listing users should be able to get
all users in an instance based on the query filters. Alternatively a
user may have user.read permissions on one or more orgs. They should be
able to read just those users.
# How the Problems Are Solved
## Role permission mapping
The role permission mappings defined from `defaults.yaml` or local
config override are synchronized to the database on every run of
`zitadel setup`:
- A single query per **aggregate** builds a list of `add` and `remove`
actions needed to reach the desired state or role permission mappings
from the config.
- The required events based on the actions are pushed to the event
store.
- Events define search fields so that permission checking can use the
indices and is strongly consistent for both query and command sides.
The migration is split in the following aggregates:
- System aggregate for for roles prefixed with `SYSTEM`
- Each instance for roles not prefixed with `SYSTEM`. This is in
anticipation of instance level management over the API.
## Membership
Current instance / org / project membership events now have field table
definitions. Like the role permissions this ensures strong consistency
while still being able to use the indices of the fields table. A
migration is provided to fill the membership fields.
## Permission check
I aimed keeping the mental overhead to the developer to a minimal. The
provided implementation only provides a permission check for list
queries for org level resources, for example users. In the `query`
package there is a simple helper function `wherePermittedOrgs` which
makes sure the underlying database function is called as part of the
`SELECT` query and the permitted organizations are part of the `WHERE`
clause. This makes sure results from non-permitted organizations are
omitted. Under the hood:
- A Pg/PlSQL function searches for a list of organization IDs the passed
user has the passed permission.
- When the user has the permission on instance level, it returns early
with all organizations.
- The functions uses a number of views. The views help mapping the
fields entries into relational data and simplify the code use for the
function. The views provide some pre-filters which allow proper index
usage once the final `WHERE` clauses are set by the function.
# Additional Changes
# Additional Context
Closes#9032
Closes https://github.com/zitadel/zitadel/issues/9014https://github.com/zitadel/zitadel/issues/9188 defines follow-ups for
the new permission framework based on this concept.
# Which Problems Are Solved
When an org is removed, the corresponding fields are not deleted. This
creates issues, such as recreating a new org with the same verified
domain.
# How the Problems Are Solved
Remove the search fields by the org aggregate, instead of just setting
the removed state.
# Additional Changes
- Cleanup migration script that removed current stale fields.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8943
- Related to https://github.com/zitadel/zitadel/pull/8790
---------
Co-authored-by: Silvan <silvan.reusser@gmail.com>
# Which Problems Are Solved
Improve the performance of human imports by optimizing the query that
finds domains claimed by other organizations.
# How the Problems Are Solved
Use the fields search table introduced in
https://github.com/zitadel/zitadel/pull/8191 by storing each
organization domain as Object ID and the verified status as field value.
# Additional Changes
- Feature flag for this optimization
# Additional Context
- Performance improvements for import are evaluated and acted upon
internally at the moment
---------
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
# Which Problems Are Solved
To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.
The table allows to store one primitive value (numeric, text) per row.
The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.
# How the Problems Are Solved
This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.
# Open Tasks
- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table
# Additional Changes
The `Search`-method is currently used by `ProjectGrant`-command side.
# Additional Context
- Closes https://github.com/zitadel/zitadel/issues/8094
# Which Problems Are Solved
ZITADEL currently always uses
`urn:oasis:names:tc:SAML:2.0:nameid-format:persistent` in SAML requests,
relying on the IdP to respect that flag and always return a peristent
nameid in order to be able to map the external user with an existing
user (idp link) in ZITADEL.
In case the IdP however returns a
`urn:oasis:names:tc:SAML:2.0:nameid-format:transient` (transient)
nameid, the attribute will differ between each request and it will not
be possible to match existing users.
# How the Problems Are Solved
This PR adds the following two options on SAML IdP:
- **nameIDFormat**: allows to set the nameid-format used in the SAML
Request
- **transientMappingAttributeName**: allows to set an attribute name,
which will be used instead of the nameid itself in case the returned
nameid-format is transient
# Additional Changes
To reduce impact on current installations, the `idp_templates6_saml`
table is altered with the two added columns by a setup job. New
installations will automatically get the table with the two columns
directly.
All idp unit tests are updated to use `expectEventstore` instead of the
deprecated `eventstoreExpect`.
# Additional Context
Closes#7483Closes#7743
---------
Co-authored-by: peintnermax <max@caos.ch>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Even though this is a feature it's released as fix so that we can back port to earlier revisions.
As reported by multiple users startup of ZITADEL after leaded to downtime and worst case rollbacks to the previously deployed version.
The problem starts rising when there are too many events to process after the start of ZITADEL. The root cause are changes on projections (database tables) which must be recomputed. This PR solves this problem by adding a new step to the setup phase which prefills the projections. The step can be enabled by adding the `--init-projections`-flag to `setup`, `start-from-init` and `start-from-setup`. Setting this flag results in potentially longer duration of the setup phase but reduces the risk of the problems mentioned in the paragraph above.
* cleanup todo
* pass id token details to oidc
* feat(oidc): id token for device authorization
This changes updates to the newest oidc version,
so the Device Authorization grant can return ID tokens when
the scope `openid` is set.
There is also some refactoring done, so that the eventstore can be
queried directly when polling for state.
The projection is cleaned up to a minimum with only data required for the login UI.
* try to be explicit wit hthe timezone to fix github
* pin oidc v3.8.0
* remove TBD entry
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* add management in console
* pass enabledTheme
* render login ui based on enabled theme
* add in branding / settings service and name consistently
* update console to latest proto state
* fix console linting
* fix linting
* cleanup
* add translations
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
This PR adds an option to the LoginPolicy to "Force MFA for local users", so that users authenticated through an IDP must not configure (and verify) an MFA.
* device auth: implement the write events
* add grant type device code
* fix(init): check if default value implements stringer
---------
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
Adds a id_attribute to the GenericOAuthProvider, which is used to map the external User. Further mapping can be done in actions by using the `rawInfo` of the new `ctx.v1.providerInfo` field.
Add management functionality for LDAP idps with templates and the basic functionality for the LDAP provider, which can then be used with a separate login page in the future.
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* docs: update cockroachdb version to 22.2
* feat(adminAPI): ListEventTypes returns the list of event types ZITADEL implements
* feat(adminAPI): ListAggregateTypes returns the list of aggregate types ZITADEL implements
* feat(adminAPI): ListEvents allows `IAM_OWNERS` to search for events
* feat(command): remove org
* refactor: imports, unused code, error handling
* reduce org removed in action
* add org deletion to projections
* add org removal to projections
* add org removal to projections
* org removed projection
* lint import
* projections
* fix: table names in tests
* fix: table names in tests
* logging
* add org state
* fix(domain): add Owner removed to object details
* feat(ListQuery): add with owner removed
* fix(org-delete): add bool to functions to select with owner removed
* fix(org-delete): add bools to user grants with events to determine if dependencies lost owner
* fix(org-delete): add unit tests for owner removed and org removed events
* fix(org-delete): add handling of org remove for grants and members
* fix(org-delete): correction of unit tests for owner removed
* fix(org-delete): update projections, unit tests and get functions
* fix(org-delete): add change date to authnkeys and owner removed to org metadata
* fix(org-delete): include owner removed for login names
* fix(org-delete): some column fixes in projections and build for queries with owner removed
* indexes
* fix(org-delete): include review changes
* fix(org-delete): change user projection name after merge
* fix(org-delete): include review changes for project grant where no project owner is necessary
* fix(org-delete): include auth and adminapi tables with owner removed information
* fix(org-delete): cleanup username and orgdomain uniqueconstraints when org is removed
* fix(org-delete): add permissions for org.remove
* remove unnecessary unique constraints
* fix column order in primary keys
* fix(org-delete): include review changes
* fix(org-delete): add owner removed indexes and chang setup step to create tables
* fix(org-delete): move PK order of instance_id and change added user_grant from review
* fix(org-delete): no params for prepareUserQuery
* change to step 6
* merge main
* fix(org-delete): OldUserName rename to private
* fix linting
* cleanup
* fix: remove org test
* create prerelease
* chore: delete org-delete as prerelease
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
