This PR extends the user schema service (V3 API) with the possibility to ListUserSchemas and GetUserSchemaByID.
The previously started guide is extended to demonstrate how to retrieve the schema(s) and notes the generated revision property.
Add organizationID as query for ListUsers and clean up the deprecated Organisation attributes in other queries.
This PR removes the following fields from API requests (user service v2):
organisation from AddHumanUser (deprecated some time ago, organization still exists)
organization from GetUserByID
* add token exchange feature flag
* allow setting reason and actor to access tokens
* impersonation
* set token types and scopes in response
* upgrade oidc to working draft state
* fix tests
* audience and scope validation
* id toke and jwt as input
* return id tokens
* add grant type token exchange to app config
* add integration tests
* check and deny actors in api calls
* fix instance setting tests by triggering projection on write and cleanup
* insert sleep statements again
* solve linting issues
* add translations
* pin oidc v3.15.0
* resolve comments, add event translation
* fix refreshtoken test
* use ValidateAuthReqScopes from oidc
* apparently the linter can't make up its mind
* persist actor thru refresh tokens and check in tests
* remove unneeded triggers
* feat: add projections and query side to executions and targets
* feat: add list and get endpoints for targets
* feat: add integration tests for query endpoints target and execution
* fix: linting
* fix: linting
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: review changes, renames and corrections
* fix: remove position from list details
This PR adds the functionality to manage user schemas through the new user schema service.
It includes the possibility to create a basic JSON schema and also provides a way on defining permissions (read, write) for owner and self context with an annotation.
Further annotations for OIDC claims and SAML attribute mappings will follow.
A guide on how to create a schema and assign permissions has been started. It will be extended though out the process of implementing the schema and users based on those.
Note:
This feature is in an early stage and therefore not enabled by default. To test it out, please enable the UserSchema feature flag on your instance / system though the feature service.
Add additional permission integration tests to the user v2 query endpoints including some fixes to correctly check the permissions after the data is known which you want to query.
* feat: improve instance not found error
* unit tests
* check if is templatable
* lint
* assert
* compile tests
* remove error templates
* link to instance not found page
* fmt
* cleanup
* lint
* partial work done
* test IAM membership roles
* org membership tests
* console :(, translations and docs
* fix integration test
* fix tests
* add EnableImpersonation to security policy API
* fix integration test timestamp checking
* add security policy tests and fix projections
* add impersonation setting in console
* add security settings to the settings v2 API
* fix typo
* move impersonation to instance
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(api): feature API proto definitions
* update proto based on discussion with @livio-a
* cleanup old feature flag stuff
* authz instance queries
* align defaults
* projection definitions
* define commands and event reducers
* implement system and instance setter APIs
* api getter implementation
* unit test repository package
* command unit tests
* unit test Get queries
* grpc converter unit tests
* migrate the V1 features
* migrate oidc to dynamic features
* projection unit test
* fix instance by host
* fix instance by id data type in sql
* fix linting errors
* add system projection test
* fix behavior inversion
* resolve proto file comments
* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event
* use write models and conditional set events
* system features integration tests
* instance features integration tests
* error on empty request
* documentation entry
* typo in feature.proto
* fix start unit tests
* solve linting error on key case switch
* remove system defaults after discussion with @eliobischof
* fix system feature projection
* resolve comments in defaults.yaml
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat: add events for execution
* feat: add events for execution and command side
* feat: add events for execution and command side
* feat: add api endpoints for set and delete executions with integration tests
* feat: add integration and unit tests and more existence checks
* feat: add integration and unit tests and more existence checks
* feat: unit tests for includes in executions
* feat: integration tests for includes in executions
* fix: linting
* fix: update internal/api/api.go
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: update internal/command/command.go
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: apply suggestions from code review
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: change api return
* fix: change aggregateID with prefix of execution type and add to documentation
* fix: change body in proto for documentation and correct linting
* fix: changed existing check to single query in separate writemodel
* fix: linter changes and list endpoints for conditions in executions
* fix: remove writemodel query on exeuction set as state before is irrelevant
* fix: testing for exists write models and correction
* fix: translations for errors and event types
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: add implementation for resend of email and phone code
* fix: apply suggestions from code review
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: review changes to remove resourceowner as parameters
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: add granted org info to user grants query response
* fix: show user info, tests and add columns to user grant
* fix: add check for org membership
* fix: typo in find logic
---------
Co-authored-by: Max Peintner <max@caos.ch>
* feat: add query endpoints for user v2 api
* fix: correct integration tests
* fix: correct linting
* fix: correct linting
* fix: comment out permission check on user get and list
* fix: permission check on user v2 query
* fix: merge back origin/main
* fix: add search query in user emails
* fix: reset count for SearchUser if users are removed due to permissions
* fix: reset count for SearchUser if users are removed due to permissions
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
* fix: change logic for usergrants projection with no selects
* fix: change logic for usergrants projection with one select
* fix: move resource owner select to single function
* fix: move resource owner select to single function
* fix: changes after merge
* fix: changes after merge
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* feat: user service v2 remove user
* feat: user service v2 add user human
* feat: user service v2 change user human
* feat: user service v2 change user human unit tests
* feat: user service v2 reactivate, deactivate, lock, unlock user
* feat: user service v2 integration tests
* fix: merge back origin/main
* lint: linter corrections
* fix: move permission check for isVerfied and password change
* fix: add deprecated notices and other review comments
* fix: consistent naming in proto
* fix: errors package renaming
* fix: remove / delete user renaming in integration test
* fix: machine user status changes through user v2 api
* fix: linting changes
* fix: linting changes
* fix: changes from review
* fix: changes from review
* fix: changes from review
* fix: changes from review
* fix: changes from review
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
* start user by id
* ignore debug bin
* use new user by id
* new sql
* fix(sql): replace STRING with text for psql compatabilit
* some changes
* fix: correct user queries
* fix tests
* unify sql statements
* use specific get user methods
* search login name case insensitive
* refactor: optimise user statements
* add index
* fix queries
* fix: correct domain segregation
* return all login names
* fix queries
* improve readability
* query should be correct now
* cleanup statements
* fix username / loginname handling
* fix: psql doesn't support create view if not exists
* fix: create pre-release
* ignore release comments
* add lower fields
* fix: always to lower
* update to latest projection
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(console): add time range events filter
* deprecate creation_date, use oneof filter
* use range or from
* implement api
* fix timestamp format
* translate
* styles
* lint
* integration tests
* fix until date
* rearrange sorting control
* sort creation date
* fix events e2e test
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* lint
* lint
* don't use utc call time
---------
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* feat: return 404 or 409 if org reg disallowed
* fix: system limit permissions
* feat: add iam limits api
* feat: disallow public org registrations on default instance
* add integration test
* test: integration
* fix test
* docs: describe public org registrations
* avoid updating docs deps
* fix system limits integration test
* silence integration tests
* fix linting
* ignore strange linter complaints
* review
* improve reset properties naming
* redefine the api
* use restrictions aggregate
* test query
* simplify and test projection
* test commands
* fix unit tests
* move integration test
* support restrictions on default instance
* also test GetRestrictions
* self review
* lint
* abstract away resource owner
* fix tests
* configure supported languages
* fix allowed languages
* fix tests
* default lang must not be restricted
* preferred language must be allowed
* change preferred languages
* check languages everywhere
* lint
* test command side
* lint
* add integration test
* add integration test
* restrict supported ui locales
* lint
* lint
* cleanup
* lint
* allow undefined preferred language
* fix integration tests
* update main
* fix env var
* ignore linter
* ignore linter
* improve integration test config
* reduce cognitive complexity
* compile
* fix(console): switch back to saved language
* feat(API): get allowed languages
* fix(console): only make allowed languages selectable
* warn when editing not allowed languages
* check for duplicates
* remove useless restriction checks
* review
* revert restriction renaming
* fix language restrictions
* lint
* generate
* allow custom texts for supported langs for now
* fix tests
* cleanup
* cleanup
* cleanup
* lint
* unsupported preferred lang is allowed
* fix integration test
* allow unsupported preferred languages
* lint
* load languages for tests
* cleanup
* lint
* cleanup
* get allowed only on admin
* cleanup
* reduce flakiness on very limited postgres
* simplify langSvc
* refactor according to suggestions in pr
* lint
* set first allowed language as default
* selectionchange for language in msg texts
* initialize login texts
* init message texts
* lint
---------
Co-authored-by: peintnermax <max@caos.ch>
* feat: return 404 or 409 if org reg disallowed
* fix: system limit permissions
* feat: add iam limits api
* feat: disallow public org registrations on default instance
* add integration test
* test: integration
* fix test
* docs: describe public org registrations
* avoid updating docs deps
* fix system limits integration test
* silence integration tests
* fix linting
* ignore strange linter complaints
* review
* improve reset properties naming
* redefine the api
* use restrictions aggregate
* test query
* simplify and test projection
* test commands
* fix unit tests
* move integration test
* support restrictions on default instance
* also test GetRestrictions
* self review
* lint
* abstract away resource owner
* fix tests
* configure supported languages
* fix allowed languages
* fix tests
* default lang must not be restricted
* preferred language must be allowed
* change preferred languages
* check languages everywhere
* lint
* test command side
* lint
* add integration test
* add integration test
* restrict supported ui locales
* lint
* lint
* cleanup
* lint
* allow undefined preferred language
* fix integration tests
* update main
* fix env var
* ignore linter
* ignore linter
* improve integration test config
* reduce cognitive complexity
* compile
* check for duplicates
* remove useless restriction checks
* review
* revert restriction renaming
* fix language restrictions
* lint
* generate
* allow custom texts for supported langs for now
* fix tests
* cleanup
* cleanup
* cleanup
* lint
* unsupported preferred lang is allowed
* fix integration test
* finish reverting to old property name
* finish reverting to old property name
* load languages
* refactor(i18n): centralize translators and fs
* lint
* amplify no validations on preferred languages
* fix integration test
* lint
* fix resetting allowed languages
* test unchanged restrictions
* fix: add https status to activity log
* create prerelease
* create RC
* pass info from gateway to grpc server
* fix: update releaserc to create RC version
* cleanup
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
This change adds a core_generate_all make target.
It installs the required tools and runs generate on the complete project.
`golang/mock` is no longer maintained and a fork is available
from the Uber folks. So the latter is used as tool.
All the mock files have been regenerated and are part of the PR.
The obsolete `tools` directory has been removed,
as all the tools are now part of specific make targets.
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* fix: correct method and path for session api activity
* fix: correct method and path for session api activity
* fix: correct function name for activity trigger
* get key by id and cache them
* userinfo from events for v2 tokens
* improve keyset caching
* concurrent token and client checks
* client and project in single query
* logging and otel
* drop owner_removed column on apps and authN tables
* userinfo and project roles in go routines
* get oidc user info from projections and add actions
* add avatar URL
* some cleanup
* pull oidc work branch
* remove storage from server
* add config flag for experimental introspection
* legacy introspection flag
* drop owner_removed column on user projections
* drop owner_removed column on useer_metadata
* query userinfo unit test
* query introspection client test
* add user_grants to the userinfo query
* handle PAT scopes
* bring triggers back
* test instance keys query
* add userinfo unit tests
* unit test keys
* go mod tidy
* solve some bugs
* fix missing preferred login name
* do not run triggers in go routines, they seem to deadlock
* initialize the trigger handlers late with a sync.OnceValue
* Revert "do not run triggers in go routines, they seem to deadlock"
This reverts commit 2a03da2127.
* add missing translations
* chore: update go version for linting
* pin oidc version
* parse a global time location for query test
* fix linter complains
* upgrade go lint
* fix more linting issues
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
* fix: add resource owner of user and change the one of session to instance
* use user resource owner from session projection
* fix session permission check
* integration tests and fixes
* update api docs
* chore(deps): upgrade all go modules
This change upgrades all go.mod dependecies. As well as Makefile tools.
There where some imports that still used the old and deprecated
`github.com/golang/protobuf/ptypes` package.
These have been moved to the equivelant
`google.golang.org/protobuf/types/known` package.
The `internal/proto` package is removed as was only used once.
With a simple refactor in the Validator it became completely obsolete.
* fix validate unit test
* cleanup merge
* update otel
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(user/v1): support composite queries
* fix: added proper error handling for NotQuery
* Added error when there are too many levels of nesting
* Add localization keys for english
* Update internal/api/grpc/user/query.go
* feat: extend session search service (#6029)
add two more searching criteria - human user id and session creation date
optional sorting by the session creation date
* fix: use correct column identifier
* fix: implement Col()
* chore: fix unit tests
* chore: fix linter warnings
---------
Co-authored-by: Fabi <fabienne@zitadel.com>
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* feat: enable only specific themes in label policy
* add management in console
* pass enabledTheme
* render login ui based on enabled theme
* add in branding / settings service and name consistently
* update console to latest proto state
* fix console linting
* fix linting
* cleanup
* add translations
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* feat(oidc): use the new oidc server interface
* rename from provider to server
* pin logging and oidc packages
* use oidc introspection fix branch
* add overloaded methods with tracing
* cleanup unused code
* include latest oidc fixes
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* define roles and permissions
* support system user memberships
* don't limit system users
* cleanup permissions
* restrict memberships to aggregates
* default to SYSTEM_OWNER
* update unit tests
* test: system user token test (#6778)
* update unit tests
* refactor: make authz testable
* move session constants
* cleanup
* comment
* comment
* decode member type string to enum (#6780)
* decode member type string to enum
* handle all membership types
* decode enums where necessary
* decode member type in steps config
* update system api docs
* add technical advisory
* tweak docs a bit
* comment in comment
* lint
* extract token from Bearer header prefix
* review changes
* fix tests
* fix: add fix for activityhandler
* add isSystemUser
* remove IsSystemUser from activity info
* fix: add fix for activityhandler
---------
Co-authored-by: Stefan Benz <stefan@caos.ch>
* feat: add activity logs on user actions with authentication, resourceAPI and sessionAPI
* feat: add activity logs on user actions with authentication, resourceAPI and sessionAPI
* feat: add activity logs on user actions with authentication, resourceAPI and sessionAPI
* feat: add activity logs on user actions with authentication, resourceAPI and sessionAPI
* feat: add activity logs on user actions with authentication, resourceAPI and sessionAPI
* fix: add unit tests to info package for context changes
* fix: add activity_interceptor.go suggestion
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: refactoring and fixes through PR review
* fix: add auth service to lists of resourceAPIs
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Fabi <fabienne@zitadel.com>
* feat: support list milestones api
* show milestones in onboarding view
* add authenticated milestone
* add icon to login milestone
* update main
* lint
* fix import
* fix import
* lint
* reuse proto milestone type mapping
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
* start feature flags
* base feature events on domain const
* setup default features
* allow setting feature in system api
* allow setting feature in admin api
* set settings in login based on feature
* fix rebasing
* unit tests
* i18n
* update policy after domain discovery
* some changes from review
* check feature and value type
* check feature and value type
* feat: delete organizations
* feat: tests and delete all that depends on org
* fix: grpc delete description
* fix: get back reduce OrgRemovedEvent
* fix: add @muhlemmer review suggestions
* fix: new e2e for add/delete org
* feat: add reply-to header to smtp messages
* fix: grpc reply_to_address min 0 and js var name
* fix: add missing translations
* fix merge and linting
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat: add otp (sms and email) checks in session api
* implement sending
* fix tests
* add tests
* add integration tests
* fix merge main and add tests
* put default OTP Email url into config
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
* fix: restrict AllowRegistration check to local registration
* add comment
* add additional tests
* hide registration fields if no registration allowed
* fix: always allow linking and creation of external idps on users in userV2 and admin import
* chore: exclude console dist and node_module folders from cache
* chore: include node_module folders into cache again
* linting
* fix(api): rename first and last name to given and family name, intent to idp_intent, remove _ actions
* fix merge
* fully rename intent to idp intent in api
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix: handling of ldap login through separate endpoint
* fix: handling of ldap login through separate endpoint
* fix: handling of ldap login through separate endpoint
* fix: successful intent for ldap
* fix: successful intent for ldap
* fix: successful intent for ldap
* fix: add changes from code review
* fix: remove set intent credentials and handle ldap errors
* fix: remove set intent credentials and handle ldap errors
* refactor into separate methods and fix merge
* remove mocks
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat: add phone change and code verification for user v2 api
* feat: add phone change and code verification for user v2 api
* fix: add ignored phone.proto
* fix: integration tests
* Update proto/zitadel/user/v2alpha/user_service.proto
* Update idp_template.go
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
This PR adds an option to the LoginPolicy to "Force MFA for local users", so that users authenticated through an IDP must not configure (and verify) an MFA.
This PR adds support for the OIDC end_session_endpoint for V2 tokens. Sending an id_token_hint as parameter will directly terminate the underlying (SSO) session and all its tokens. Without this param, the user will be redirected to the Login UI, where he will able to choose if to logout.
This PR adds support for userinfo and introspection of V2 tokens. Further V2 access tokens and session tokens can be used for authentication on the ZITADEL API (like the current access tokens).
* feat: use passwap for human user passwords
* fix tests
* passwap config
* add the event mapper
* cleanup query side and api
* solve linting errors
* regression test
* try to fix linter errors again
* pass systemdefaults into externalConfigChange migration
* fix: user password set in auth view
* pin passwap v0.2.0
* v2: validate hashed password hash based on prefix
* resolve remaining comments
* add error tag and translation for unsupported hash encoding
* fix unit test
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
This PR starts the OIDC implementation for the API V2 including the Implicit and Code Flow.
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
