# Which Problems Are Solved
Integration tests are flaky due to eventual consistency.
# How the Problems Are Solved
Remove t.Parallel so that less concurrent requests on multiple instance
happen. This allows the projections to catch up more easily.
# Additional Changes
- none
# Additional Context
- none
Improve integration tests:
- spliting the tests in TokenExchange to isolated instances and in
parallel
- corrected some test structure so that the check for Details is no done
anymore if the test already failed
- replace required-calls with assert-calls to not stop the testing
- add gofakeit for application, project and usernames(emails)
- add eventually checks for testing in actions v2, so the request only
get called when the execution is defined
- check for length of results in list/search endpoints to avoid index
errors
# Which Problems Are Solved
We identified the need of caching.
Currently we have a number of places where we use different ways of
caching, like go maps or LRU.
We might also want shared chaches in the future, like Redis-based or in
special SQL tables.
# How the Problems Are Solved
Define a generic Cache interface which allows different implementations.
- A noop implementation is provided and enabled as.
- An implementation using go maps is provided
- disabled in defaults.yaml
- enabled in integration tests
- Authz middleware instance objects are cached using the interface.
# Additional Changes
- Enabled integration test command raceflag
- Fix a race condition in the limits integration test client
- Fix a number of flaky integration tests. (Because zitadel is super
fast now!) 🎸🚀
# Additional Context
Related to https://github.com/zitadel/zitadel/issues/8648
# Which Problems Are Solved
Use a single server instance for API integration tests. This optimizes
the time taken for the integration test pipeline,
because it allows running tests on multiple packages in parallel. Also,
it saves time by not start and stopping a zitadel server for every
package.
# How the Problems Are Solved
- Build a binary with `go build -race -cover ....`
- Integration tests only construct clients. The server remains running
in the background.
- The integration package and tested packages now fully utilize the API.
No more direct database access trough `query` and `command` packages.
- Use Makefile recipes to setup, start and stop the server in the
background.
- The binary has the race detector enabled
- Init and setup jobs are configured to halt immediately on race
condition
- Because the server runs in the background, races are only logged. When
the server is stopped and race logs exist, the Makefile recipe will
throw an error and print the logs.
- Makefile recipes include logic to print logs and convert coverage
reports after the server is stopped.
- Some tests need a downstream HTTP server to make requests, like quota
and milestones. A new `integration/sink` package creates an HTTP server
and uses websockets to forward HTTP request back to the test packages.
The package API uses Go channels for abstraction and easy usage.
# Additional Changes
- Integration test files already used the `//go:build integration`
directive. In order to properly split integration from unit tests,
integration test files need to be in a `integration_test` subdirectory
of their package.
- `UseIsolatedInstance` used to overwrite the `Tester.Client` for each
instance. Now a `Instance` object is returned with a gRPC client that is
connected to the isolated instance's hostname.
- The `Tester` type is now `Instance`. The object is created for the
first instance, used by default in any test. Isolated instances are also
`Instance` objects and therefore benefit from the same methods and
values. The first instance and any other us capable of creating an
isolated instance over the system API.
- All test packages run in an Isolated instance by calling
`NewInstance()`
- Individual tests that use an isolated instance use `t.Parallel()`
# Additional Context
- Closes#6684
- https://go.dev/doc/articles/race_detector
- https://go.dev/doc/build-cover
---------
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
The current v3alpha actions APIs don't exactly adhere to the [new
resources API
design](https://zitadel.com/docs/apis/v3#standard-resources).
# How the Problems Are Solved
- **Improved ID access**: The aggregate ID is added to the resource
details object, so accessing resource IDs and constructing proto
messages for resources is easier
- **Explicit Instances**: Optionally, the instance can be explicitly
given in each request
- **Pagination**: A default search limit and a max search limit are
added to the defaults.yaml. They apply to the new v3 APIs (currently
only actions). The search query defaults are changed to ascending by
creation date, because this makes the pagination results the most
deterministic. The creation date is also added to the object details.
The bug with updated creation dates is fixed for executions and targets.
- **Removed Sequences**: Removed Sequence from object details and
ProcessedSequence from search details
# Additional Changes
Object details IDs are checked in unit test only if an empty ID is
expected. Centralizing the details check also makes this internal object
more flexible for future evolutions.
# Additional Context
- Closes#8169
- Depends on https://github.com/zitadel/zitadel/pull/8225
---------
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
# Which Problems Are Solved
After migrating the access token events in #7822, milestones based on
authentication, resp. theses events would not be reached.
# How the Problems Are Solved
Additionally use the `oidc_session.Added` event to check for
`milestone.AuthenticationSucceededOnInstance` and
`milestone.AuthenticationSucceededOnApplication`.
# Additional Changes
None.
# Additional Context
- relates to #7822
- noticed internally
chore(fmt): run gci on complete project
Fix global import formatting in go code by running the `gci` command. This allows us to just use the command directly, instead of fixing the import order manually for the linter, on each PR.
Co-authored-by: Elio Bischof <elio@zitadel.com>
* feat(api): feature API proto definitions
* update proto based on discussion with @livio-a
* cleanup old feature flag stuff
* authz instance queries
* align defaults
* projection definitions
* define commands and event reducers
* implement system and instance setter APIs
* api getter implementation
* unit test repository package
* command unit tests
* unit test Get queries
* grpc converter unit tests
* migrate the V1 features
* migrate oidc to dynamic features
* projection unit test
* fix instance by host
* fix instance by id data type in sql
* fix linting errors
* add system projection test
* fix behavior inversion
* resolve proto file comments
* rename SystemDefaultLoginInstanceEventType to SystemLoginDefaultOrgEventType so it's consistent with the instance level event
* use write models and conditional set events
* system features integration tests
* instance features integration tests
* error on empty request
* documentation entry
* typo in feature.proto
* fix start unit tests
* solve linting error on key case switch
* remove system defaults after discussion with @eliobischof
* fix system feature projection
* resolve comments in defaults.yaml
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* feat(console): add time range events filter
* deprecate creation_date, use oneof filter
* use range or from
* implement api
* fix timestamp format
* translate
* styles
* lint
* integration tests
* fix until date
* rearrange sorting control
* sort creation date
* fix events e2e test
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* Update console/src/app/modules/filter-events/filter-events.component.html
Co-authored-by: Max Peintner <max@caos.ch>
* lint
* lint
* don't use utc call time
---------
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* feat: return 404 or 409 if org reg disallowed
* fix: system limit permissions
* feat: add iam limits api
* feat: disallow public org registrations on default instance
* add integration test
* test: integration
* fix test
* docs: describe public org registrations
* avoid updating docs deps
* fix system limits integration test
* silence integration tests
* fix linting
* ignore strange linter complaints
* review
* improve reset properties naming
* redefine the api
* use restrictions aggregate
* test query
* simplify and test projection
* test commands
* fix unit tests
* move integration test
* support restrictions on default instance
* also test GetRestrictions
* self review
* lint
* abstract away resource owner
* fix tests
* configure supported languages
* fix allowed languages
* fix tests
* default lang must not be restricted
* preferred language must be allowed
* change preferred languages
* check languages everywhere
* lint
* test command side
* lint
* add integration test
* add integration test
* restrict supported ui locales
* lint
* lint
* cleanup
* lint
* allow undefined preferred language
* fix integration tests
* update main
* fix env var
* ignore linter
* ignore linter
* improve integration test config
* reduce cognitive complexity
* compile
* fix(console): switch back to saved language
* feat(API): get allowed languages
* fix(console): only make allowed languages selectable
* warn when editing not allowed languages
* check for duplicates
* remove useless restriction checks
* review
* revert restriction renaming
* fix language restrictions
* lint
* generate
* allow custom texts for supported langs for now
* fix tests
* cleanup
* cleanup
* cleanup
* lint
* unsupported preferred lang is allowed
* fix integration test
* allow unsupported preferred languages
* lint
* load languages for tests
* cleanup
* lint
* cleanup
* get allowed only on admin
* cleanup
* reduce flakiness on very limited postgres
* simplify langSvc
* refactor according to suggestions in pr
* lint
* set first allowed language as default
* selectionchange for language in msg texts
* initialize login texts
* init message texts
* lint
---------
Co-authored-by: peintnermax <max@caos.ch>
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
* start feature flags
* base feature events on domain const
* setup default features
* allow setting feature in system api
* allow setting feature in admin api
* set settings in login based on feature
* fix rebasing
* unit tests
* i18n
* update policy after domain discovery
* some changes from review
* check feature and value type
* check feature and value type
* fix: nil pointer on create instance add machine
* fix: instance setup with machine user pat
* fix: correct logic to write pat and key from setup without configurable scope
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
* chore(proto): update versions
* change protoc plugin
* some cleanups
* define api for setting emails in new api
* implement user.SetEmail
* move SetEmail buisiness logic into command
* resuse newCryptoCode
* command: add ChangeEmail unit tests
Not complete, was not able to mock the generator.
* Revert "resuse newCryptoCode"
This reverts commit c89e90ae35.
* undo change to crypto code generators
* command: use a generator so we can test properly
* command: reorganise ChangeEmail
improve test coverage
* implement VerifyEmail
including unit tests
* add URL template tests
* proto: change context to object
* remove old auth option
* remove old auth option
* fix linting errors
run gci on modified files
* add permission checks and fix some errors
* comments
* comments
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* feat(instance): implement create instance with direct machine user and credentials
* fix: deprecated add endpoint and variable declaration
* fix(instance): update logic for pats and machinekeys
* fix(instance): unit test corrections and additional unit test for pats and machinekeys
* fix(instance-create): include review changes
* fix(instance-create): linter fixes
* move iframe usage to solution scenarios configurations
* Revert "move iframe usage to solution scenarios configurations"
This reverts commit 9db31f3808.
* fix merge
* fix: add review suggestions
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: add review changes
* fix: add review changes for default definitions
* fix: add review changes for machinekey details
* fix: add machinekey output when setup with machineuser
* fix: add changes from review
* fix instance converter for machine and allow overwriting of further machine fields
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: filter failed events and current sequence correctly
* fix failed events sorting column
* feat: save last occurrence of failed event
* fix failedEvents query and update sql statements
* change sql statement to only create index
* fix linting
* fix linting
* Update internal/query/failed_events.go
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* update job name on test-docs to match the one from test-code
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* feat(instance): add remove instance event with projections cleanup
* fix(instance): corrected used id to clean up projections
* fix merge
* fix: correct unit test projection names
* fix: current sequence of lists and query for ensuring keypair based projections
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
* feat: add new org scope
* change default of UserLoginMustBeDomain to false
* return resource owner claims
* fix: use email style for first user
* fix: ensure email style for default users (backwards compatibility)
* change to external domain (as it was before UserLoginMustBeDomain change)
* update e2e tests to use email style usernames
* document new scope
* lint e2e
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
