Commit Graph

8 Commits

Author SHA1 Message Date
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Elio Bischof
1d4ec6cdba
fix: set quotas (#6597)
* feat: set quotas

* fix: start new period on younger anchor

* cleanup e2e config

* fix set notifications

* lint

* test: fix quota projection tests

* fix add quota tests

* make quota fields nullable

* enable amount 0

* fix initial setup

* create a prerelease

* avoid success comments

* fix quota projection primary key

* Revert "fix quota projection primary key"

This reverts commit e72f4d7fa1.

* simplify write model

* fix aggregate id

* avoid push without changes

* test set quota lifecycle

* test set quota mutations

* fix quota unit test

* fix: quotas

* test quota.set event projection

* use SetQuota in integration tests

* fix: release quotas 3

* reset releaserc

* fix comment

* test notification order doesn't matter

* test notification order doesn't matter

* test with unmarshalled events

* test with unmarshalled events

(cherry picked from commit ae1af6bc8c)
2023-09-22 13:06:59 +02:00
Elio Bischof
5823fdbef9
perf: project quotas and usages (#6441)
* project quota added

* project quota removed

* add periods table

* make log record generic

* accumulate usage

* query usage

* count action run seconds

* fix filter in ReportQuotaUsage

* fix existing tests

* fix logstore tests

* fix typo

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* move notifications into debouncer and improve limit querying

* cleanup

* comment

* fix: add quota unit tests command side

* fix remaining quota usage query

* implement InmemLogStorage

* cleanup and linting

* improve test

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* action notifications and fixes for notifications query

* revert console prefix

* fix: add quota unit tests command side

* fix: add quota integration tests

* improve accountable requests

* improve accountable requests

* fix: add quota integration tests

* fix: add quota integration tests

* fix: add quota integration tests

* comment

* remove ability to store logs in db and other changes requested from review

* changes requested from review

* changes requested from review

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* tests: fix quotas integration tests

* improve incrementUsageStatement

* linting

* fix: delete e2e tests as intergation tests cover functionality

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* backup

* fix conflict

* create rc

* create prerelease

* remove issue release labeling

* fix tracing

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
(cherry picked from commit 1a49b7d298)
2023-09-15 17:00:12 +02:00
Stefan Benz
3b9dabcf36
chore: build image for digest and release on workflow_dispatch (#6287) 2023-08-10 12:21:01 +00:00
Elio Bischof
747f1b3d6d
test(quotas): test exhausted cookie handling (#5788)
* fix: remove access interceptor for console

* feat: template quota cookie value

* feat: show dialog on exhausted requests

* test: test quota exhausted screen

* chore: add convencience npm scripts

* test: ensure correct exhausted cookie handling

* fix exhausted cookie handling

* fix: send exhausted cookie from grpc-gateway

* fix linting

* refactor: remove ineffectual err assignments

* create prerelease

* test: simulate upgrading an instance

* run all quota tests

* fix quota exhausted screen

* fix: release

* fix linting

* Update internal/api/grpc/server/gateway.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* use dynamic host header to find instance

* add instance mgmt url to environment.json

* read instance mgmt from environment.json

* configure console

* fix interceptors

* fix: release

* mgmt url comes from environment.json

* don't check cookie on environment.json

* fix: release

* support hosts with default ports

* fix: release

* fix environment.json path

* fix linting

* docs: update lb example

* print access logs to stdout

* fix grpc gateway exhausted cookies

* update backend

* cleanup

* fix: release

* fix: release

* exclude environment.json from cookie check

* fix: release

* fix: release

* remove cookie before loading env

* use UTC to delete the cookie

* delete cookie before fetching env

* fix: release

* simplify cookie handling

* lint

* fix: set exhausted cookie with env json

* lint

* review cleanup

* fix: release

* use exhausted property from env json

* fix: send exhausted property in env json

* fix bootstrapping

* lint

* fix: release

* always open mgmt link if present

* fix: release

* fix: release

* chore: fetch env json before ng serve

* wait for cookie to be removed

* fix: release

* fix typo

* fix: release

* fix: debug

* fix: delete cookies

* don't wait for cookie to be set

* fix: delete cookies

* cleanup

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-05-22 16:32:50 +02:00
Elio Bischof
cccccd005c
feat: call webhooks at least once (#5454)
* feat: call webhooks at least once

* self review

* feat: improve notification observability

* feat: add notification tracing

* test(e2e): test at-least-once webhook delivery

* fix webhook notifications

* dedicated quota notifications handler

* fix linting

* fix e2e test

* wait less in e2e test

* fix: don't ignore failed events in handlers

* fix: don't ignore failed events in handlers

* faster requeues

* question

* fix retries

* fix retries

* retry

* don't instance ids query

* revert handler_projection

* statements can be nil

* cleanup

* make unit tests pass

* add comments

* add comments

* lint

* spool only active instances

* feat(config): handle inactive instances

* customizable HandleInactiveInstances

* call inactive instances quota webhooks

* test: handling with and w/o inactive instances

* omit retrying noop statements

* docs: describe projection options

* enable global handling of inactive instances

* self review

* requeue quota notifications every 5m

* remove caos_errors reference

* fix comment styles

* make handlers package flat

* fix linting

* fix repeating quota notifications

* test with more usage

* debug log channel init failures
2023-03-28 22:09:06 +00:00
Elio Bischof
e00cc187fa
fix: make user creation errors helpful (#5382)
* fix: make user creation errors helpful

* fix linting and unit testing errors

* fix linting

* make zitadel config reusable

* fix human validations

* translate ssr errors

* make zitadel config reusable

* cover more translations for ssr

* handle email validation message centrally

* fix unit tests

* fix linting

* align signatures

* use more precise wording

* handle phone validation message centrally

* fix: return specific profile errors

* docs: edit comments

* fix unit tests

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-14 19:20:38 +00:00
Elio Bischof
abacb6c5aa
chore: improve development for non-Linux contributors (#5288)
* test: fix e2e against console dev server

* chore: get rid of network_mode host

* explain e2e commands

* chore: fix pipelines

* fix e2e paths

* fix dockerized e2e

* chore: map cypress run service ports

* simplify localhost

* access db via compose service

* access db via compose service

* fix npm run open:angular and e2e:angular

* docs: add empty line

* chore: remove unused file

* docs:  update contrib

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-02-28 17:49:19 +00:00