202 Commits

Author SHA1 Message Date
Elio Bischof
477cb4ad79
cleanup 2023-06-30 12:56:30 +02:00
Elio Bischof
1a530953a7
organize imports 2023-06-30 12:43:44 +02:00
Elio Bischof
053212c45e
improve dev and db performance 2023-06-30 12:39:22 +02:00
Elio Bischof
8fac74e7ee
statement unit tests 2023-06-30 10:40:40 +02:00
Elio Bischof
39e8e869ac
fix text array contains 2023-06-30 02:11:17 +02:00
Elio Bischof
d5d365d154
ignore client ids 2023-06-29 18:43:02 +02:00
Elio Bischof
3a338cdb44
push 6 in 6 milestones 2023-06-29 14:28:27 +02:00
Elio Bischof
ad13a65d1d
update base 2023-06-29 12:45:45 +02:00
Elio Bischof
41bb4f77a9
push 5 in 6 milestones 2023-06-29 12:18:16 +02:00
Elio Bischof
2b8dac40de
fix scheduled pseudo event projection 2023-06-28 17:43:19 +02:00
Elio Bischof
a14871ce46
reduce milestone pushed 2023-06-28 11:35:22 +02:00
Elio Bischof
51a9a54cfd
calculate and push 4 in 6 milestones 2023-06-28 08:19:34 +02:00
Livio Spring
82e7333169
feat(api): add password reset and change to user service (#6036)
* feat(api): add password reset and change to user service

* integration tests

* invalidate password check after password change

* handle notification type

* fix proto
2023-06-20 17:34:06 +02:00
Tim Möhlmann
09aafb35eb
feat(v2): implement user register OTP (#6030)
* feat(v2): implement user register OTP

* feat(v2): implement user verify OTP

* session: retry on permission denied
2023-06-20 10:36:21 +00:00
Elio Bischof
61feb9d19f
fix: more silence (#5986) 2023-06-06 15:12:54 +00:00
Silvan
39bdef35e7
chore: merge (#5773)
* feat: allow skip of success page for native apps (#5627)

add possibility to return to callback directly after login without rendering the successful login page

* build next

* fix(console): disallow inline fonts, critical styles (#5714)

fix: disallow inline

* fix(setup): step 10 for postgres (#5717)

* fix(setup): smaller transactions (#5743)

* fix: order by sequence by default

* test: add allowCreationDateFilter

* fix(step10): separate executions (#5754)

* feat: allow skip of success page for native apps (#5627)

add possibility to return to callback directly after login without rendering the successful login page

* build next

* fix(console): disallow inline fonts, critical styles (#5714)

fix: disallow inline

* fix(setup): step 10 for postgres (#5717)

* fix(setup): smaller transactions (#5743)

* fix(step10): split statements

* fix(step10): split into separate execs

* chore: prerelease

* add truncate before insert

* fix: add truncate

* Merge branch 'main' into optimise-step-10

* chore: reset release definition

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2023-04-28 14:56:51 +00:00
Livio Spring
458a383de2
fix: use current sequence for refetching of events (#5772)
* fix: use current sequence for refetching of events

* fix: use client ids
2023-04-28 16:28:13 +02:00
Elio Bischof
c12c2f09a4
fix: switch log level of failed locks to debug (#5746) 2023-04-25 19:20:59 +02:00
James Schinner
2a79e77c7b
docs: Fix typo (#5727)
Fix typo
2023-04-24 13:23:38 +02:00
Tim Möhlmann
5819924275
feat: device authorization RFC 8628 (#5646)
* device auth: implement the write events

* add grant type device code

* fix(init): check if default value implements stringer

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-04-19 08:46:02 +00:00
Silvan
8da8fbe6ce
fix(eventstore): correct creation date of events (#5683)
* fix: add setup step to correct creation dates

* fix(eventstore): replace now with statement ts

* fix(step10): correct number

* fix: handle wrong instance domain removed events
2023-04-18 19:29:04 +02:00
Silvan
4c1169b562
feat(eventstore): order by creation_date and sequence (#5568)
* feat(eventstore): order by `creation_date` and `sequence`

* fix(logstore): use correct event type

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-04-05 09:34:24 +00:00
Silvan
dd84b93ac3
fix(token): filter users by instance id (#5596)
* fix(token): filter users by instance id
2023-04-03 12:56:37 +00:00
Elio Bischof
887e2f474d
fix: use fixed active instances duration (#5567)
* fix: use fixed active instances duration

* fix active instances tests

* fix syntax error

* run pipeline

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-30 13:01:27 +02:00
Elio Bischof
62bd606593
feat: enable handling projections for inactive instances (#5523)
* fix: don't ignore failed events in handlers

* question

* fix retries

* don't instance ids query

* statements can be nil

* make unit tests pass

* add comments

* spool only active instances

* feat(config): handle inactive instances

* customizable HandleInactiveInstances

* test: handling with and w/o inactive instances

* docs: describe projection options

* enable global handling of inactive instances

* accept NowFunc, not Clock interface

* add comment about stringer usage

* remove enum stringer implementations

* fix enum format types

* Update internal/eventstore/repository/mock/repository.mock.impl.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-03-27 12:34:01 +00:00
Silvan
e38abdcdf3
perf: query data AS OF SYSTEM TIME (#5231)
Queries the data in the storage layser at the timestamp when the call hit the API layer
2023-02-27 22:36:43 +01:00
Elio Bischof
681541f41b
feat: add quotas (#4779)
adds possibilities to cap authenticated requests and execution seconds of actions on a defined intervall
2023-02-15 02:52:11 +01:00
Silvan
7b5135e637
fix(adminAPI): localize event type (#5059)
* fix(adminAPI): localisation of event types, aggregate types
* fix(adminAPI): validations of ListEvent request
* implement caching of editor user information
2023-01-19 15:50:05 +00:00
Silvan
1bf1f335dc
feat(admin-api): list events (#4989)
* docs: update cockroachdb version to 22.2
* feat(adminAPI): ListEventTypes returns the list of event types ZITADEL implements
* feat(adminAPI): ListAggregateTypes returns the list of aggregate types ZITADEL implements
* feat(adminAPI): ListEvents allows `IAM_OWNERS` to search for events
2023-01-16 11:30:03 +00:00
Livio Spring
d21bb902f1
fix: push timeout (#4882) (#4885)
* push with timeout

* test: config for eventstore

(cherry picked from commit b9156da76d0f03075589b50eafbf9f48160a0301)

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-12-15 09:40:13 +00:00
Silvan
f3e6f3b23b
feat: remove org (#4148)
* feat(command): remove org

* refactor: imports, unused code, error handling

* reduce org removed in action

* add org deletion to projections

* add org removal to projections

* add org removal to projections

* org removed projection

* lint import

* projections

* fix: table names in tests

* fix: table names in tests

* logging

* add org state

* fix(domain): add Owner removed to object details

* feat(ListQuery): add with owner removed

* fix(org-delete): add bool to functions to select with owner removed

* fix(org-delete): add bools to user grants with events to determine if dependencies lost owner

* fix(org-delete): add unit tests for owner removed and org removed events

* fix(org-delete): add handling of org remove for grants and members

* fix(org-delete): correction of unit tests for owner removed

* fix(org-delete): update projections, unit tests and get functions

* fix(org-delete): add change date to authnkeys and owner removed to org metadata

* fix(org-delete): include owner removed for login names

* fix(org-delete): some column fixes in projections and build for queries with owner removed

* indexes

* fix(org-delete): include review changes

* fix(org-delete): change user projection name after merge

* fix(org-delete): include review changes for project grant where no project owner is necessary

* fix(org-delete): include auth and adminapi tables with owner removed information

* fix(org-delete): cleanup username and orgdomain uniqueconstraints when org is removed

* fix(org-delete): add permissions for org.remove

* remove unnecessary unique constraints

* fix column order in primary keys

* fix(org-delete): include review changes

* fix(org-delete): add owner removed indexes and chang setup step to create tables

* fix(org-delete): move PK order of instance_id and change added user_grant from review

* fix(org-delete): no params for prepareUserQuery

* change to step 6

* merge main

* fix(org-delete): OldUserName rename to private

* fix linting

* cleanup

* fix: remove org test

* create prerelease

* chore: delete org-delete as prerelease

Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2022-11-30 17:01:17 +01:00
Livio Spring
e8babf1048
fix: reduce load on view tables (#4716)
* fix: reduce load on view tables

* create prerelease

* linting: pass context to view handlers

* fix error handling of refresh token handler

* fix: improve processing of successful instanceIDs on views

* fix revert intended change in .golangci.yaml

* fix: set timeout for processInstances in spooler

* fix: reduce update to active tokens on profile change

* change token expiration query to db now()

* remove branch from .releaserc.js
2022-11-22 07:36:48 +01:00
Livio Spring
29441ce4b6
feat: save last occurrence of failed events and fix instance filtering (#4710)
* fix: filter failed events and current sequence correctly

* fix failed events sorting column

* feat: save last occurrence of failed event

* fix failedEvents query and update sql statements

* change sql statement to only create index

* fix linting

* fix linting

* Update internal/query/failed_events.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* update job name on test-docs to match the one from test-code

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-11-18 13:49:38 +01:00
Livio Spring
5ac956b0f1
fix: allow version to be overwritten by build again (#4656) 2022-11-07 09:50:44 +00:00
Livio Spring
c791f6de58
fix: improve startup times by initializing projection tables during setup (#4642)
* fix: improve startup times by initializing projections table during setup

* add missing file
2022-11-04 09:21:58 +00:00
Stefan Benz
5d17da542d
fix(user): add search query for login name (#4173)
* fix(user): add search query for login name

* fix(user): change login name query to IN from EXISTS

* fix(loginname): include InQuery into ListQuery with SubSelect as possible datasource

* fix(user): apply suggestions from code review

Co-authored-by: Livio Spring <livio.a@gmail.com>

* fix: correct unit test for search query

Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2022-10-31 13:03:23 +00:00
Livio Spring
d721f725fd
fix: instance remove (#4602) 2022-10-26 13:06:48 +00:00
Stefan Benz
c2a5b785fb
feat: instance remove (#4345)
* feat(instance): add remove instance event with projections cleanup

* fix(instance): corrected used id to clean up projections

* fix merge

* fix: correct unit test projection names

* fix: current sequence of lists and query for ensuring keypair based projections

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
2022-10-20 14:36:52 +02:00
Livio Spring
b0b1e94090
feat(login): additionally use email/phone for authentication (#4563)
* feat: add ability to disable login by email and phone

* feat: check login by email and phone

* fix: set verified email / phone correctly on notify users

* update projection version

* fix merge

* fix email/phone verified reduce tests

* fix user tests

* loginname check

* cleanup

* fix: update user projection version to handle fixed statement
2022-10-17 19:19:15 +00:00
Livio Spring
fcb36cd406
fix: wait for projection initialization to be done (#4473)
* fix: wait for projection initialization to be done

* close channel

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-10-03 14:09:59 +00:00
Livio Spring
5aa91ad105
fix: improve performance (#4300)
## Note

This release requires a setup step to fully improve performance.
Be sure to start ZITADEL with an appropriate command (zitadel start-from-init / start-from-setup)

## Changes

- fix: only run projection scheduler on active instances
- fix: set default for concurrent instances of projections to 1 (for scheduling)
- fix: create more indexes on eventstore.events table
- fix: get current sequence for token check (improve reread performance)
2022-09-02 14:05:13 +00:00
Silvan
2f647ce9a2
fix: initial failures (#4291)
* fix(cmd): read configuration correctly

* fix(database): read weakly typed config

* fix(database): correct handling of update columns

Co-authored-by: Livio Spring <livio.a@gmail.com>
2022-09-01 07:24:26 +00:00
Silvan
77b4fc5487
feat(database): support for postgres (#3998)
* beginning with postgres statements

* try pgx

* use pgx

* database

* init works for postgres

* arrays working

* init for cockroach

* init

* start tests

* tests

* TESTS

* ch

* ch

* chore: use go 1.18

* read stmts

* fix typo

* tests

* connection string

* add missing error handler

* cleanup

* start all apis

* go mod tidy

* old update

* switch back to minute

* on conflict

* replace string slice with `database.StringArray` in db models

* fix tests and start

* update go version in dockerfile

* setup go

* clean up

* remove notification migration

* update

* docs: add deploy guide for postgres

* fix: revert sonyflake

* use `database.StringArray` for daos

* use `database.StringArray` every where

* new tables

* index naming,
metadata primary key,
project grant role key type

* docs(postgres): change to beta

* chore: correct compose

* fix(defaults): add empty postgres config

* refactor: remove unused code

* docs: add postgres to self hosted

* fix broken link

* so?

* change title

* add mdx to link

* fix stmt

* update goreleaser in test-code

* docs: improve postgres example

* update more projections

* fix: add beta log for postgres

* revert index name change

* prerelease

* fix: add sequence to v1 "reduce paniced"

* log if nil

* add logging

* fix: log output

* fix(import): check if org exists and user

* refactor: imports

* fix(user): ignore malformed events

* refactor: method naming

* fix: test

* refactor: correct errors.Is call

* ci: don't build dev binaries on main

* fix(go releaser): update version to 1.11.0

* fix(user): projection should not break

* fix(user): handle error properly

* docs: correct config example

* Update .releaserc.js

* Update .releaserc.js

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
2022-08-31 07:52:43 +00:00
Silvan
60b2092d2c
fix(import): check exists (#4268)
* fix(import): check if org exists and user

* refactor: imports

* fix(user): ignore malformed events

* refactor: method naming

* fix: test

* refactor: correct errors.Is call
2022-08-29 17:09:07 +02:00
Stefan Benz
bc9a85daf3
feat: V2 alpha import and export of organizations (#3798)
* feat(import): add functionality to import data into an instance

* feat(import): move import to admin api and additional checks for nil pointer

* fix(export): export implementation with filtered members and grants

* fix: export and import implementation

* fix: add possibility to export hashed passwords with the user

* fix(import): import with structure of v1 and v2

* docs: add v1 proto

* fix(import): check im imported user is already existing

* fix(import): add otp import function

* fix(import): add external idps, domains, custom text and messages

* fix(import): correct usage of default values from login policy

* fix(export): fix renaming of add project function

* fix(import): move checks for unit tests

* expect filter

* fix(import): move checks for unit tests

* fix(import): move checks for unit tests

* fix(import): produce prerelease from branch

* fix(import): correctly use provided user id for machine user imports

* fix(import): corrected otp import and added guide for export and import

* fix: import verified and primary domains

* fix(import): add reading from gcs, s3 and localfile with tracing

* fix(import): gcs and s3, file size correction and error logging

* Delete docker-compose.yml

* fix(import): progress logging and count of resources

* fix(import): progress logging and count of resources

* log subscription

* fix(import): incorporate review

* fix(import): incorporate review

* docs: add suggestion for import

Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>

* fix(import): add verification otp event and handling of deleted but existing users

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Fabienne <fabienne.gerschwiler@gmail.com>
Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
2022-07-28 13:42:35 +00:00
Livio Spring
aed7010508
fix: scheduling (#3978)
* fix: improve scheduling

* build pre-release

* fix: locker

* fix: user handler and print stack in case of panic in reducer

* chore: remove sentry

* fix: improve handler projection and implement tests

* more tests

* fix: race condition in tests

* Update internal/eventstore/repository/sql/query.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* fix: implemented suggested changes

* fix: lock statement

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2022-07-22 10:08:39 +00:00
Livio Spring
a1d404291d
fix(notify): notify user in projection (#3889)
* start implement notify user in projection

* fix(stmt): add copy to multi stmt

* use projections for notify users

* feat: notifications from projections

* feat: notifications from projections

* cleanup

* pre-release

* fix tests

* fix types

* fix command

* fix queryNotifyUser

* fix: build version

* fix: HumanPasswordlessInitCodeSent

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2022-07-06 14:09:49 +02:00
Livio Spring
12d4d3ea0b
fix: enable env vars in setup steps (and deprecate admin subcommand) (#3871)
* fix: enable env vars in setup steps (and deprecate admin subcommand)

* fix tests and error text
2022-06-27 10:32:34 +00:00
Silvan
dd2f31683c
fix(query): realtime data on defined requests (#3726)
* feat: directly specify factors on addCustomLoginPolicy and return on LoginPolicy responses

* fix proto

* update login policy

* feat: directly specify idp on addCustomLoginPolicy and return on LoginPolicy responses

* fix: tests

* fix(projection): trigger bulk

* refactor: clean projection pkg

* instance should bulk

* fix(query): should trigger bulk on id calls

* tests

* build prerelease

* fix: add shouldTriggerBulk

* fix: test

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2022-06-14 07:51:00 +02:00
Livio Spring
ace94917da
test: ensure consistency of TestSpooler_awaitError (#3750) 2022-06-07 11:40:46 +00:00