Commit Graph

349 Commits

Author SHA1 Message Date
Elio Bischof
1a49b7d298
perf: project quotas and usages (#6441)
* project quota added

* project quota removed

* add periods table

* make log record generic

* accumulate usage

* query usage

* count action run seconds

* fix filter in ReportQuotaUsage

* fix existing tests

* fix logstore tests

* fix typo

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* move notifications into debouncer and improve limit querying

* cleanup

* comment

* fix: add quota unit tests command side

* fix remaining quota usage query

* implement InmemLogStorage

* cleanup and linting

* improve test

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* fix: add quota unit tests command side

* action notifications and fixes for notifications query

* revert console prefix

* fix: add quota unit tests command side

* fix: add quota integration tests

* improve accountable requests

* improve accountable requests

* fix: add quota integration tests

* fix: add quota integration tests

* fix: add quota integration tests

* comment

* remove ability to store logs in db and other changes requested from review

* changes requested from review

* changes requested from review

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* tests: fix quotas integration tests

* improve incrementUsageStatement

* linting

* fix: delete e2e tests as intergation tests cover functionality

* Update internal/api/http/middleware/access_interceptor.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

* backup

* fix conflict

* create rc

* create prerelease

* remove issue release labeling

* fix tracing

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2023-09-15 16:58:45 +02:00
Tim Möhlmann
5d568d82d1
fix(defaults): uncomment Verifiers (#6557)
It seems that options that are completely commented
out do not get parsed from the envionment variables.
This was also to the case for the
`ZITADEL_SYSTEMDEFAULTS_PASSWORDHASHER_VERIFIERS` option.
This change just uncomments the yaml option, so that users
can use the envorment variable to set a list of verifiers they wish to
enable.
2023-09-14 11:25:48 +00:00
Livio Spring
e17b49e4ca
feat: add apple as idp (#6442)
* feat: manage apple idp

* handle apple idp callback

* add tests for provider

* basic console implementation

* implement flow for login UI and add logos / styling

* tests

* cleanup

* add upload button

* begin i18n

* apple logo positioning, file upload component

* fix add apple instance idp

* add missing apple logos for login

* update to go 1.21

* fix slice compare

* revert permission changes

* concrete error messages

* translate login apple logo -y-2px

* change form parsing

* sign in button

* fix tests

* lint console

---------

Co-authored-by: peintnermax <max@caos.ch>
2023-08-31 08:39:16 +02:00
Miguel Cabrerizo
fd00ac533a
feat: add reply-to header in email notification (#6393)
* feat: add reply-to header to smtp messages

* fix: grpc reply_to_address min 0 and js var name

* fix: add missing translations

* fix merge and linting

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-08-29 09:08:24 +02:00
Livio Spring
bb40e173bd
feat(api): add otp (sms and email) checks in session api (#6422)
* feat: add otp (sms and email) checks in session api

* implement sending

* fix tests

* add tests

* add integration tests

* fix merge main and add tests

* put default OTP Email url into config

---------

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-08-24 09:41:52 +00:00
Livio Spring
cbd2ef0612
fix: use system secret config if generator type does not exist on instance (#6420)
* fix: use system secret config if generator type does not exist on instance

* remove unused idGenerator
2023-08-23 08:04:29 +00:00
Silvan
99e1c654a3
feat(storage): read only transactions for queries (#6415)
* fix: tests

* bastle wie en grosse

* fix(database): scan as callback

* fix tests

* fix merge failures

* remove as of system time

* refactor: remove unused test

* refacotr: remove unused lines
2023-08-22 10:49:22 +00:00
Livio Spring
7c494fd219
feat(login): add OTP (email and sms) (#6353)
* feat: login with otp

* fix(i18n): japanese translation

* add missing files

* fix provider change

* add event types translations to en

* add tests

* resourceOwner

* remove unused handler

* fix: secret generators and add comments

* add setup step

* rename

* linting

* fix setup

* improve otp handling

* fix autocomplete

* translations for login and notifications

* translations for event types

* changes from review

* check selected mfa type
2023-08-15 12:47:05 +00:00
Alan Hughes
faa9ed4de9
docs: fix external domain and external key env var names (#6367) 2023-08-15 11:53:26 +00:00
Livio Spring
372755bddd
feat(api): add organisation service (#6340)
* setup org with multiple admins

* tests

* add missing proto

* remove machine users (for now)

* update tests with idp case

* fix package

* organisation -> organization

* fix test
2023-08-11 14:19:14 +00:00
Elio Bischof
7bc4aa9c76
docs: copy config options to docs (#6143)
* poc

* wip

* works

* upgrade yaml package

* need to add global comments for shifting

* wip: need index before working on comments

* green

* clean up

* test null value

* comment

* package

* delete

* convert to module

* render md table

* tests with ESM

* comments

* top level gitignore

* wip: new cases

* arrays: green

* array_test

* treat comments on map without first element

* fix some new case

* skip leaf comments

* output folder

* comment

* finish up for poc

* arrays

* create output dir

* merge main, create tables

* copy config options to docs

* cleanup

* recommend file configuration

* language

* add some explanations

* some small typo fixes

---------

Co-authored-by: mffap <mpa@zitadel.com>
Co-authored-by: Florian Forster <florian@zitadel.com>
2023-08-07 22:32:10 +02:00
Tim Möhlmann
4d09409328
feat(crypto): add pbkdf2 support (#6303)
This change brings pbkdf2 support for password hashing and verification.
2023-08-02 11:27:18 +00:00
Livio Spring
2fe76acd14
feat: add secret generators for OTP (#6262)
This PR adds configuration options for OTP codes through Admin API.
2023-07-26 11:00:41 +00:00
Elio Bischof
35f4f74b08
fix: remove logstore beta warning (#6244)
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-25 08:44:47 +02:00
Elio Bischof
57d6113e2a
fix: upper ZITADEL banner (#6246)
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-25 05:12:13 +00:00
Livio Spring
59f3c328ec
feat(OIDC): add support for end_session for V2 tokens (#6226)
This PR adds support for the OIDC end_session_endpoint for V2 tokens. Sending an id_token_hint as parameter will directly terminate the underlying (SSO) session and all its tokens. Without this param, the user will be redirected to the Login UI, where he will able to choose if to logout.
2023-07-19 13:17:39 +02:00
Silvan
1c354ca977
ci: improve performance (#5953)
* pipeline runs on ubuntu instead of docker
* added Makefile to build zitadel core (backend) and console (frontend)
* pipeline runs in parallel where possible
* pipeline is split into multiple jobs
* removed goreleaser
* added command to check if zitadel instance is running
2023-07-17 10:08:20 +02:00
Tim Möhlmann
4589ddad4a
feat: integrate passwap for human user password hashing (#6196)
* feat: use passwap for human user passwords

* fix tests

* passwap config

* add the event mapper

* cleanup query side and api

* solve linting errors

* regression test

* try to fix linter errors again

* pass systemdefaults into externalConfigChange migration

* fix: user password set in auth view

* pin passwap v0.2.0

* v2: validate hashed password hash based on prefix

* resolve remaining comments

* add error tag and translation for unsupported hash encoding

* fix unit test

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-07-14 09:49:57 +03:00
Miguel Cabrerizo
0f3c33cb86
feat: show basic info start-from-init (#6183) 2023-07-12 20:46:50 +03:00
Livio Spring
14b8cf4894
feat(api): add OIDC session service (#6157)
This PR starts the OIDC implementation for the API V2 including the Implicit and Code Flow.


Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2023-07-10 13:27:00 +00:00
Fabi
48bda9aa07
fix: Role (#6160)
* fix: add user self manager role (can delete himself)

* fix: add user self manager role (can delete himself)

* fix: add user self manager role (can delete himself)
2023-07-07 20:14:07 +00:00
Elio Bischof
9b768003b7
feat: improve milestone format (#6150)
* feat: milestone format

* feat: push external domain

* cleanup

* Revert "remove prerelease"

This reverts commit 7417fdbeb3.

* fix branch

* remove prerelease
2023-07-06 19:31:08 +02:00
Elio Bischof
bb756482c7
feat: push telemetry (#6027)
* document analytics config

* rework configuration and docs

* describe HandleActiveInstances better

* describe active instances on quotas better

* only projected events are considered

* cleanup

* describe changes at runtime

* push milestones

* stop tracking events

* calculate and push 4 in 6 milestones

* reduce milestone pushed

* remove docs

* fix scheduled pseudo event projection

* push 5 in 6 milestones

* push 6 in 6 milestones

* ignore client ids

* fix text array contains

* push human readable milestone type

* statement unit tests

* improve dev and db performance

* organize imports

* cleanup

* organize imports

* test projection

* check rows.Err()

* test search query

* pass linting

* review

* test 4 milestones

* simplify milestone by instance ids query

* use type NamespacedCondition

* cleanup

* lint

* lint

* dont overwrite original error

* no opt-in in examples

* cleanup

* prerelease

* enable request headers

* make limit configurable

* review fixes

* only requeue special handlers secondly

* include integration tests

* Revert "include integration tests"

This reverts commit 96db9504ec.

* pass reducers

* test handlers

* fix unit test

* feat: increment version

* lint

* remove prerelease

* fix integration tests
2023-07-06 08:38:13 +02:00
Silvan
d0cda1b479
fix(migration): speed up step 11 (#6086) 2023-06-27 11:56:17 +02:00
Livio Spring
7046194530
feat(api): list authentication method types in user api v2 (#6058) 2023-06-20 16:23:28 +00:00
Stefan Benz
855d6b1bd5
fix: nil pointer on create instance add machine (#6000)
* fix: nil pointer on create instance add machine

* fix: instance setup with machine user pat

* fix: correct logic to write pat and key from setup without configurable scope

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-06-15 06:16:39 +00:00
Silvan
6be41ea257
fix(setup): steps 10 and 11 (#5987)
* fix(step10): count amount of wrong_events manually

* fix(step11): create index for better performance
2023-06-07 16:30:19 +00:00
Silvan
ebca7b6e3d
fix(setup): update chunks in step 11 (#5965) 2023-06-02 09:31:18 +00:00
Stefan Benz
fa8f191812
feat: v2alpha user service idp endpoints (#5879)
* feat: v2alpha user service idp endpoints

* feat: v2alpha user service intent endpoints

* begin idp intents (callback)

* some cleanup

* runnable idp authentication

* cleanup

* proto cleanup

* retrieve idp info

* improve success and failure handling

* some unit tests

* grpc unit tests

* add permission check AddUserIDPLink

* feat: v2alpha intent writemodel refactoring

* feat: v2alpha intent writemodel refactoring

* feat: v2alpha intent writemodel refactoring

* provider from write model

* fix idp type model and add integration tests

* proto cleanup

* fix integration test

* add missing import

* add more integration tests

* auth url test

* feat: v2alpha intent writemodel refactoring

* remove unused functions

* check token on RetrieveIdentityProviderInformation

* feat: v2alpha intent writemodel refactoring

* fix TestServer_RetrieveIdentityProviderInformation

* fix test

* i18n and linting

* feat: v2alpha intent review changes

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-05-24 18:29:58 +00:00
Tim Möhlmann
a301c40f9f
feat: implement register Passkey user API v2 (#5873)
* command/crypto: DRY the code

- reuse the the algorithm switch to create a secret generator
- add a verifyCryptoCode function

* command: crypto code tests

* migrate webauthn package

* finish integration tests with webauthn mock client
2023-05-24 10:22:00 +00:00
Livio Spring
2962870bdd
Merge branch 'main' into eventstore-created-at 2023-05-16 08:44:25 +02:00
Elio Bischof
0e251a29c8
fix: set exhausted cookie with env json (#5868)
* fix: set exhausted cookie with env json

* lint
2023-05-15 08:51:02 +02:00
adlerhurst
4d7a733e11 fix(step11): execute step 10 to make sure events are in correct order 2023-05-11 11:53:52 +02:00
Silvan
defa429eb7
Merge branch 'main' into eventstore-created-at 2023-05-11 11:24:23 +02:00
Stefan Benz
8d13f170e8
feat(api): new settings service (#5775)
* feat: add v2alpha policies service

* feat: add v2alpha policies service

* fix: rename of attributes and messages in v2alpha api

* fix: rename of attributes and messages in v2alpha api

* fix: linter corrections

* fix: review corrections

* fix: review corrections

* fix: review corrections

* fix: review corrections

* fix grpc

* refactor: rename to settings and more

* Apply suggestions from code review

Co-authored-by: Fabi <fabienne.gerschwiler@gmail.com>

* add service to docs and rename legal settings

* unit tests for converters

* go mod tidy

* ensure idp name and return list details

* fix: use correct resource owner for active idps

* change query to join

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Fabi <fabienne.gerschwiler@gmail.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-05-11 09:23:40 +00:00
adlerhurst
b9a3fac3d2 fix(eventstore): backfill column 2023-05-11 11:14:42 +02:00
Livio Spring
c07411e314
fix: only reuse port for integration tests (#5817)
* fix: only reuse port for integration tests

* exclude default listenConfig from integration build
2023-05-11 08:58:35 +00:00
adlerhurst
5a3d09d3a3 fix(eventstore): use creation_date for existing columns 2023-05-11 10:31:33 +02:00
adlerhurst
c6d29fc201 fix(eventstore): new column to test clock_timestamp() 2023-05-11 10:04:35 +02:00
Elio Bischof
35a0977663
fix: improve exhausted SetCookie header (#5789)
* fix: remove access interceptor for console

* feat: template quota cookie value

* fix: send exhausted cookie from grpc-gateway

* refactor: remove ineffectual err assignments

* Update internal/api/grpc/server/gateway.go

Co-authored-by: Livio Spring <livio.a@gmail.com>

* use dynamic host header to find instance

* add instance mgmt url to environment.json

* support hosts with default ports

* fix linting

* docs: update lb example

* print access logs to stdout

* fix grpc gateway exhausted cookies

* cleanup

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-05-11 09:24:44 +02:00
Livio Spring
c2cb84cd24
feat(api): new session service (#5801)
* backup new protoc plugin

* backup

* session

* backup

* initial implementation

* change to specific events

* implement tests

* cleanup

* refactor: use new protoc plugin for api v2

* change package

* simplify code

* cleanup

* cleanup

* fix merge

* start queries

* fix tests

* improve returned values

* add token to projection

* tests

* test db map

* update query

* permission checks

* fix tests and linting

* rework token creation

* i18n

* refactor token check and fix tests

* session to PB test

* request to query tests

* cleanup proto

* test user check

* add comment

* simplify database map type

* Update docs/docs/guides/integrate/access-zitadel-system-api.md

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>

* fix test

* cleanup

* docs

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-05-05 15:34:53 +00:00
Miguel Cabrerizo
3ca7147808
fix: introduce measures to avoid bots crawling and indexing activities (#5728)
* fix: 404 for robots.txt and meta robots tags

* fix: add unit tests for robots txt and tag

* fix: add meta tag robots none for login pages

* fix: weird format issue in header.go

* fix: add x-robots-tag=none to grpcwebserver

* fix linting

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-05-05 10:25:02 +02:00
Tim Möhlmann
4934d6f4fa Merge branch 'main' into integration-tests 2023-05-02 19:24:37 +03:00
Tim Möhlmann
c839cb3ce0 tie loose ends, documentation 2023-05-02 19:24:24 +03:00
Silvan
39bdef35e7
chore: merge (#5773)
* feat: allow skip of success page for native apps (#5627)

add possibility to return to callback directly after login without rendering the successful login page

* build next

* fix(console): disallow inline fonts, critical styles (#5714)

fix: disallow inline

* fix(setup): step 10 for postgres (#5717)

* fix(setup): smaller transactions (#5743)

* fix: order by sequence by default

* test: add allowCreationDateFilter

* fix(step10): separate executions (#5754)

* feat: allow skip of success page for native apps (#5627)

add possibility to return to callback directly after login without rendering the successful login page

* build next

* fix(console): disallow inline fonts, critical styles (#5714)

fix: disallow inline

* fix(setup): step 10 for postgres (#5717)

* fix(setup): smaller transactions (#5743)

* fix(step10): split statements

* fix(step10): split into separate execs

* chore: prerelease

* add truncate before insert

* fix: add truncate

* Merge branch 'main' into optimise-step-10

* chore: reset release definition

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2023-04-28 14:56:51 +00:00
Silvan
c8c5cf3c5f
feat(cli): add setup cleanup sub command (#5770)
* feat(cli): add `setup cleanup` sub command

* chore: logging

* chore: logging
2023-04-28 11:55:35 +00:00
Tim Möhlmann
11ab645bb7 Merge branch 'main' into integration-tests 2023-04-27 12:47:35 +03:00
Silvan
86f4477ae1
fix(step10): separate executions (#5754)
* feat: allow skip of success page for native apps (#5627)

add possibility to return to callback directly after login without rendering the successful login page

* build next

* fix(console): disallow inline fonts, critical styles (#5714)

fix: disallow inline

* fix(setup): step 10 for postgres (#5717)

* fix(setup): smaller transactions (#5743)

* fix(step10): split statements

* fix(step10): split into separate execs

* chore: prerelease

* add truncate before insert

* fix: add truncate

* Merge branch 'main' into optimise-step-10

* chore: reset release definition

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Max Peintner <max@caos.ch>
2023-04-27 08:28:26 +00:00
Tim Möhlmann
90ba3a8d92 poll on test start 2023-04-26 19:54:47 +03:00
bitfactory-sem-denbroeder
19f2f83b61
feat: add otp name and make it configurable (#5631)
* feat: add otp name and make it configurable

* feat: use pre-existing otp env var

* feat: use requested domain if otp issuer is empty

* cleanup

---------

Co-authored-by: Sem den Broeder <semnelldenbroeder@gmail.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-04-26 07:17:23 +02:00