Tim Möhlmann
093dd57a78
fix(db): wrap BeginTx in spans to get acquire metrics ( #7689 )
...
feat(db): wrap BeginTx in spans to get acquire metrics
This changes adds a span around most db.BeginTx calls so we can get tracings about the connection pool acquire process.
This might help us pinpoint why sometimes some query package traces show longer execution times, while this was not reflected on database side execution times.
Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-04-03 11:48:24 +03:00
Silvan
56df515e5f
chore: use pgx v5 ( #7577 )
...
* chore: use pgx v5
* chore: update go version
* remove direct pq dependency
* remove unnecessary type
* scan test
* map scanner
* converter
* uint8 number array
* duration
* most unit tests work
* unit tests work
* chore: coverage
* go 1.21
* linting
* int64 gopfertammi
* retry go 1.22
* retry go 1.22
* revert to go v1.21.5
* update go toolchain to 1.21.8
* go 1.21.8
* remove test flag
* go 1.21.5
* linting
* update toolchain
* use correct array
* use correct array
* add byte array
* correct value
* correct error message
* go 1.21 compatible
2024-03-27 15:48:22 +02:00
Filip Sandborg
f0ba2b955a
fix: db MaxOpenConns setting ( #7360 )
...
* Fix pg MaxOpenConns setting
* fix: crdb open conns
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-02-09 13:43:01 +01:00
Silvan
8bc56f6fe7
fix(query): escape wildcards in text search ( #7131 ) ( #7135 )
...
* fix(query): escape like wildcards
* test: search query wildcards
* add do nothing
2024-01-02 16:27:36 +01:00
Tim Möhlmann
fe1337536f
fix(db): add additional connection pool for projection spooling ( #7094 )
...
* fix(db): add additional connection pool for projection spooling
* use correct connection pool for projections
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
2023-12-20 16:13:04 +00:00
Tim Möhlmann
f680dd934d
refactor: rename package errors to zerrors ( #7039 )
...
* chore: rename package errors to zerrors
* rename package errors to gerrors
* fix error related linting issues
* fix zitadel error assertion
* fix gosimple linting issues
* fix deprecated linting issues
* resolve gci linting issues
* fix import structure
---------
Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-08 15:30:55 +01:00
Tim Möhlmann
ec03340b67
perf(oidc): optimize client verification ( #6999 )
...
* fix some spelling errors
* client credential auth
* implementation of client auth
* improve error handling
* unit test command package
* unit test database package
* unit test query package
* cleanup unused tracing func
* fix integration tests
* errz to zerrors
* fix linting and import issues
* fix another linting error
* integration test with client secret
* Revert "integration test with client secret"
This reverts commit 0814ba522f
2023-12-05 17:01:03 +00:00
Elio Bischof
385a55bd21
feat: limit audit trail ( #6744 )
...
* feat: enable limiting audit trail
* support AddExclusiveQuery
* fix invalid condition
* register event mappers
* fix NullDuration validity
* test query side for limits
* lint
* acceptance test audit trail limit
* fix acceptance test
* translate limits not found
* update tests
* fix linting
* add audit log retention to default instance
* fix tests
* update docs
* remove todo
* improve test name
2023-10-25 11:42:00 +00:00
Silvan
4d4f649eda
fix(db): allow unlimited connections ( #6758 )
2023-10-19 13:37:22 +00:00
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities ( #5940 )
...
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005 ) and [06](https://zitadel.com/docs/support/advisory/a10006 ).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00
Elio Bischof
5823fdbef9
perf: project quotas and usages ( #6441 )
...
* project quota added
* project quota removed
* add periods table
* make log record generic
* accumulate usage
* query usage
* count action run seconds
* fix filter in ReportQuotaUsage
* fix existing tests
* fix logstore tests
* fix typo
* fix: add quota unit tests command side
* fix: add quota unit tests command side
* fix: add quota unit tests command side
* move notifications into debouncer and improve limit querying
* cleanup
* comment
* fix: add quota unit tests command side
* fix remaining quota usage query
* implement InmemLogStorage
* cleanup and linting
* improve test
* fix: add quota unit tests command side
* fix: add quota unit tests command side
* fix: add quota unit tests command side
* fix: add quota unit tests command side
* action notifications and fixes for notifications query
* revert console prefix
* fix: add quota unit tests command side
* fix: add quota integration tests
* improve accountable requests
* improve accountable requests
* fix: add quota integration tests
* fix: add quota integration tests
* fix: add quota integration tests
* comment
* remove ability to store logs in db and other changes requested from review
* changes requested from review
* changes requested from review
* Update internal/api/http/middleware/access_interceptor.go
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* tests: fix quotas integration tests
* improve incrementUsageStatement
* linting
* fix: delete e2e tests as intergation tests cover functionality
* Update internal/api/http/middleware/access_interceptor.go
Co-authored-by: Silvan <silvan.reusser@gmail.com>
* backup
* fix conflict
* create rc
* create prerelease
* remove issue release labeling
* fix tracing
---------
Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
(cherry picked from commit 1a49b7d298
2023-09-15 17:00:12 +02:00
Livio Spring
523dee8801
chore: remove postgres beta warning ( #6394 )
...
fix: remove postgres beta warning
Co-authored-by: Fabi <fabienne@zitadel.com>
2023-09-08 15:44:34 +00:00
Silvan
99e1c654a3
feat(storage): read only transactions for queries ( #6415 )
...
* fix: tests
* bastle wie en grosse
* fix(database): scan as callback
* fix tests
* fix merge failures
* remove as of system time
* refactor: remove unused test
* refacotr: remove unused lines
2023-08-22 10:49:22 +00:00
Fabian Haenel
c5c773531c
fix: Improve and sync checkSSL functions for CockroachDB and PostgreSQL ( #6271 )
...
* Improve and sync checkSSL functions for cockroach and postgres
* Add missing prefer mode
* Fix missing return in postgres checkSSL on disable
2023-08-14 13:51:33 +00:00
András Tóth
8c926366a9
fix(database): allow postgres sslmode=require without root cert ( #4972 )
...
* fix(database): allow postgres sslmode=require without root cert
* fix(database): allow postgres sslmode=require without root cert (fix)
Co-authored-by: Silvan <silvan.reusser@gmail.com>
---------
Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-05-23 14:29:13 +00:00
Livio Spring
c2cb84cd24
feat(api): new session service ( #5801 )
...
* backup new protoc plugin
* backup
* session
* backup
* initial implementation
* change to specific events
* implement tests
* cleanup
* refactor: use new protoc plugin for api v2
* change package
* simplify code
* cleanup
* cleanup
* fix merge
* start queries
* fix tests
* improve returned values
* add token to projection
* tests
* test db map
* update query
* permission checks
* fix tests and linting
* rework token creation
* i18n
* refactor token check and fix tests
* session to PB test
* request to query tests
* cleanup proto
* test user check
* add comment
* simplify database map type
* Update docs/docs/guides/integrate/access-zitadel-system-api.md
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
* fix test
* cleanup
* docs
---------
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2023-05-05 15:34:53 +00:00
Silvan
c8142030e9
fix(postgres): set dbname=postgres in admin mode ( #5346 )
2023-03-03 09:26:50 +00:00
Silvan
e38abdcdf3
perf: query data AS OF SYSTEM TIME ( #5231 )
...
Queries the data in the storage layser at the timestamp when the call hit the API layer
2023-02-27 22:36:43 +01:00
Silvan
fabecca4e4
fix(database): add MaxIdleConns configuration ( #4856 )
2022-12-08 15:22:39 +00:00
Stefan Benz
5d17da542d
fix(user): add search query for login name ( #4173 )
...
* fix(user): add search query for login name
* fix(user): change login name query to IN from EXISTS
* fix(loginname): include InQuery into ListQuery with SubSelect as possible datasource
* fix(user): apply suggestions from code review
Co-authored-by: Livio Spring <livio.a@gmail.com>
* fix: correct unit test for search query
Co-authored-by: Fabi <38692350+hifabienne@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2022-10-31 13:03:23 +00:00
Silvan
2f647ce9a2
fix: initial failures ( #4291 )
...
* fix(cmd): read configuration correctly
* fix(database): read weakly typed config
* fix(database): correct handling of update columns
Co-authored-by: Livio Spring <livio.a@gmail.com>
2022-09-01 07:24:26 +00:00
Silvan
77b4fc5487
feat(database): support for postgres ( #3998 )
...
* beginning with postgres statements
* try pgx
* use pgx
* database
* init works for postgres
* arrays working
* init for cockroach
* init
* start tests
* tests
* TESTS
* ch
* ch
* chore: use go 1.18
* read stmts
* fix typo
* tests
* connection string
* add missing error handler
* cleanup
* start all apis
* go mod tidy
* old update
* switch back to minute
* on conflict
* replace string slice with `database.StringArray` in db models
* fix tests and start
* update go version in dockerfile
* setup go
* clean up
* remove notification migration
* update
* docs: add deploy guide for postgres
* fix: revert sonyflake
* use `database.StringArray` for daos
* use `database.StringArray` every where
* new tables
* index naming,
metadata primary key,
project grant role key type
* docs(postgres): change to beta
* chore: correct compose
* fix(defaults): add empty postgres config
* refactor: remove unused code
* docs: add postgres to self hosted
* fix broken link
* so?
* change title
* add mdx to link
* fix stmt
* update goreleaser in test-code
* docs: improve postgres example
* update more projections
* fix: add beta log for postgres
* revert index name change
* prerelease
* fix: add sequence to v1 "reduce paniced"
* log if nil
* add logging
* fix: log output
* fix(import): check if org exists and user
* refactor: imports
* fix(user): ignore malformed events
* refactor: method naming
* fix: test
* refactor: correct errors.Is call
* ci: don't build dev binaries on main
* fix(go releaser): update version to 1.11.0
* fix(user): projection should not break
* fix(user): handle error properly
* docs: correct config example
* Update .releaserc.js
* Update .releaserc.js
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
2022-08-31 07:52:43 +00:00
Livio Spring
f610d48569
feat: prepare for multiple database types ( #4068 )
...
BREAKING CHANGE: the database and admin user config has changed.
2022-07-28 16:25:42 +02:00
Florian Forster
fa9f581d56
chore(v2): move to new org ( #3499 )
...
* chore: move to new org
* logging
* fix: org rename caos -> zitadel
Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2022-04-26 23:01:45 +00:00
Silvan
c5b99274d7
feat(cli): setup ( #3267 )
...
* commander
* commander
* selber!
* move to packages
* fix(errors): implement Is interface
* test: command
* test: commands
* add init steps
* setup tenant
* add default step yaml
* possibility to set password
* merge v2 into v2-commander
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: rename iam command side to instance
* fix: search query builder can filter events in memory
* fix: filters for add member
* fix(setup): add `ExternalSecure` to config
* chore: name iam to instance
* fix: matching
* remove unsued func
* base url
* base url
* test(command): filter funcs
* test: commands
* fix: rename orgiampolicy to domain policy
* start from init
* commands
* config
* fix indexes and add constraints
* fixes
* fix: merge conflicts
* fix: protos
* fix: md files
* setup
* add deprecated org iam policy again
* typo
* fix search query
* fix filter
* Apply suggestions from code review
* remove custom org from org setup
* add todos for verification
* change apps creation
* simplify package structure
* fix error
* move preparation helper for tests
* fix unique constraints
* fix config mapping in setup
* fix error handling in encryption_keys.go
* fix projection config
* fix query from old views to projection
* fix setup of mgmt api
* set iam project and fix instance projection
* imports
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
2022-03-28 10:05:09 +02:00
Silvan
4272ea6fe1
fix: init sub commands ( #3218 )
...
* fix(init): add sub commands
* fix(init): admin user in config,
test(init): verify functions
* refactor: config, remove second commands
* refactor: init steps
* chore: fix link in readme
* chore: numerate sql files
* Update cmd/admin/initialise/sql/README.md
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* Update cmd/admin/initialise/sql/README.md
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
* fix(init): remove unused index
* user
* fix database username in defaults.yaml
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2022-02-16 12:30:49 +00:00
Livio Amstutz
389eb4a27a
feat: run on a single port ( #3163 )
...
* start v2
* start
* run
* some cleanup
* remove v2 pkg again
* simplify
* webauthn
* remove unused config
* fix login path in Dockerfile
* fix asset_generator.go
* health handler
* fix grpc web
* refactor
* merge
* build new main.go
* run new main.go
* update logging pkg
* fix error msg
* update logging
* cleanup
* cleanup
* go mod tidy
* change localDevMode
* fix customEndpoints
* update logging
* comments
* change local flag to external configs
* fix location generated go code
* fix
Co-authored-by: fforootd <florian@caos.ch>
2022-02-14 17:22:30 +01:00
Silvan
b44b48fa1e
fix(init): flags ( #3192 )
2022-02-11 11:52:50 +01:00
Silvan
e8ab237ada
fix(init): prepare database ( #3191 )
...
* fix(init): prepare database
* fix(defaults): cockroach local defaults
2022-02-11 11:02:47 +01:00
Silvan
9d471d0d30
feat(cli): init cli ( #3186 )
...
* feat(cli): initilize cli
* fix(config): allow multiple files
* refactor(cli): constructor naming
* go mod tidy
* refactor: move code out of v2 package
* chore: logging v0.1
* chore: remove old gitignore
* fix func
Co-authored-by: Livio Amstutz <livio.a@gmail.com>
2022-02-09 15:01:19 +01:00
