Commit Graph

17 Commits

Author SHA1 Message Date
Tim Möhlmann
aeb379e7de
fix(eventstore): revert precise decimal (#8527) (#8679) 2024-09-24 18:43:29 +02:00
Silvan
b522588d98
fix(eventstore): precise decimal (#8527)
# Which Problems Are Solved

Float64 which was used for the event.Position field is [not precise in
go and gets rounded](https://github.com/golang/go/issues/47300). This
can lead to unprecies position tracking of events and therefore
projections especially on cockcoachdb as the position used there is a
big number.

example of a unprecies position:
exact: 1725257931223002628
float64: 1725257931223002624.000000

# How the Problems Are Solved

The float64 was replaced by
[github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal).

# Additional Changes

Correct behaviour of makefile for load tests.
Rename `latestSequence`-queries to `latestPosition`
2024-09-06 12:19:19 +03:00
Silvan
523d73f674
fix(fields): use read commit isolation level in trigger (#8410)
# Which Problems Are Solved

If the processing time of serializable transactions in the fields
handler take too long, the next iteration can fail.

# How the Problems Are Solved

Changed the isolation level of the current states query to Read Commited
2024-08-09 09:24:28 +00:00
Silvan
99c645cc60
refactor(database): exchange connection pool (#8325)
# Which Problems Are Solved

The connection pool of go uses a high amount of database connections.

# How the Problems Are Solved

The standard lib connection pool was replaced by `pgxpool.Pool`

# Additional Changes

The `db.BeginTx`-spans are removed because they cause to much noise in
the traces.

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/7639
2024-07-17 15:16:02 +00:00
Silvan
1d84635836
feat(eventstore): add search table (#8191)
# Which Problems Are Solved

To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.

The table allows to store one primitive value (numeric, text) per row.

The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.

# How the Problems Are Solved

This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.

# Open Tasks

- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table

# Additional Changes

The `Search`-method is currently used by `ProjectGrant`-command side.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8094
2024-07-03 15:00:56 +00:00
Tim Möhlmann
029a6d393a
fix(crdb): obtain latest sequences when the tx is retried (#7795) 2024-04-18 13:07:05 +00:00
Tim Möhlmann
093dd57a78
fix(db): wrap BeginTx in spans to get acquire metrics (#7689)
feat(db): wrap BeginTx in spans to get acquire metrics

This changes adds a span around most db.BeginTx calls so we can get tracings about the connection pool acquire process.
This might help us pinpoint why sometimes some query package traces show longer execution times, while this was not reflected on database side execution times.

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2024-04-03 11:48:24 +03:00
Silvan
56df515e5f
chore: use pgx v5 (#7577)
* chore: use pgx v5

* chore: update go version

* remove direct pq dependency

* remove unnecessary type

* scan test

* map scanner

* converter

* uint8 number array

* duration

* most unit tests work

* unit tests work

* chore: coverage

* go 1.21

* linting

* int64 gopfertammi

* retry go 1.22

* retry go 1.22

* revert to go v1.21.5

* update go toolchain to 1.21.8

* go 1.21.8

* remove test flag

* go 1.21.5

* linting

* update toolchain

* use correct array

* use correct array

* add byte array

* correct value

* correct error message

* go 1.21 compatible
2024-03-27 15:48:22 +02:00
Silvan
60ee2610f2
fix(eventstore): consider IsGlobal-flag of constraints (#7518)
* fix(eventstore): consider `IsGlobal`-flag of constraints

* fix(setup): set `instance_domain`-constraint global
2024-03-08 13:33:53 +00:00
Silvan
b7d027e2fd
fix(db): always use begin tx (#7142)
* fix(db): always use begin tx

* fix(handler): timeout for begin
2024-01-04 16:12:20 +00:00
Silvan
cc2dd8b20b
fix(eventstore): increase performance on push (#7125) 2023-12-31 15:30:25 +01:00
Tim Möhlmann
f680dd934d
refactor: rename package errors to zerrors (#7039)
* chore: rename package errors to zerrors

* rename package errors to gerrors

* fix error related linting issues

* fix zitadel error assertion

* fix gosimple linting issues

* fix deprecated linting issues

* resolve gci linting issues

* fix import structure

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
2023-12-08 15:30:55 +01:00
Elio Bischof
8982e1aae3
fix(postgres <=15): delete unique constraints (#6971)
fix(postgres): delete unique constraints
2023-11-24 07:23:23 +01:00
Silvan
9ed956383f
fix(eventstore): correct handling of wrong unique fields (#6961) 2023-11-23 06:15:40 +01:00
Livio Spring
b563041103
fix: ensure uniqueness (#6956)
* fix: ensure uniqueness

* only update wrong ones

* Update cmd/setup/16.go

Co-authored-by: Silvan <silvan.reusser@gmail.com>

---------

Co-authored-by: Silvan <silvan.reusser@gmail.com>
2023-11-22 12:05:14 +00:00
Silvan
f8bf8ea256
fix(eventstore): differentiate unique constraint error (#6832)
* fix(eventstore): differentiate unique constraint error format

* docs: add comment to eventstore vars

* fix(eventstore): return correct error type if unique constraint already exists
2023-10-27 14:10:01 +02:00
Silvan
b5564572bc
feat(eventstore): increase parallel write capabilities (#5940)
This implementation increases parallel write capabilities of the eventstore.
Please have a look at the technical advisories: [05](https://zitadel.com/docs/support/advisory/a10005) and  [06](https://zitadel.com/docs/support/advisory/a10006).
The implementation of eventstore.push is rewritten and stored events are migrated to a new table `eventstore.events2`.
If you are using cockroach: make sure that the database user of ZITADEL has `VIEWACTIVITY` grant. This is used to query events.
2023-10-19 12:19:10 +02:00