zitadel/cmd/setup
Silvan 2243306ef6
feat(cmd): mirror (#7004)
# Which Problems Are Solved

Adds the possibility to mirror an existing database to a new one. 

For that a new command was added `zitadel mirror`. Including it's
subcommands for a more fine grained mirror of the data.

Sub commands:

* `zitadel mirror eventstore`: copies only events and their unique
constraints
* `zitadel mirror system`: mirrors the data of the `system`-schema
*  `zitadel mirror projections`: runs all projections
*  `zitadel mirror auth`: copies auth requests
* `zitadel mirror verify`: counts the amount of rows in the source and
destination database and prints the diff.

The command requires one of the following flags:
* `--system`: copies all instances of the system
* `--instance <instance-id>`, `--instance <comma separated list of
instance ids>`: copies only the defined instances

The command is save to execute multiple times by adding the
`--replace`-flag. This replaces currently existing data except of the
`events`-table

# Additional Changes

A `--for-mirror`-flag was added to `zitadel setup` to prepare the new
database. The flag skips the creation of the first instances and initial
run of projections.

It is now possible to skip the creation of the first instance during
setup by setting `FirstInstance.Skip` to true in the steps
configuration.

# Additional info

It is currently not possible to merge multiple databases. See
https://github.com/zitadel/zitadel/issues/7964 for more details.

It is currently not possible to use files. See
https://github.com/zitadel/zitadel/issues/7966 for more information.

closes https://github.com/zitadel/zitadel/issues/7586
closes https://github.com/zitadel/zitadel/issues/7486

### Definition of Ready

- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [x] Acceptance criteria are met
- [x] All open todos and follow ups are defined in a new ticket and
justified
- [x] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [x] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [x] Documentation/examples are up-to-date
- [x] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2024-05-30 09:35:30 +00:00
..
01_sql feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
06 fix(setup): create indexes only if not exists (#4801) 2022-12-01 14:31:01 +01:00
07 fix(init): correct quoting of database and user (#6928) 2023-12-03 08:30:08 +00:00
08 fix(setup): optimise events indexes (#5316) 2023-03-01 00:11:23 +00:00
10 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
12 feat(login): add OTP (email and sms) (#6353) 2023-08-15 12:47:05 +00:00
13 fix: set quotas (#6597) 2023-09-22 13:06:59 +02:00
14 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
15 feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
01.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
02.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
03.go feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
05.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
05.sql fix(database): create column only if not exists (#4812) 2022-12-02 10:48:39 +00:00
06.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
07.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
08.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
10.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
12.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
13.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
14.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
15.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
16.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
16.sql fix: ensure uniqueness (#6956) 2023-11-22 12:05:14 +00:00
17.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
17.sql fix(eventstore): improve pagination of handler filter (#6968) 2023-12-01 12:25:41 +00:00
18.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
18.sql fix(query): user performance (#6537) 2023-12-08 13:14:22 +01:00
19.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
19.sql fix(eventstore): increase performance on push (#7125) 2023-12-31 15:30:25 +01:00
20.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
20.sql fix(auth): efficient user session projection (#7187) 2024-01-09 18:36:46 +00:00
21.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
21.sql feat: block instances (#7129) 2024-01-17 10:16:48 +00:00
22.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
22.sql fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
23.go fix(eventstore): consider IsGlobal-flag of constraints (#7518) 2024-03-08 13:33:53 +00:00
23.sql fix(eventstore): consider IsGlobal-flag of constraints (#7518) 2024-03-08 13:33:53 +00:00
24.go feat(oidc): token exchange impersonation (#7516) 2024-03-20 10:18:46 +00:00
24.sql feat(oidc): token exchange impersonation (#7516) 2024-03-20 10:18:46 +00:00
25.go feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
25.sql feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
26.go fix(login): improve auth handlers (#7969) 2024-05-22 15:26:02 +00:00
26.sql fix(login): improve auth handlers (#7969) 2024-05-22 15:26:02 +00:00
27.go feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
27.sql feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
cleanup.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
config_change.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
config_test.go fix(setup): decode complex config strings (#7854) 2024-05-01 12:17:27 +02:00
config.go feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
projections.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
setup.go feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
steps.yaml feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00