zitadel/internal/query/projection
Silvan 1d84635836
feat(eventstore): add search table (#8191)
# Which Problems Are Solved

To improve performance a new table and method is implemented on
eventstore. The goal of this table is to index searchable fields on
command side to use it on command and query side.

The table allows to store one primitive value (numeric, text) per row.

The eventstore framework is extended by the `Search`-method which allows
to search for objects.
The `Command`-interface is extended by the `SearchOperations()`-method
which does manipulate the the `search`-table.

# How the Problems Are Solved

This PR adds the capability of improving performance for command and
query side by using the `Search`-method of the eventstore instead of
using one of the `Filter`-methods.

# Open Tasks

- [x] Add feature flag
- [x] Unit tests
- [ ] ~~Benchmarks if needed~~
- [x] Ensure no behavior change
- [x] Add setup step to fill table with current data
- [x] Add projection which ensures data added between setup and start of
the new version are also added to the table

# Additional Changes

The `Search`-method is currently used by `ProjectGrant`-command side.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/8094
2024-07-03 15:00:56 +00:00
..
action_test.go feat: query side for executions and targets for actions v2 (#7524) 2024-03-14 09:56:23 +00:00
action.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
app_test.go feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
app.go feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
assert_test.go feat: push telemetry (#6027) 2023-07-06 08:38:13 +02:00
assert.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
auth_request_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
auth_request.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
authn_key_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
authn_key.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
config.go feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
custom_text_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
custom_text.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
debug_notification_provider_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
debug_notification.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
device_auth.go perf(oidc): optimize token creation (#7822) 2024-05-16 07:07:56 +02:00
domain_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
domain_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
event_test.go feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
eventstore_field.go feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
eventstore_mock_test.go feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
executer_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
execution_test.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
execution.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
flow_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
flow.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_login_policy_link_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_login_policy_link.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_template_test.go feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
idp_template.go feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
idp_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_user_link_test.go fix: update external username on idp if auto update is enabled (#7048) 2023-12-08 18:22:07 +01:00
idp_user_link.go fix: update external username on idp if auto update is enabled (#7048) 2023-12-08 18:22:07 +01:00
idp.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
instance_domain_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
instance_domain.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
instance_features_test.go fix: assign instance ID to aggregate ID when converting from v1 to v2 feature (#7505) 2024-03-05 16:12:49 +01:00
instance_features.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
instance_member_test.go fix: combine resourceowner query in reduce function for user grant (#7383) 2024-03-08 07:52:59 +00:00
instance_member.go perf: user grant owner removed (#6962) 2024-01-08 15:26:30 +00:00
instance_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
instance.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
key_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
key.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
label_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
label_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
limits_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
limits.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
lockout_policy_test.go feat: provide option to limit (T)OTP checks (#7693) 2024-04-10 09:14:55 +00:00
lockout_policy.go feat: provide option to limit (T)OTP checks (#7693) 2024-04-10 09:14:55 +00:00
login_name_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
login_name.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
login_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
login_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
mail_template_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
mail_template.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
main_test.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
member.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00
message_text_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
message_texts.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
milestones_test.go fix: consider oidc session events for authN milestones (#8089) 2024-06-12 06:49:14 +02:00
milestones.go fix: consider oidc session events for authN milestones (#8089) 2024-06-12 06:49:14 +02:00
notification_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
notification_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
oidc_settings_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
oidc_settings.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_domain_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_domain.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_member_test.go fix: combine resourceowner query in reduce function for user grant (#7383) 2024-03-08 07:52:59 +00:00
org_member.go perf: user grant owner removed (#6962) 2024-01-08 15:26:30 +00:00
org_metadata_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
org_metadata.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
org_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_age_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_age_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_complexity_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_complexity_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
privacy_policy_test.go feat(cnsl): docs link can be customized and custom button is available (#7840) 2024-05-13 16:01:50 +02:00
privacy_policy.go feat(cnsl): docs link can be customized and custom button is available (#7840) 2024-05-13 16:01:50 +02:00
project_grant_member_test.go fix: combine resourceowner query in reduce function for user grant (#7383) 2024-03-08 07:52:59 +00:00
project_grant_member.go fix: combine resourceowner query in reduce function for user grant (#7383) 2024-03-08 07:52:59 +00:00
project_grant_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_grant.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_member_test.go fix: combine resourceowner query in reduce function for user grant (#7383) 2024-03-08 07:52:59 +00:00
project_member.go perf: user grant owner removed (#6962) 2024-01-08 15:26:30 +00:00
project_role_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_role.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
projection.go feat(eventstore): add search table (#8191) 2024-07-03 15:00:56 +00:00
quota_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
quota.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
restrictions_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
restrictions.go fix: projection version of restrictions (#7028) 2023-12-06 10:30:56 +00:00
secret_generator_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
secret_generator.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
security_policy.go feat: impersonation roles (#7442) 2024-02-28 10:21:11 +00:00
session_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
session.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
sms_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
sms.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
smtp_test.go feat: SMTP Templates (#6932) 2024-04-11 09:16:10 +02:00
smtp.go feat: SMTP Templates (#6932) 2024-04-11 09:16:10 +02:00
system_features_test.go feat(api): feature flags (#7356) 2024-02-28 10:55:54 +02:00
system_features.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
target_test.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
target.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
user_auth_method_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
user_auth_method.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
user_grant_test.go fix: fill resourceowner of project into usergrant projection (#7605) 2024-03-21 10:31:06 +00:00
user_grant.go fix: fill resourceowner of project into usergrant projection (#7605) 2024-03-21 10:31:06 +00:00
user_metadata_test.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
user_metadata.go fix(projections): stop updating creation dates (#6930) 2024-02-22 17:25:47 +00:00
user_personal_access_token_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
user_personal_access_token.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
user_schema_test.go feat(api): add possibility to retrieve user schemas (#7614) 2024-03-22 13:26:13 +00:00
user_schema.go feat(api): add possibility to retrieve user schemas (#7614) 2024-03-22 13:26:13 +00:00
user_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00