zitadel/internal/query
Tim Möhlmann 7967e6f98b
perf(import): optimize search for domains claimed by other organizations (#8200)
# Which Problems Are Solved

Improve the performance of human imports by optimizing the query that
finds domains claimed by other organizations.

# How the Problems Are Solved

Use the fields search table introduced in
https://github.com/zitadel/zitadel/pull/8191 by storing each
organization domain as Object ID and the verified status as field value.

# Additional Changes

- Feature flag for this optimization

# Additional Context

- Performance improvements for import are evaluated and acted upon
internally at the moment

---------

Co-authored-by: adlerhurst <silvan.reusser@gmail.com>
2024-07-05 09:36:00 +02:00
..
projection perf(import): optimize search for domains claimed by other organizations (#8200) 2024-07-05 09:36:00 +02:00
testdata feat(6222): remove @ and project from OIDC client ID (#8178) 2024-07-04 08:31:40 +00:00
access_token.go perf(authZ): improve oidc session check (#8091) 2024-06-12 09:11:36 +00:00
action_flow_test.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00
action_flow.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
action_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
action.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
app_test.go feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
app.go fix(query): reduce app query overhead (#7817) 2024-04-22 11:30:56 +02:00
auth_request_by_id.sql feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
auth_request_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
auth_request.go feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
authn_key_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
authn_key.go perf(oidc): optimize token creation (#7822) 2024-05-16 07:07:56 +02:00
certificate_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
certificate.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
converter.go feat(api): feature flags (#7356) 2024-02-28 10:55:54 +02:00
current_state_test.go feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
current_state.go fix(db): wrap BeginTx in spans to get acquire metrics (#7689) 2024-04-03 11:48:24 +03:00
custom_text_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
custom_text.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
device_auth_test.go perf(oidc): optimize token creation (#7822) 2024-05-16 07:07:56 +02:00
device_auth.go perf(oidc): optimize token creation (#7822) 2024-05-16 07:07:56 +02:00
domain_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
domain_policy.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
event.go chore: remove bloating span (#7780) 2024-04-16 11:19:17 +00:00
execution_targets.sql feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
execution_test.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
execution.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
failed_events_test.go feat(eventstore): increase parallel write capabilities (#5940) 2023-10-19 12:19:10 +02:00
failed_events.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
generic.go feat: query side for executions and targets for actions v2 (#7524) 2024-03-14 09:56:23 +00:00
iam_member_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
iam_member.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_login_policy_link_test.go feat(idp): provide option to auto link user (#7734) 2024-04-10 15:46:30 +00:00
idp_login_policy_link.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_template_test.go feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
idp_template.go feat(saml): allow setting nameid-format and alternative mapping for transient format (#7979) 2024-05-23 05:04:07 +00:00
idp_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp_user_link_test.go feat(idp): provide option to auto link user (#7734) 2024-04-10 15:46:30 +00:00
idp_user_link.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
idp.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
instance_by_domain.sql fix(query): optimize instance by domain query (#7513) 2024-03-06 18:02:16 +00:00
instance_by_id.sql fix: assign instance ID to aggregate ID when converting from v1 to v2 feature (#7505) 2024-03-05 16:12:49 +01:00
instance_domain_test.go feat(storage): read only transactions for queries (#6415) 2023-08-22 10:49:22 +00:00
instance_domain.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
instance_features_model.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
instance_features_test.go fix: add action v2 execution to features (#7597) 2024-04-09 20:21:21 +03:00
instance_features.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
instance_test.go feat(api): feature flags (#7356) 2024-02-28 10:55:54 +02:00
instance.go feat: improve instance not found error (#7413) 2024-02-28 10:49:57 +00:00
introspection_client_by_id.sql feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
introspection_test.go feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
introspection.go feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
key_test.go fix(oidc): ignore public key expiry for ID Token hints (#7293) 2024-01-29 15:11:52 +00:00
key.go fix(oidc): ignore public key expiry for ID Token hints (#7293) 2024-01-29 15:11:52 +00:00
label_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
lockout_policy_test.go feat: provide option to limit (T)OTP checks (#7693) 2024-04-10 09:14:55 +00:00
lockout_policy.go feat: provide option to limit (T)OTP checks (#7693) 2024-04-10 09:14:55 +00:00
login_name.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00
login_policy_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
login_policy.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
mail_template.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
member_roles.go fix(query): realtime data on defined requests (#3726) 2022-06-14 07:51:00 +02:00
member.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
message_text_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
message_text.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
milestone_test.go feat(storage): read only transactions for queries (#6415) 2023-08-22 10:49:22 +00:00
milestone.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
notification_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
notification_policy.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
notification_provider_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
notification_provider.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
oidc_client_by_id.sql feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
oidc_client_test.go feat(6222): remove @ and project from OIDC client ID (#8178) 2024-07-04 08:31:40 +00:00
oidc_client.go feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
oidc_settings_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
oidc_settings.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_domain_test.go feat(storage): read only transactions for queries (#6415) 2023-08-22 10:49:22 +00:00
org_domain.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_member_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
org_member.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_metadata_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
org_metadata.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
org_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
org.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
password_age_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_age_policy.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
password_complexity_policy_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
password_complexity_policy.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
prepare_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
privacy_policy_test.go feat(cnsl): docs link can be customized and custom button is available (#7840) 2024-05-13 16:01:50 +02:00
privacy_policy.go feat(cnsl): docs link can be customized and custom button is available (#7840) 2024-05-13 16:01:50 +02:00
project_grant_member_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
project_grant_member.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_grant_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_grant.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
project_member_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
project_member.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project_role_test.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00
project_role.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
project_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
project.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
query_test.go fix(setup): init projections (#7194) 2024-01-25 17:28:20 +01:00
query.go feat(cmd): mirror (#7004) 2024-05-30 09:35:30 +00:00
quota_notifications_test.go perf: project quotas and usages (#6441) 2023-09-15 16:58:45 +02:00
quota_notifications.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
quota_periods_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
quota_periods.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
quota_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
quota.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
restrictions_test.go fix: projection version of restrictions (#7028) 2023-12-06 10:30:56 +00:00
restrictions.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
search_query_test.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
search_query.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
secret_generator_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
secret_generators.go fix(import): add tracing spans to all import related functions (#8160) 2024-06-19 12:56:33 +02:00
security_policy.go feat: impersonation roles (#7442) 2024-02-28 10:21:11 +00:00
session.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
sessions_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
sms_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
sms.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
smtp_test.go feat: SMTP Templates (#6932) 2024-04-11 09:16:10 +02:00
smtp.go feat: SMTP Templates (#6932) 2024-04-11 09:16:10 +02:00
system_features_model.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
system_features_test.go fix: add action v2 execution to features (#7597) 2024-04-09 20:21:21 +03:00
system_features.go refactor(query): use new packages for org by id query (#7826) 2024-05-24 13:32:57 +02:00
target_test.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
target.go feat: add action v2 execution on requests and responses (#7637) 2024-05-04 11:55:57 +02:00
targets_by_execution_id.sql fix(actions): correct statements to query targets (#8006) 2024-05-24 14:13:36 +00:00
targets_by_execution_ids.sql fix(actions): correct statements to query targets (#8006) 2024-05-24 14:13:36 +00:00
user_auth_method_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_auth_method.go fix(oidc): remove MFA requirement on ZITADEL API based on user auth methods (#8069) 2024-06-12 12:24:17 +00:00
user_by_id.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_by_login_name.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_grant_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_grant.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
user_membership_test.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00
user_membership.go fix: remove non-existent colums in getMembershipFromQuery queries (#7869) 2024-07-02 12:08:28 +00:00
user_metadata_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
user_metadata.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
user_notify_by_id.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_notify_by_login_name.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user_otp.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
user_password.go feat(crypto): use passwap for machine and app secrets (#7657) 2024-04-05 09:35:49 +00:00
user_personal_access_token_test.go refactor: rename package errors to zerrors (#7039) 2023-12-08 15:30:55 +01:00
user_personal_access_token.go refactor(fmt): run gci on complete project (#7557) 2024-04-03 10:43:43 +00:00
user_schema_test.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
user_schema.go chore: use pgx v5 (#7577) 2024-03-27 15:48:22 +02:00
user_test.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
user.go feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
userinfo_by_id.sql feat: password age policy (#8132) 2024-06-18 11:27:44 +00:00
userinfo_client_by_id.sql feat(oidc): optimize the userinfo endpoint (#7706) 2024-04-09 15:15:35 +02:00
userinfo_test.go feat(oidc): organization roles scope (#8120) 2024-06-14 10:00:43 +02:00
userinfo.go feat(oidc): organization roles scope (#8120) 2024-06-14 10:00:43 +02:00
zitadel_permission.go perf: remove owner removed columns from projections for oidc (#6925) 2023-11-20 17:21:08 +02:00