Commit Graph

3796 Commits

Author SHA1 Message Date
Stefan Benz
8fdd58f506 docs: add actions v2 docs (#9652)
# Which Problems Are Solved

There is no documentation yet for Actions v2 regarding the different
usable types and for migrating from Actions v1 to v2.

# How the Problems Are Solved

Add documentation for all use-cases.

# Additional Changes

None

# Additional Context

Closes #9456

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Ramon <mail@conblem.me>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Kenta Yamaguchi <56732734+KEY60228@users.noreply.github.com>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Livio Spring <livio@zitadel.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Iraq <66622793+kkrime@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2025-04-09 13:29:58 +02:00
Livio Spring
a7a5124643 fix(mirror): initialize meter to prevent panic (#9712)
# Which Problems Are Solved

With the change of #9561, the `mirror` command panics as there's no
metrics provider configured.

# How the Problems Are Solved

Correctly initialize the provider (no-op by default) for the mirror
command.

# Additional Changes

None

# Additional Context

relates to #9561 -> needs backports to 2.66.x - 2.71.x and 3.0.0-rc

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2025-04-07 16:00:15 +00:00
Livio Spring
79a5585f91 fix(login): handle requests without auth request correctly (#9713)
# Which Problems Are Solved

We found some paths in the login UI, where requests without any
`AuthRequest` were not handled correctly and could potentially panic.
This also includes providing the `AuthRequest` as part of `ctx` object
in actions V1.

# How the Problems Are Solved

- Check for the existance of an `AuthRequest` were needed and return an
error otherwise.
- Provide correct state of the `AuthRequest` for actions V1

# Additional Changes

None

# Additional Context

- Noticed as part of a support request
- requires backport to at least 2.70.x

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2025-04-07 17:40:42 +02:00
Emanuel Vintilă
7a5d1d67ba docs: fix typo in default-settings.mdx (#9719)
# Which Problems Are Solved

Typo in `default-settings.mdx`

# How the Problems Are Solved

Corrects the typo 🙂

# Additional Changes

None

# Additional Context

None
2025-04-07 12:14:44 +03:00
dependabot[bot]
c815ff2a35 chore(deps): bump image-size from 1.1.1 to 1.2.1 in /docs (#9700)
Bumps [image-size](https://github.com/image-size/image-size) from 1.1.1
to 1.2.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/image-size/image-size/releases">image-size's
releases</a>.</em></p>
<blockquote>
<h2>v1.2.1</h2>
<h2>Fixes</h2>
<ul>
<li>fix potential Denial of Service via specially crafted payloads in <a
href="640a67d9e8</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/image-size/image-size/compare/v1.2.0...v1.2.1">https://github.com/image-size/image-size/compare/v1.2.0...v1.2.1</a></p>
<h2>v1.2.0</h2>
<p>This release adds support for JPEG-XL ( <a
href="https://redirect.github.com/image-size/image-size/issues/409">#409</a>
)</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="a4178fbb33"><code>a4178fb</code></a>
1.2.1</li>
<li><a
href="640a67d9e8"><code>640a67d</code></a>
fix potential Denial of Service via specially crafted payloads</li>
<li><a
href="9d41448d78"><code>9d41448</code></a>
1.2.0</li>
<li><a
href="405a244dae"><code>405a244</code></a>
fixups</li>
<li><a
href="76c5c9a8aa"><code>76c5c9a</code></a>
mention jpeg-xl in the readme</li>
<li><a
href="a10262c7c3"><code>a10262c</code></a>
Add support for JPEG XL (<a
href="https://redirect.github.com/image-size/image-size/issues/409">#409</a>)</li>
<li><a
href="a7a24a3fc4"><code>a7a24a3</code></a>
(app): Fix typo in comments (<a
href="https://redirect.github.com/image-size/image-size/issues/411">#411</a>)</li>
<li><a
href="9f482134b3"><code>9f48213</code></a>
update dependencies, and reformat code with eslint 9</li>
<li><a
href="64dda84cca"><code>64dda84</code></a>
refactor formats that use a ISO-BMFF container</li>
<li><a
href="e3ea53801d"><code>e3ea538</code></a>
no need to create hex strings in j2c</li>
<li>Additional commits viewable in <a
href="https://github.com/image-size/image-size/compare/v1.1.1...v1.2.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=image-size&package-manager=npm_and_yarn&previous-version=1.1.1&new-version=1.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/zitadel/zitadel/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-04-03 10:31:27 +01:00
Fabienne Bühler
07ce3b6905 chore!: Introduce ZITADEL v3 (#9645)
This PR summarizes multiple changes specifically only available with
ZITADEL v3:

- feat: Web Keys management
(https://github.com/zitadel/zitadel/pull/9526)
- fix(cmd): ensure proper working of mirror
(https://github.com/zitadel/zitadel/pull/9509)
- feat(Authz): system user support for permission check v2
(https://github.com/zitadel/zitadel/pull/9640)
- chore(license): change from Apache to AGPL
(https://github.com/zitadel/zitadel/pull/9597)
- feat(console): list v2 sessions
(https://github.com/zitadel/zitadel/pull/9539)
- fix(console): add loginV2 feature flag
(https://github.com/zitadel/zitadel/pull/9682)
- fix(feature flags): allow reading "own" flags
(https://github.com/zitadel/zitadel/pull/9649)
- feat(console): add Actions V2 UI
(https://github.com/zitadel/zitadel/pull/9591)

BREAKING CHANGE
- feat(webkey): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9445)
- chore!: remove CockroachDB Support
(https://github.com/zitadel/zitadel/pull/9444)
- feat(actions): migrate to v2beta API
(https://github.com/zitadel/zitadel/pull/9489)

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Ramon <mail@conblem.me>
Co-authored-by: Elio Bischof <elio@zitadel.com>
Co-authored-by: Kenta Yamaguchi <56732734+KEY60228@users.noreply.github.com>
Co-authored-by: Harsha Reddy <harsha.reddy@klaviyo.com>
Co-authored-by: Livio Spring <livio@zitadel.com>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Iraq <66622793+kkrime@users.noreply.github.com>
Co-authored-by: Florian Forster <florian@zitadel.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Max Peintner <peintnerm@gmail.com>
2025-04-02 16:53:06 +02:00
Elio Bischof
d14a23ae7e chore(deps): group easily mergable dependabot PRs (#9689)
# Which Problems Are Solved

Dependabot creates a PR for each individual dependency update. This is
noisy and toil.

# How the Problems Are Solved

The easily mergable updates are grouped into fewer PRs.
Major updates and security updates are not grouped, as they might need
special attention.
For the github-actions updates, also the major updates are grouped, as
we have immediate and good enough feedback from the pipeline if a new
major doesn't work.

# Additional Changes

The e2e dependencies are also auto-updated.

# Additional Context

- We aggreed on grouping the dependabot PRs in todays Scrum daily.
- I don't know how to test the config. We might just have to merge it
and look how it goes 🤷‍♂️
2025-04-02 15:57:47 +03:00
Livio Spring
315503beab Merge commit from fork 2025-03-31 12:45:11 +02:00
Livio Spring
14de8ecac2 Merge commit from fork 2025-03-31 12:44:49 +02:00
Stefan Benz
2eb187f141 fix(migration): check if ldap2 already exists (#9674)
# Which Problems Are Solved

With v2.71.0 the `idp_templates6_ldap3` projection was created but never
filled, as it was a subtable. To fix this we altered the
`idp_templates6_ldap3` to `idp_templates6_ldap2` with v2.71.5.
This was unfortunately without a check that the `idp_templates_ldap2`was
already existing, which resulted in an error in the setup step.

# How the Problems Are Solved

Add check if `idp_templates6_ldap2` is already existing, before renaming
`idp_templates6_ldap3` -> `idp_templates6_ldap2`.

# Additional Changes

None

# Additional Context

Closes #9669
2025-03-31 10:06:40 +00:00
Silvan
817670f1f7 perf: improve scalability of session api (#9635)
This pull request improves the scalability of the session API by
enhancing middleware tracing and refining SQL query behavior for user
authentication methods.

# Which Problems Are Solved

- Eventstore subscriptions locked each other during they wrote the
events to the event channels of the subscribers in push.
- `ListUserAuthMethodTypesRequired` query used `Bitmap heap scan` to
join the tables needed.
- The auth and oidc package triggered projections often when data were
read.
- The session API triggered the user projection each time a user was
searched to write the user check command.

# How the Problems Are Solved

- the `sync.Mutex` was replaced with `sync.RWMutex` to allow parallel
read of the map
- The query was refactored to use index scans only
- if the data should already be up-to-date `shouldTriggerBulk` is set to
false
- as the user should already exist for some time the trigger was
removed.

# Additional Changes

- refactoring of `tracing#Span.End` calls

# Additional Context

- part of https://github.com/zitadel/zitadel/issues/9239

---------

Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
2025-03-28 12:36:05 +00:00
dependabot[bot]
79d1e7d434 chore(deps): bump axios from 1.7.4 to 1.8.3 in /e2e (#9572)
Bumps [axios](https://github.com/axios/axios) from 1.7.4 to 1.8.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/axios/axios/releases">axios's
releases</a>.</em></p>
<blockquote>
<h2>Release v1.8.3</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li>add missing type for allowAbsoluteUrls (<a
href="https://redirect.github.com/axios/axios/issues/6818">#6818</a>)
(<a
href="10fa70ef14">10fa70e</a>)</li>
<li><strong>xhr/fetch:</strong> pass <code>allowAbsoluteUrls</code> to
<code>buildFullPath</code> in <code>xhr</code> and <code>fetch</code>
adapters (<a
href="https://redirect.github.com/axios/axios/issues/6814">#6814</a>)
(<a
href="ec159e507b">ec159e5</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/Electroid"
title="+6/-0 ([#6811](https://github.com/axios/axios/issues/6811)
)">Ashcon Partovi</a></li>
<li><!-- raw HTML omitted --> <a href="https://github.com/StefanBRas"
title="+4/-0 ([#6818](https://github.com/axios/axios/issues/6818)
)">StefanBRas</a></li>
<li><!-- raw HTML omitted --> <a href="https://github.com/mhassan1"
title="+2/-2 ([#6814](https://github.com/axios/axios/issues/6814)
)">Marc Hassan</a></li>
</ul>
<h2>Release v1.8.2</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>http-adapter:</strong> add allowAbsoluteUrls to path
building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)
(<a
href="fb8eec214c">fb8eec2</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/lexcorp16"
title="+1/-1 ([#6810](https://github.com/axios/axios/issues/6810)
)">Fasoro-Joseph Alexander</a></li>
</ul>
<h2>Release v1.8.1</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>utils:</strong> move <code>generateString</code> to platform
utils to avoid importing crypto module into client builds; (<a
href="https://redirect.github.com/axios/axios/issues/6789">#6789</a>)
(<a
href="36a5a620be">36a5a62</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a
href="https://github.com/DigitalBrainJS" title="+51/-47
([#6789](https://github.com/axios/axios/issues/6789) )">Dmitriy
Mozgovoy</a></li>
</ul>
<h2>Release v1.8.0</h2>
<h2>Release notes:</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>examples:</strong> application crashed when navigating
examples in browser (<a
href="https://redirect.github.com/axios/axios/issues/5938">#5938</a>)
(<a
href="1260ded634">1260ded</a>)</li>
<li>missing word in SUPPORT_QUESTION.yml (<a
href="https://redirect.github.com/axios/axios/issues/6757">#6757</a>)
(<a
href="1f890b13f2">1f890b1</a>)</li>
<li><strong>utils:</strong> replace getRandomValues with crypto module
(<a
href="https://redirect.github.com/axios/axios/issues/6788">#6788</a>)
(<a
href="23a25af068">23a25af</a>)</li>
</ul>
<h3>Features</h3>
<ul>
<li>Add config for ignoring absolute URLs (<a
href="https://redirect.github.com/axios/axios/issues/5902">#5902</a>)
(<a
href="https://redirect.github.com/axios/axios/issues/6192">#6192</a>)
(<a
href="32c7bcc0f2">32c7bcc</a>)</li>
</ul>
<h3>Reverts</h3>
<ul>
<li>Revert &quot;chore: expose fromDataToStream to be consumable (<a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a>)&quot;
(<a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a>)
(<a
href="1317261125">1317261</a>),
closes <a
href="https://redirect.github.com/axios/axios/issues/6731">#6731</a> <a
href="https://redirect.github.com/axios/axios/issues/6732">#6732</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/axios/axios/blob/v1.x/CHANGELOG.md">axios's
changelog</a>.</em></p>
<blockquote>
<h2><a
href="https://github.com/axios/axios/compare/v1.8.2...v1.8.3">1.8.3</a>
(2025-03-10)</h2>
<h3>Bug Fixes</h3>
<ul>
<li>add missing type for allowAbsoluteUrls (<a
href="https://redirect.github.com/axios/axios/issues/6818">#6818</a>)
(<a
href="10fa70ef14">10fa70e</a>)</li>
<li><strong>xhr/fetch:</strong> pass <code>allowAbsoluteUrls</code> to
<code>buildFullPath</code> in <code>xhr</code> and <code>fetch</code>
adapters (<a
href="https://redirect.github.com/axios/axios/issues/6814">#6814</a>)
(<a
href="ec159e507b">ec159e5</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/Electroid"
title="+6/-0 ([#6811](https://github.com/axios/axios/issues/6811)
)">Ashcon Partovi</a></li>
<li><!-- raw HTML omitted --> <a href="https://github.com/StefanBRas"
title="+4/-0 ([#6818](https://github.com/axios/axios/issues/6818)
)">StefanBRas</a></li>
<li><!-- raw HTML omitted --> <a href="https://github.com/mhassan1"
title="+2/-2 ([#6814](https://github.com/axios/axios/issues/6814)
)">Marc Hassan</a></li>
</ul>
<h2><a
href="https://github.com/axios/axios/compare/v1.8.1...v1.8.2">1.8.2</a>
(2025-03-07)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>http-adapter:</strong> add allowAbsoluteUrls to path
building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)
(<a
href="fb8eec214c">fb8eec2</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a href="https://github.com/lexcorp16"
title="+1/-1 ([#6810](https://github.com/axios/axios/issues/6810)
)">Fasoro-Joseph Alexander</a></li>
</ul>
<h2><a
href="https://github.com/axios/axios/compare/v1.8.0...v1.8.1">1.8.1</a>
(2025-02-26)</h2>
<h3>Bug Fixes</h3>
<ul>
<li><strong>utils:</strong> move <code>generateString</code> to platform
utils to avoid importing crypto module into client builds; (<a
href="https://redirect.github.com/axios/axios/issues/6789">#6789</a>)
(<a
href="36a5a620be">36a5a62</a>)</li>
</ul>
<h3>Contributors to this release</h3>
<ul>
<li><!-- raw HTML omitted --> <a
href="https://github.com/DigitalBrainJS" title="+51/-47
([#6789](https://github.com/axios/axios/issues/6789) )">Dmitriy
Mozgovoy</a></li>
</ul>
<h1><a
href="https://github.com/axios/axios/compare/v1.7.9...v1.8.0">1.8.0</a>
(2025-02-25)</h1>
<h3>Bug Fixes</h3>
<ul>
<li><strong>examples:</strong> application crashed when navigating
examples in browser (<a
href="https://redirect.github.com/axios/axios/issues/5938">#5938</a>)
(<a
href="1260ded634">1260ded</a>)</li>
<li>missing word in SUPPORT_QUESTION.yml (<a
href="https://redirect.github.com/axios/axios/issues/6757">#6757</a>)
(<a
href="1f890b13f2">1f890b1</a>)</li>
<li><strong>utils:</strong> replace getRandomValues with crypto module
(<a
href="https://redirect.github.com/axios/axios/issues/6788">#6788</a>)
(<a
href="23a25af068">23a25af</a>)</li>
</ul>
<h3>Features</h3>
<ul>
<li>Add config for ignoring absolute URLs (<a
href="https://redirect.github.com/axios/axios/issues/5902">#5902</a>)
(<a
href="https://redirect.github.com/axios/axios/issues/6192">#6192</a>)
(<a
href="32c7bcc0f2">32c7bcc</a>)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="39ec206483"><code>39ec206</code></a>
chore(release): v1.8.3 (<a
href="https://redirect.github.com/axios/axios/issues/6819">#6819</a>)</li>
<li><a
href="10fa70ef14"><code>10fa70e</code></a>
fix: add missing type for allowAbsoluteUrls (<a
href="https://redirect.github.com/axios/axios/issues/6818">#6818</a>)</li>
<li><a
href="7821ef9f5b"><code>7821ef9</code></a>
docs: update readme to include bun install (<a
href="https://redirect.github.com/axios/axios/issues/6811">#6811</a>)</li>
<li><a
href="ec159e507b"><code>ec159e5</code></a>
fix(xhr/fetch): pass <code>allowAbsoluteUrls</code> to
<code>buildFullPath</code> in <code>xhr</code> and `fet...</li>
<li><a
href="a9f7689b0c"><code>a9f7689</code></a>
chore(release): v1.8.2 (<a
href="https://redirect.github.com/axios/axios/issues/6812">#6812</a>)</li>
<li><a
href="fb8eec214c"><code>fb8eec2</code></a>
fix(http-adapter): add allowAbsoluteUrls to path building (<a
href="https://redirect.github.com/axios/axios/issues/6810">#6810</a>)</li>
<li><a
href="9812045755"><code>9812045</code></a>
chore(sponsor): update sponsor block (<a
href="https://redirect.github.com/axios/axios/issues/6804">#6804</a>)</li>
<li><a
href="72acf75937"><code>72acf75</code></a>
chore(sponsor): update sponsor block (<a
href="https://redirect.github.com/axios/axios/issues/6794">#6794</a>)</li>
<li><a
href="2e64afdff5"><code>2e64afd</code></a>
chore(release): v1.8.1 (<a
href="https://redirect.github.com/axios/axios/issues/6800">#6800</a>)</li>
<li><a
href="36a5a620be"><code>36a5a62</code></a>
fix(utils): move <code>generateString</code> to platform utils to avoid
importing crypto...</li>
<li>Additional commits viewable in <a
href="https://github.com/axios/axios/compare/v1.7.4...v1.8.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=axios&package-manager=npm_and_yarn&previous-version=1.7.4&new-version=1.8.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/zitadel/zitadel/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-27 08:00:16 +00:00
dependabot[bot]
bdf15a20f7 chore(deps): bump nanoid from 3.3.7 to 3.3.8 in /docs (#9057)
Bumps [nanoid](https://github.com/ai/nanoid) from 3.3.7 to 3.3.8.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/ai/nanoid/blob/main/CHANGELOG.md">nanoid's
changelog</a>.</em></p>
<blockquote>
<h2>3.3.8</h2>
<ul>
<li>Fixed a way to break Nano ID by passing non-integer size (by <a
href="https://github.com/myndzi"><code>@​myndzi</code></a>).</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="3044cd5e73"><code>3044cd5</code></a>
Release 3.3.8 version</li>
<li><a
href="4fe34959c3"><code>4fe3495</code></a>
Update size limit</li>
<li><a
href="d643045f40"><code>d643045</code></a>
Fix pool pollution, infinite loop (<a
href="https://redirect.github.com/ai/nanoid/issues/510">#510</a>)</li>
<li>See full diff in <a
href="https://github.com/ai/nanoid/compare/3.3.7...3.3.8">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nanoid&package-manager=npm_and_yarn&previous-version=3.3.7&new-version=3.3.8)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/zitadel/zitadel/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-27 07:37:18 +00:00
Zach Hirschtritt
c1535b7b49 feat: add prometheus metrics on projection handlers (#9561)
# Which Problems Are Solved

With current provided telemetry it's difficult to predict when a
projection handler is under increased load until it's too late and
causes downstream issues. Importantly, projection updating is in the
critical path for many login flows and increased latency there can
result in system downtime for users.

# How the Problems Are Solved

This PR adds three new prometheus-style metrics: 
1. **projection_events_processed** (_labels: projection, success_) -
This metric gives us a counter of the number of events processed per
projection update run and whether they we're processed without error. A
high number of events being processed can let us know how busy a
particular projection handler is.

2. **projection_handle_timer** _(labels: projection)_ - This is the time
it takes to process a projection update given a batch of events - time
to take the current_states lock, query for new events, reduce,
update_the projection, and update current_states.

3. **projection_state_latency** _(labels: projection)_ - This is the
time from the last event processed in the current_states table for a
given projection. It tells us how old was the last event you processed?
Or, how far behind are you running for this projection? Higher latencies
could mean high load or stalled projection handling.

# Additional Changes

I also had to initialize the global otel metrics provider (`metrics.M`)
in the `setup` step additionally to `start` since projection handlers
are initialized at setup. The initialization checks if a metrics
provider is already set (in case of `start-from-setup` or
`start-from-init` to prevent overwriting, which causes the otel metrics
provider to stop working.

# Additional Context

## Example Dashboards


![image](https://github.com/user-attachments/assets/94ba5c2b-9c62-44cd-83ee-4db4a8859073)

![image](https://github.com/user-attachments/assets/60a1b406-a8c6-48dc-a925-575359f97e1e)

---------

Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-27 07:40:27 +01:00
Stefan Benz
6b23c33cb6 fix: rename idp_templates6_ldap3 to ldap2 if necessary (#9565)
# Which Problems Are Solved

Zitadel setup with v2.71.0 could result in errors regarding the
idp_templates6_ldap3 subtable.

# How the Problems Are Solved

Rename the subtable idp_templates6_ldap3 to idp_templates6_ldap2 if no
idp_templates6_ldap2 is existing and rename column `rootCA` to
`root_ca`.

# Additional Changes

None

# Additional Context

Related PR #9292

---------

Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
2025-03-26 19:26:16 +00:00
Stefan Benz
0e10ed0e0b fix: SAML and OIDC issuer (in proxied use cases) (#9638)
# Which Problems Are Solved

When using implicit flow through the session API and a login UI on a
custom domain (proxy), the tokens were signed by the API domain of the
instance, rather than the public (proxy) domain.
The SAML response had the same issue. Additionally, the saml library had
an issue and lost the issuer context. This prevented also a successful
login through the hosted login UI.

# How the Problems Are Solved

- The issuer of the SAML and Auth request is persisted to provide the
information when signing the responses and tokens.
- The SAML library is updated to the latest version.

# Additional Changes

None

# Additional Context

None
2025-03-26 17:08:13 +00:00
Max
1c0c08307f feat: add EXPOSE command to Dockerfile (#9458)
# Which Problems Are Solved
- Solves a problem of healthcheck/use of zitadel in gitlab CI pipeline

# How the Problems Are Solved

- Adds an EXPOSE command to the Dockerfile

# Additional Changes
# Additional Context

- https://docs.docker.com/reference/dockerfile/#expose
-
cc254e85ed/16/bookworm/Dockerfile (L218)

Co-authored-by: Maksim Khardin <maksim.khardin@distribusion.com>
Co-authored-by: Ramon <mail@conblem.me>
Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-26 13:48:04 +00:00
Livio Spring
3424204291 chore(workflow): run jobs on public runners (#9643)
# Which Problems Are Solved

Build and test workflows are currently running on specific GitHub hosted
runners. These is not needed for most worklfows and just costs more.

# How the Problems Are Solved

Moved all the steps apart from integration-tests to public runners.

# Additional Changes

None

# Additional Context

None
2025-03-26 10:06:46 +00:00
Ramon
b04babb4ef fix: update console deps (#9628)
# Which Problems Are Solved
Closes most of the dependabot alerts

# How the Problems Are Solved
Updating Versions as much as possible. Angular upgrade was left out on
purpose.

# Additional Changes
Some refactoring of unused code, which I stumbled upon looking at our
used deps.
2025-03-26 10:15:17 +01:00
Fabienne Bühler
febf410d80 docs: Update customer portal docs/screenshots (#9604)
# Which Problems Are Solved

With the recent updates of our customer portal design, our onboarding
guide and customer portal docs were outdated.

# How the Problems Are Solved

- Updated screenshots of customer portal
- Updated screenshots of onboarding in management console
- Updated quickstarts with changes that have been made in the onbaording

---------

Co-authored-by: Maximilian <mpa@zitadel.com>
2025-03-25 18:20:49 +01:00
Livio Spring
57bfecf7f7 fix(actions): correctly handle api.v1.appendUserGrant (#9598)
# Which Problems Are Solved

It was not possible to use the `api.v1.appendUserGrant` function in the
`postCreation` trigger action as documented.

# How the Problems Are Solved

- Correctly initialize the javascript / Goja function
- Added `projectID` and `projectGrantID` (as documented), but kept
`projectId` and `projectGrantId` (for backwards compatibility) when
mapping the object in the `appendUserGrant` function

# Additional Changes

None

# Additional Context

- A customer reached out to support regarding this issue.
- requires back port to 2.70.x
2025-03-25 11:07:54 +00:00
Iraq
596970cc7e chore: updating go to 1.24 (#9507)
# Which Problems Are Solved

Updating go to 1.24

---------

Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
2025-03-25 07:01:29 +00:00
Livio Spring
e7c7f9b800 fix: correct event type for permission removed (#9626)
# Which Problems Are Solved

When we recently changed some permission for the `SYSTEM_OWNER` role on
QA, we noticed that there we multiple `permission.added` even when we
removed specific permissions.

# How the Problems Are Solved

Fixed the event type when removing permissions.

# Additional Changes

None

# Additional Context

Noticed when rolling out some changes on QA
2025-03-24 17:00:44 +00:00
Livio Spring
54e513f312 fix(console): pass verification code for phone verification (#9616)
# Which Problems Are Solved

While testing totally unrelated issues, I noticed that the phone
verification in Console didn't work anymore.

# How the Problems Are Solved

Pass the verification `code` entered by the user to the API.

# Additional Changes

None

# Additional Context

probably relates to https://github.com/zitadel/zitadel/pull/9312
2025-03-24 16:10:47 +01:00
Livio Spring
c2c02492b9 chore: update dependencies (#9614)
# Which Problems Are Solved

Multiple dependencies used by the Zitadel backend are outdated.

# How the Problems Are Solved

Update all direct dependencies (apart from the following two) to the
latest version:
- github.com/go-webauthn/webauthn: see
https://github.com/zitadel/zitadel/issues/9615
- github.com/nicksnyder/go-i18n/v2 (v2.5.0 / 2.5.1 prevent the use of
reserved keys such as `description`:
https://github.com/nicksnyder/go-i18n/releases/tag/v2.5.0)

# Additional Changes

None

# Additional Context

None
2025-03-24 12:06:35 +02:00
Tim Möhlmann
c3258ecf78 chore(deps): upgrade go mods from zitadel organization (#9601)
# Which Problems Are Solved

Outdated dependencies foir packages developed by Zitadel.
Some of them included important security updates from sub-dependencies.

# How the Problems Are Solved

Upgrade all packages under `github.com/zitadel/*` to the latest released
version.

# Additional Changes

- The `github.com/superseriousbusiness/exifremove` was removed from
Github. We copied the cached go mod code to
https://github.com/zitadel/exifremove and use this module now.

# Additional Context

- Related to https://github.com/zitadel/zitadel/issues/9422
- Closes https://github.com/zitadel/zitadel/issues/9443

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-24 06:03:24 +00:00
Elio Bischof
103b190af7 docs(self-hosting): use postgres for deployment docs (#9571)
# Which Problems Are Solved

The Kubernetes and Knative deployment docs suggest to depend on
CockroachDB installations even though we sunset the CockroachDB support
with the upcoming major version. This can be annoying for users who
create new environments using CockroachDB.

# How the Problems Are Solved

- The Kubernetes deployment is removed and points to examples in the
chart repo directy. This removes redundancies that are prone to getting
outdated without notice.

- The Knative deployment uses commands to deploy a PostgreSQL
StatefulSet instead of a CockroachDB StatefulSet. The DB command from
the Knative Tutorial is used, as users are likely to be familiar with
the tutorials configuration already. The static Kubernetes YAML files
for the DB as well as for the Zitadel Knative service are removed
altogether, as they are prone to getting outdated without notice and
don't serve any purpose.

# Additional Changes

- The outdated and boring video guide for Knative is removed.
- The Knative configuration is changed, so the first-time login becomes
easier which improves DevX.
- The current docker compose file doesn't work, this PR fixes it and
upgrades the used Postgres to v17.

# Additional Context

- Closes https://github.com/zitadel/zitadel-charts/issues/322
- Replaces https://github.com/zitadel/zitadel/pull/9540

Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
2025-03-21 15:49:37 +00:00
Ramon
8b1b9cbb98 fix: allow native applications to use https:// on loopback redirect addresses (#9073)
# Which Problems Are Solved

- The current validation for native redirect URIs does not allow HTTPS
loopback addresses.

# How the Problems Are Solved

- Enhanced the validation logic to permit HTTPS loopback addresses,
ensuring that developers can use these addresses without encountering
validation errors.
- Updated zitadel/oidc to latest version

# Additional Context

- Closes #4091
- This pr need to be closed first in our OIDC lib:
https://github.com/zitadel/oidc/pull/691

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-21 13:55:16 +00:00
Miro Trisc
e4c12864e5 feat(crypto): support md5 salted for imported password hashes (#9596)
# Which Problems Are Solved

Allow verification of imported salted passwords hashed with plain md5.

# How the Problems Are Solved

- Upgrade passwap to
[v0.7.0](https://github.com/zitadel/passwap/releases/tag/v0.7.0)
- Add md5salted as a new verifier option in `defaults.yaml`

# Additional Changes

- go version and libraries updated  (required by passkey v0.7.0)
- secrets.md verifiers updated  
- configuration verifiers updated 
- added MD5salted and missing MD5Plain to test cases
2025-03-21 12:25:52 +00:00
Harsha Reddy
dc64e35128 feat: Make service name configurable for Metrics and Tracing (#9563)
# Which Problems Are Solved

The service name is hardcoded in the metrics code. Making the service
name to be configurable helps when running multiple instances of
Zitadel.

The defaults remain unchanged, the service name will be defaulted to
ZITADEL.

# How the Problems Are Solved

Add a config option to override the name in defaults.yaml and pass it
down to the corresponding metrics or tracing module (google or otel)

# Additional Changes
NA

# Additional Context
NA
2025-03-20 09:35:54 +00:00
Faey
833e654a07 feat(actions): Add refresh token to post authentication action context (#9493)
# Which Problems Are Solved

- Refresh Tokens issued by third party authentication providers are lost

# How the Problems Are Solved

- Allows the existing post authentication action to capture the refresh
token

# Additional Changes

- Docs updated to reflect the new property

# Additional Context

- Partially addresses #7851 by allowing the refresh token to be
captured.

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
2025-03-20 09:00:36 +00:00
Stefan Benz
352fa6aa6f fix: milestone multiple results per instance domain instead of primary instance domain (#9564)
# Which Problems Are Solved

The milestones query returns multiple results for every milestone for
every instance domain.

# How the Problems Are Solved

Corrected where condition on milestone query.

# Additional Changes

None

# Additional Context

None

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-20 07:57:36 +00:00
Stefan Benz
5486ef2627 test: correct sorting of scim list users integration tests (#9568)
# Which Problems Are Solved

SCIM integration test failed sometimes, as ListUsers with usernames-sort
was not reliable if the asserted list is not sorted as well.

# How the Problems Are Solved

Sort the list of results in the sorted integration tests.

# Additional Changes

None

# Additional Context

Relates to
https://github.com/zitadel/zitadel/actions/runs/13922326003/job/38960759621
2025-03-20 07:05:01 +00:00
Stefan Benz
5ca76af779 test: correct notifications integration test with eventual consistency (#9569)
# Which Problems Are Solved

Quota notification integration test failed sometimes due to eventual
consistency issues, which resulted in calls which should have been
counted to the quota not being added. This resulted in flaky integration
tests as the expected API calls to be limited were executed normally.

# How the Problems Are Solved

As there is no API call to query the currently applied Quota, there was
a sleep added as a last effort, to give some time that the event gets
processed into the projection.

# Additional Changes

None

# Additional Context

Related to
https://github.com/zitadel/zitadel/actions/runs/13922326003/job/38959595055

Co-authored-by: Livio Spring <livio.a@gmail.com>
2025-03-20 07:47:19 +01:00
Maximilian
988146b2ad fix: sidebar imports (#9589)
# Which Problems Are Solved

Running local development of the docs didn't work due to errors with
sidebar imports.

# How the Problems Are Solved

The imports held the sidebar items in a "default" object. Changed the
imports

# Additional Changes

removed some comments
2025-03-19 18:21:41 +02:00
Max Peintner
83108d1dc3 fix(console): Implement generic feature toggle (#9516)
This PR implements a component which is used to render the feature
settings in a generic way.
All features besides`loginV2` and `improvedPerformance` are rendered.

Note: The feature flags are not autogenerated as the console needs
descriptions for the respective features and there are multiple types
where additional fields like text input is required

Closes #9090

---------

Co-authored-by: conblem <mail@conblem.me>
2025-03-19 15:52:09 +00:00
Ramon
b418ea75bb feat: create human user v2 page (#9506)
# Which Problems Are Solved
Allows users to be created using the V2 User API

# How the Problems Are Solved
I added a seperate V2 create user page with the new code using the new
apis.

# Additional Changes
I did some refactorings arround our interceptors as they used an
obselete syntax.
The password complexity form takes the Buf definitions.

# Additional Context
- Closes #9430

---------

Co-authored-by: Max Peintner <peintnerm@gmail.com>
2025-03-19 13:27:59 +01:00
Iraq
11c9be3b8d chore: updating projections.idp_templates6 to projections.idp_templates7 (#9517)
# Which Problems Are Solved

This was left out as part of
https://github.com/zitadel/zitadel/pull/9292

- Closes https://github.com/zitadel/zitadel/issues/9514

---------

Co-authored-by: Iraq Jaber <IraqJaber@gmail.com>
2025-03-18 16:23:12 +01:00
Livio Spring
f1f500d0e7 chore: use crdb 24.3 (#9558)
# Which Problems Are Solved

E2E tests in pipelines started to fail randomly. While debugging it, i
noticed that we use the `latest` tag of cockroach's docker image. They
tagged 25.1 as latest yesterday.

# How the Problems Are Solved

Since we drop support for CRDB with version 3 as there are anyway
multiple issues with various versions, I pinned the docker image tag to
`latest-v24.3`.

# Additional Changes

None

# Additional Context

relates to https://github.com/zitadel/zitadel/actions/runs/13917603587
and https://github.com/zitadel/zitadel/actions/runs/13904928050
2025-03-18 10:52:38 +01:00
Stefan Benz
6cd5e717a4 chore: correct integration tests for query endpoints eventual consistent (#9554)
# Which Problems Are Solved

Integration tests for OIDC service failed irregularly.

# How the Problems Are Solved

Add eventual consistent checks for querying endpoints to the integration
tests for the OIDC service.

# Additional Changes

None

# Additional Context

None
2025-03-18 08:16:48 +01:00
Harsha Reddy
599850e7e8 fix: reduce cardinality in metrics and tracing for unknown paths (#9523)
# Which Problems Are Solved
Zitadel should not record 404 response counts of unknown paths (check
`/debug/metrics`).
This can lead to high cardinality on metrics endpoint and in traces.

```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
BAD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="404",uri="/junk"} 2000
```

After
```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
```

# How the Problems Are Solved

This PR makes sure, that any unknown path is recorded as `UNKNOWN_PATH`
instead of the actual path.

# Additional Changes

N/A

# Additional Context

On our production instance, when a penetration test was run, it caused
our metric count to blow up to many thousands due to Zitadel recording
404 response counts.

Next nice to have steps, remove 404 timer recordings which serve no
purpose

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio@zitadel.com>
2025-03-17 17:37:59 +01:00
Kenta Yamaguchi
d57fa819cb chore: Replace deprecated io/ioutil functions with recommended alternatives (#9542)
# Which Problems Are Solved

- The `io/ioutil` package was deprecated in Go 1.16.  
  - Reference: https://go.dev/doc/go1.16#ioutil  

# How the Problems Are Solved

- Replaced deprecated functions with their recommended alternatives:  
  - `ioutil.ReadFile` → `os.ReadFile`  
  - `ioutil.ReadAll` → `io.ReadAll`  
  - `ioutil.NopCloser` → `io.NopCloser`
2025-03-17 13:17:14 +00:00
Livio Spring
19f022e1cf fix(login): cache scripts and other assets correctly (#9551)
# Which Problems Are Solved

Scripts and other assets for the hosted login UI are served with a
public cache with `max-age` and `s-maxage`. After changing scripts or
assets, old versions might be still used as they might be cached locally
or in a shared cache (CDN, proxy, ...). This can lead to unwanted
behaviour or even errors.

# How the Problems Are Solved

To ensure the correct file is served a query parameter with the build
time is added to the assets filename dynamically. (`?v=2025-03-17...`)

# Additional Changes

None

# Additional Context

- relates to #9485
- requires backport to at least 2.70.x
2025-03-17 11:13:24 +00:00
Kenta Yamaguchi
9767519db6 chore: Update contrib.rocks link in README.md (#9543)
# Which Problems Are Solved

- The current contrib.rocks link (`https://contrib.rocks`) does not
directly preview the contributor graph for the zitadel/zitadel
repository.

# How the Problems Are Solved

- Updated the contrib.rocks link to
`https://contrib.rocks/preview?repo=zitadel/zitadel`, which directly
shows the contributor graph for this repository.
2025-03-17 10:59:19 +00:00
Elio Bischof
9bbc3c48fc docs: show proxy link only once (#9537)
# Which Problems Are Solved

The reverse proxy docs have too many links to the third party proxy
provider. This is noisy and might result in unintentional redirects.
![image
(1)](https://github.com/user-attachments/assets/3127f3bc-a6e1-47c5-a565-4f68ce1650d3)


# How the Problems Are Solved

The link to the proxy provider is only shown on the first occurence of
the provider name instead of all occurences.
2025-03-14 11:52:09 +01:00
Livio Spring
61c4b1c3fd fix(console): allow management of metadata of users of other organizations again (#9490)
# Which Problems Are Solved

With the recent change in Console to use the User V2 API
(https://github.com/zitadel/zitadel/pull/9312), some functionality still
needs to call the management API, which requires the organization
context. The context was not passed anymore, leading to error in cases
where the calling user (e.g. an IAM_OWNER) was not part of the same
organization.

# How the Problems Are Solved

Added an interceptor to provide the `x-zitadel-orgid` header for the new
management client.

# Additional Changes

None

# Additional Context

- closes #9488
2025-03-14 06:05:45 +00:00
Silvan
e36f402e09 fix(perf): simplify eventstore queries by removing or in projection handlers (#9530)
# Which Problems Are Solved

[A recent performance
enhancement]((https://github.com/zitadel/zitadel/pull/9497)) aimed at
optimizing event store queries, specifically those involving multiple
aggregate type filters, has successfully improved index utilization.
While the query planner now correctly selects relevant indexes, it
employs [bitmap index
scans](https://www.postgresql.org/docs/current/indexes-bitmap-scans.html)
to retrieve data.

This approach, while beneficial in many scenarios, introduces a
potential I/O bottleneck. The bitmap index scan first identifies the
required database blocks and then utilizes a bitmap to access the
corresponding rows from the table's heap. This subsequent "bitmap heap
scan" can result in significant I/O overhead, particularly when queries
return a substantial number of rows across numerous data pages.

## Impact:

Under heavy load or with queries filtering for a wide range of events
across multiple aggregate types, this increased I/O activity may lead
to:

- Increased query latency.
- Elevated disk utilization.
- Potential performance degradation of the event store and dependent
systems.

# How the Problems Are Solved

To address this I/O bottleneck and further optimize query performance,
the projection handler has been modified. Instead of employing multiple
OR clauses for each aggregate type, the aggregate and event type filters
are now combined using IN ARRAY filters.

Technical Details:

This change allows the PostgreSQL query planner to leverage [index-only
scans](https://www.postgresql.org/docs/current/indexes-index-only-scans.html).
By utilizing IN ARRAY filters, the database can efficiently retrieve the
necessary data directly from the index, eliminating the need to access
the table's heap. This results in:

* Reduced I/O: Index-only scans significantly minimize disk I/O
operations, as the database avoids reading data pages from the main
table.
* Improved Query Performance: By reducing I/O, query execution times are
substantially improved, leading to lower latency.

# Additional Changes

- rollback of https://github.com/zitadel/zitadel/pull/9497

# Additional Information

## Query Plan of previous query

```sql
SELECT 
    created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision 
FROM 
    eventstore.events2 
WHERE 
    instance_id = '<INSTANCE_ID>'
    AND (
        (
            instance_id = '<INSTANCE_ID>'
            AND "position" > <POSITION>
            AND aggregate_type = 'project'
            AND event_type = ANY(ARRAY[
                                'project.application.added'
                                ,'project.application.changed'
                                ,'project.application.deactivated'
                                ,'project.application.reactivated'
                                ,'project.application.removed'
                                ,'project.removed'
                                ,'project.application.config.api.added'
                                ,'project.application.config.api.changed'
                                ,'project.application.config.api.secret.changed'
                ,'project.application.config.api.secret.updated'
                                ,'project.application.config.oidc.added'
                                ,'project.application.config.oidc.changed'
                                ,'project.application.config.oidc.secret.changed'
                ,'project.application.config.oidc.secret.updated'
                                ,'project.application.config.saml.added'
                                ,'project.application.config.saml.changed'
                        ])
        ) OR (
            instance_id = '<INSTANCE_ID>'
            AND "position" > <POSITION>
            AND aggregate_type = 'org'
            AND event_type = 'org.removed'
        ) OR (
            instance_id = '<INSTANCE_ID>'
            AND "position" > <POSITION>
            AND aggregate_type = 'instance'
            AND event_type = 'instance.removed'
        )
    ) 
    AND "position" > 1741600905.3495
    AND "position" < (
        SELECT 
            COALESCE(EXTRACT(EPOCH FROM min(xact_start)), EXTRACT(EPOCH FROM now())) 
        FROM 
            pg_stat_activity
        WHERE 
            datname = current_database() 
            AND application_name = ANY(ARRAY['zitadel_es_pusher_', 'zitadel_es_pusher', 'zitadel_es_pusher_<INSTANCE_ID>']) 
            AND state <> 'idle'
    ) 
ORDER BY "position", in_tx_order LIMIT 200 OFFSET 1;
```

```
Limit  (cost=120.08..120.09 rows=7 width=361) (actual time=2.167..2.172 rows=0 loops=1)
   Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
   InitPlan 1
     ->  Aggregate  (cost=2.74..2.76 rows=1 width=32) (actual time=1.813..1.815 rows=1 loops=1)
           Output: COALESCE(EXTRACT(epoch FROM min(s.xact_start)), EXTRACT(epoch FROM now()))
           ->  Nested Loop  (cost=0.00..2.74 rows=1 width=8) (actual time=1.803..1.805 rows=0 loops=1)
                 Output: s.xact_start
                 Join Filter: (d.oid = s.datid)
                 ->  Seq Scan on pg_catalog.pg_database d  (cost=0.00..1.07 rows=1 width=4) (actual time=0.016..0.021 rows=1 loops=1)
                       Output: d.oid, d.datname, d.datdba, d.encoding, d.datlocprovider, d.datistemplate, d.datallowconn, d.dathasloginevt, d.datconnlimit, d.datfrozenxid, d.datminmxid, d.dattablespace, d.datcollate, d.datctype, d.datlocale, d.daticurules, d.datcollversion, d.datacl
                       Filter: (d.datname = current_database())
                       Rows Removed by Filter: 4
                 ->  Function Scan on pg_catalog.pg_stat_get_activity s  (cost=0.00..1.63 rows=3 width=16) (actual time=1.781..1.781 rows=0 loops=1)
                       Output: s.datid, s.pid, s.usesysid, s.application_name, s.state, s.query, s.wait_event_type, s.wait_event, s.xact_start, s.query_start, s.backend_start, s.state_change, s.client_addr, s.client_hostname, s.client_port, s.backend_xid, s.backend_xmin, s.backend_type, s.ssl, s.sslversion, s.sslcipher, s.sslbits, s.ssl_client_dn, s.ssl_client_serial, s.ssl_issuer_dn, s.gss_auth, s.gss_princ, s.gss_enc, s.gss_delegation, s.leader_pid, s.query_id
                       Function Call: pg_stat_get_activity(NULL::integer)
                       Filter: ((s.state <> 'idle'::text) AND (s.application_name = ANY ('{zitadel_es_pusher_,zitadel_es_pusher,zitadel_es_pusher_<INSTANCE_ID>}'::text[])))
                       Rows Removed by Filter: 49
   ->  Sort  (cost=117.31..117.33 rows=8 width=361) (actual time=2.167..2.168 rows=0 loops=1)
         Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
         Sort Key: events2."position", events2.in_tx_order
         Sort Method: quicksort  Memory: 25kB
         ->  Bitmap Heap Scan on eventstore.events2  (cost=84.92..117.19 rows=8 width=361) (actual time=2.088..2.089 rows=0 loops=1)
               Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
               Recheck Cond: (((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'project'::text) AND (events2.event_type = ANY ('{project.application.added,project.application.changed,project.application.deactivated,project.application.reactivated,project.application.removed,project.removed,project.application.config.api.added,project.application.config.api.changed,project.application.config.api.secret.changed,project.application.config.api.secret.updated,project.application.config.oidc.added,project.application.config.oidc.changed,project.application.config.oidc.secret.changed,project.application.config.oidc.secret.updated,project.application.config.saml.added,project.application.config.saml.changed}'::text[])) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1)) OR ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'org'::text) AND (events2.event_type = 'org.removed'::text) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1)) OR ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'instance'::text) AND (events2.event_type = 'instance.removed'::text) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1)))
               ->  BitmapOr  (cost=84.88..84.88 rows=8 width=0) (actual time=2.080..2.081 rows=0 loops=1)
                     ->  Bitmap Index Scan on es_projection  (cost=0.00..75.44 rows=8 width=0) (actual time=2.016..2.017 rows=0 loops=1)
                           Index Cond: ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'project'::text) AND (events2.event_type = ANY ('{project.application.added,project.application.changed,project.application.deactivated,project.application.reactivated,project.application.removed,project.removed,project.application.config.api.added,project.application.config.api.changed,project.application.config.api.secret.changed,project.application.config.api.secret.updated,project.application.config.oidc.added,project.application.config.oidc.changed,project.application.config.oidc.secret.changed,project.application.config.oidc.secret.updated,project.application.config.saml.added,project.application.config.saml.changed}'::text[])) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1))
                     ->  Bitmap Index Scan on es_projection  (cost=0.00..4.71 rows=1 width=0) (actual time=0.016..0.016 rows=0 loops=1)
                           Index Cond: ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'org'::text) AND (events2.event_type = 'org.removed'::text) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1))
                     ->  Bitmap Index Scan on es_projection  (cost=0.00..4.71 rows=1 width=0) (actual time=0.045..0.045 rows=0 loops=1)
                           Index Cond: ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = 'instance'::text) AND (events2.event_type = 'instance.removed'::text) AND (events2."position" > <POSITION>) AND (events2."position" > 1741600905.3495) AND (events2."position" < (InitPlan 1).col1))
 Query Identifier: 3194938266011254479
 Planning Time: 1.295 ms
 Execution Time: 2.832 ms
```

## Query Plan of new query

```sql
SELECT 
    created_at, event_type, "sequence", "position", payload, creator, "owner", instance_id, aggregate_type, aggregate_id, revision 
FROM 
    eventstore.events2 
WHERE 
    instance_id = '<INSTANCE_ID>'
    AND "position" > <POSITION>
    AND aggregate_type = ANY(ARRAY['project', 'instance', 'org'])
    AND event_type = ANY(ARRAY[
        'project.application.added'
        ,'project.application.changed'
        ,'project.application.deactivated'
        ,'project.application.reactivated'
        ,'project.application.removed'
        ,'project.removed'
        ,'project.application.config.api.added'
        ,'project.application.config.api.changed'
        ,'project.application.config.api.secret.changed'
        ,'project.application.config.api.secret.updated'
        ,'project.application.config.oidc.added'
        ,'project.application.config.oidc.changed'
        ,'project.application.config.oidc.secret.changed'
        ,'project.application.config.oidc.secret.updated'
        ,'project.application.config.saml.added'
        ,'project.application.config.saml.changed'
        ,'org.removed'
        ,'instance.removed'
    ])
    AND "position" < (
        SELECT 
            COALESCE(EXTRACT(EPOCH FROM min(xact_start)), EXTRACT(EPOCH FROM now())) 
        FROM 
            pg_stat_activity
        WHERE 
            datname = current_database() 
            AND application_name = ANY(ARRAY['zitadel_es_pusher_', 'zitadel_es_pusher', 'zitadel_es_pusher_<INSTANCE_ID>']) 
            AND state <> 'idle'
    ) 
ORDER BY "position", in_tx_order LIMIT 200 OFFSET 1;
```

```
Limit  (cost=293.34..293.36 rows=8 width=361) (actual time=4.686..4.689 rows=0 loops=1)
   Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
   InitPlan 1
     ->  Aggregate  (cost=2.74..2.76 rows=1 width=32) (actual time=1.717..1.719 rows=1 loops=1)
           Output: COALESCE(EXTRACT(epoch FROM min(s.xact_start)), EXTRACT(epoch FROM now()))
           ->  Nested Loop  (cost=0.00..2.74 rows=1 width=8) (actual time=1.658..1.659 rows=0 loops=1)
                 Output: s.xact_start
                 Join Filter: (d.oid = s.datid)
                 ->  Seq Scan on pg_catalog.pg_database d  (cost=0.00..1.07 rows=1 width=4) (actual time=0.026..0.028 rows=1 loops=1)
                       Output: d.oid, d.datname, d.datdba, d.encoding, d.datlocprovider, d.datistemplate, d.datallowconn, d.dathasloginevt, d.datconnlimit, d.datfrozenxid, d.datminmxid, d.dattablespace, d.datcollate, d.datctype, d.datlocale, d.daticurules, d.datcollversion, d.datacl
                       Filter: (d.datname = current_database())
                       Rows Removed by Filter: 4
                 ->  Function Scan on pg_catalog.pg_stat_get_activity s  (cost=0.00..1.63 rows=3 width=16) (actual time=1.628..1.628 rows=0 loops=1)
                       Output: s.datid, s.pid, s.usesysid, s.application_name, s.state, s.query, s.wait_event_type, s.wait_event, s.xact_start, s.query_start, s.backend_start, s.state_change, s.client_addr, s.client_hostname, s.client_port, s.backend_xid, s.backend_xmin, s.backend_type, s.ssl, s.sslversion, s.sslcipher, s.sslbits, s.ssl_client_dn, s.ssl_client_serial, s.ssl_issuer_dn, s.gss_auth, s.gss_princ, s.gss_enc, s.gss_delegation, s.leader_pid, s.query_id
                       Function Call: pg_stat_get_activity(NULL::integer)
                       Filter: ((s.state <> 'idle'::text) AND (s.application_name = ANY ('{zitadel_es_pusher_,zitadel_es_pusher,zitadel_es_pusher_<INSTANCE_ID>}'::text[])))
                       Rows Removed by Filter: 42
   ->  Sort  (cost=290.58..290.60 rows=9 width=361) (actual time=4.685..4.685 rows=0 loops=1)
         Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
         Sort Key: events2."position", events2.in_tx_order
         Sort Method: quicksort  Memory: 25kB
         ->  Index Scan using es_projection on eventstore.events2  (cost=0.70..290.43 rows=9 width=361) (actual time=4.616..4.617 rows=0 loops=1)
               Output: events2.created_at, events2.event_type, events2.sequence, events2."position", events2.payload, events2.creator, events2.owner, events2.instance_id, events2.aggregate_type, events2.aggregate_id, events2.revision, events2.in_tx_order
               Index Cond: ((events2.instance_id = '<INSTANCE_ID>'::text) AND (events2.aggregate_type = ANY ('{project,instance,org}'::text[])) AND (events2.event_type = ANY ('{project.application.added,project.application.changed,project.application.deactivated,project.application.reactivated,project.application.removed,project.removed,project.application.config.api.added,project.application.config.api.changed,project.application.config.api.secret.changed,project.application.config.api.secret.updated,project.application.config.oidc.added,project.application.config.oidc.changed,project.application.config.oidc.secret.changed,project.application.config.oidc.secret.updated,project.application.config.saml.added,project.application.config.saml.changed,org.removed,instance.removed}'::text[])) AND (events2."position" > <POSITION>) AND (events2."position" < (InitPlan 1).col1))
 Query Identifier: -8254550537132386499
 Planning Time: 2.864 ms
 Execution Time: 5.414 ms
 ```
2025-03-13 16:50:23 +01:00
Stefan Benz
5eb3a543e8 fix: add permission check for saml request query (#9520) 2025-03-12 21:53:16 +01:00
Livio Spring
d527a1c824 feat(console): specify login UI for SAML apps (#9486)
# Which Problems Are Solved

Users were not yet able to specify (and test) the new login UI or
self-hosted login UI for SAML applications through Console.

# How the Problems Are Solved

Added the configuration for SAML apps (as already available for OIDC) in
Console.

# Additional Changes

None

# Additional Context

- closes #9354
2025-03-12 12:37:43 +00:00
Silvan
b578137139 fix(eventstore): optimise query hints for event filters (#9497) 2025-03-12 09:44:06 +00:00