diff --git a/docs/docs/support/advisory/a10016.md b/docs/docs/support/advisory/a10016.md new file mode 100644 index 0000000000..794c354e42 --- /dev/null +++ b/docs/docs/support/advisory/a10016.md @@ -0,0 +1,98 @@ +--- +title: Technical Advisory 10016 +--- + +## Date + +Versions:[^1] + +- v2.65.x: > v2.65.9 + +Date: 2025-05-14 + +Last updated: 2025-05-14 + +[^1]: The mentioned fix is being rolled out gradually on multiple patch releases of Zitadel. This advisory will be updated as we release these versions. + +## Description + +### Background + +Zitadel uses a eventstore table as main source of truth for state changes. +Projections are tables which provide alternative views of state, which are built using events. +In order to know which events are reduced into projections, we use a `position` column in the eventstore and a dedicated table which records the current state. + +### Problem + +Zitadel prior to the listed version had a precision bug. The `position` column uses a fixed-point numeric type. In Zitadel's Go code we used a `float64`. In certain cases we noticed a precision loss when Zitadel updated the `current_states` table. + +## Impact + +During a past attempt to fix this, we got reports of failing projections inside Zitadel. Because the precision became exact certain compare operations like *equal*, *less then*, etc would now return different results. This was because the values in `current_states` would already have lost precision from a broken version. This might happen to **some** deployments or projections: there is only a small probability. + +We are releasing the fix again and your system might get affected. + +- Original issue: [8671](https://github.com/zitadel/zitadel/issues/8671) +- Follow-up issue: [8863](https://github.com/zitadel/zitadel/issues/8863) + +## Mitigation + +When **after** deploying a fixed version and only when experiencing problems described by issue [8863](https://github.com/zitadel/zitadel/issues/8863), the following queries can be executed to fix `current_state` rows which have "broken" values. We recommend doing this in a transaction in order to double-check the affected rows, before committing the update. + +```sql +begin; + +with + broken as ( + select + s.projection_name, + s.instance_id, + s.aggregate_id, + s.aggregate_type, + s.sequence, + s."position" as old_position, + e."position" as new_position + from + projections.current_states s + join eventstore.events2 e on s.instance_id = e.instance_id + and s.aggregate_id = e.aggregate_id + and s.aggregate_type = e.aggregate_type + and s.sequence = e.sequence + and s."position" != e."position" + where + s."position" != 0 + and projection_name != 'projections.execution_handler' + ),fixed as ( + update projections.current_states s + set + "position" = b.new_position + from + broken b + where + s.instance_id = b.instance_id + and s.projection_name = b.projection_name + and s.aggregate_id = b.aggregate_id + and s.aggregate_type = b.aggregate_type + and s.sequence = b.sequence + ) +select + b.projection_name, + b.instance_id, + b.aggregate_id, + b.aggregate_type, + b.sequence, + b.old_position, + b.new_position +from + broken b; +``` + +If the output from the above looks reasonable, for example not a huge difference between `old_position` and `new_position`, commit the transaction: + +```sql +commit; +``` + +When there are no rows returned, your system was not affected by precision loss. + +When there's unexpected output, use `rollback;` instead. diff --git a/docs/docs/support/technical_advisory.mdx b/docs/docs/support/technical_advisory.mdx index 0d8818c32c..5d13cfe3e5 100644 --- a/docs/docs/support/technical_advisory.mdx +++ b/docs/docs/support/technical_advisory.mdx @@ -238,6 +238,18 @@ We understand that these advisories may include breaking changes, and we aim to 3.0.0 2025-03-31 + + + A-10016 + + Position precision fix + Manual Intervention + + + + 2.65.10 + 2025-05-14 + ## Subscribe to our Mailing List