Commit Graph

2 Commits

Author SHA1 Message Date
Tim Möhlmann
ee7beca61f
fix(cache): ignore NOSCRIPT errors in redis circuit breaker (#9022)
# Which Problems Are Solved

When Zitadel starts the first time with a configured Redis cache, the
circuit break would open on the first requests, with no explanatory
error and only log-lines explaining the state of the Circuit breaker.

Using a debugger, `NOSCRIPT No matching script. Please use EVAL.` was
found the be passed to `Limiter.ReportResult`. This error is actually
retried by go-redis after a
[`Script.Run`](https://pkg.go.dev/github.com/redis/go-redis/v9@v9.7.0#Script.Run):

> Run optimistically uses EVALSHA to run the script. If script does not
exist it is retried using EVAL.

# How the Problems Are Solved

Add the `NOSCRIPT` error prefix to the whitelist.

# Additional Changes

- none

# Additional Context

- Introduced in: https://github.com/zitadel/zitadel/pull/8890
- Workaround for: https://github.com/redis/go-redis/issues/3203
2024-12-09 08:20:21 +00:00
Tim Möhlmann
3b7b0c69e6
feat(cache): redis circuit breaker (#8890)
# Which Problems Are Solved

If a redis cache has connection issues or any other type of permament
error,
it tanks the responsiveness of ZITADEL.
We currently do not support things like Redis cluster or sentinel. So
adding a simple redis cache improves performance but introduces a single
point of failure.

# How the Problems Are Solved

Implement a [circuit
breaker](https://learn.microsoft.com/en-us/previous-versions/msp-n-p/dn589784(v=pandp.10)?redirectedfrom=MSDN)
as
[`redis.Limiter`](https://pkg.go.dev/github.com/redis/go-redis/v9#Limiter)
by wrapping sony's [gobreaker](https://github.com/sony/gobreaker)
package. This package is picked as it seems well maintained and we
already use their `sonyflake` package

# Additional Changes

- The unit tests constructed an unused `redis.Client` and didn't cleanup
the connector. This is now fixed.

# Additional Context

Closes #8864
2024-11-13 19:11:48 +01:00