TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2025-08-11 23:47:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: reduce cardinality in metrics and tracing for unknown paths (#9523)

Browse Source 
# Which Problems Are Solved
Zitadel should not record 404 response counts of unknown paths (check
`/debug/metrics`).
This can lead to high cardinality on metrics endpoint and in traces.

```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
BAD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="404",uri="/junk"} 2000
```

After
```
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/.well-known/openid-configuration"} 2
GOOD http_server_return_code_counter_total{method="GET",otel_scope_name="",otel_scope_version="",return_code="200",uri="/oauth/v2/keys"} 2
```

# How the Problems Are Solved

This PR makes sure, that any unknown path is recorded as `UNKNOWN_PATH`
instead of the actual path.

# Additional Changes

N/A

# Additional Context

On our production instance, when a penetration test was run, it caused
our metric count to blow up to many thousands due to Zitadel recording
404 response counts.

Next nice to have steps, remove 404 timer recordings which serve no
purpose

---------

Co-authored-by: Livio Spring <livio.a@gmail.com>
Co-authored-by: Silvan <27845747+adlerhurst@users.noreply.github.com>
Co-authored-by: Livio Spring <livio@zitadel.com>
(cherry picked from commit 599850e7e8)
This commit is contained in:
Harsha Reddy
2025-03-17 22:07:59 +05:30
committed by Livio Spring
parent ed16104835
commit 935fd14d71
1 changed files with 6 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
internal/api/grpc/server/gateway.go
View File

@@ -28,6 +28,7 @@ import (
const ( const (
mimeWildcard = "*/*" mimeWildcard = "*/*"
UnknownPath = "UNKNOWN_PATH"
) )
var ( var (
@@ -274,7 +275,11 @@ func grpcCredentials(tlsConfig *tls.Config) credentials.TransportCredentials {
func setRequestURIPattern(ctx context.Context) { func setRequestURIPattern(ctx context.Context) {
pattern, ok := runtime.HTTPPathPattern(ctx) pattern, ok := runtime.HTTPPathPattern(ctx)
if !ok { if !ok {
return // As all unmatched paths will be handled by the gateway, any request not matching a pattern,
// means there's no route to the path.
// To prevent high cardinality on metrics and tracing, we want to make sure we don't record
// the actual path as name (it will still be recorded explicitly in the span http info).
pattern = UnknownPath
} }
span := trace.SpanFromContext(ctx) span := trace.SpanFromContext(ctx)
span.SetName(pattern) span.SetName(pattern)