zitadel/internal/query/resource_counts.go
Tim Möhlmann b9c1cdf4ad
feat(projections): resource counters (#9979)
# Which Problems Are Solved

Add the ability to keep track of the current counts of projection
resources. We want to prevent calling `SELECT COUNT(*)` on tables, as
that forces a full scan and sudden spikes of DB resource uses.

# How the Problems Are Solved

- A resource_counts table is added
- Triggers that increment and decrement the counted values on inserts
and deletes
- Triggers that delete all counts of a table when the source table is
TRUNCATEd. This is not in the business logic, but prevents wrong counts
in case someone want to force a re-projection.
- Triggers that delete all counts if the parent resource is deleted
- Script to pre-populate the resource_counts table when a new source
table is added.

The triggers are reusable for any type of resource, in case we choose to
add more in the future.
Counts are aggregated by a given parent. Currently only `instance` and
`organization` are defined as possible parent. This can later be
extended to other types, such as `project`, should the need arise.

I deliberately chose to use `parent_id` to distinguish from the
de-factor `resource_owner` which is usually an organization ID. For
example:

- For users the parent is an organization and the `parent_id` matches
`resource_owner`.
- For organizations the parent is an instance, but the `resource_owner`
is the `org_id`. In this case the `parent_id` is the `instance_id`.
- Applications would have a similar problem, where the parent is a
project, but the `resource_owner` is the `org_id`


# Additional Context

Closes https://github.com/zitadel/zitadel/issues/9957
2025-06-03 14:15:30 +00:00

62 lines
1.5 KiB
Go

package query
import (
"context"
"database/sql"
_ "embed"
"time"
"github.com/zitadel/zitadel/internal/domain"
"github.com/zitadel/zitadel/internal/telemetry/tracing"
"github.com/zitadel/zitadel/internal/zerrors"
)
var (
//go:embed resource_counts_list.sql
resourceCountsListQuery string
)
type ResourceCount struct {
ID int // Primary key, used for pagination
InstanceID string
TableName string
ParentType domain.CountParentType
ParentID string
Resource string
UpdatedAt time.Time
Amount int
}
// ListResourceCounts retrieves all resource counts.
// It supports pagination using lastID and limit parameters.
//
// TODO: Currently only a proof of concept, filters may be implemented later if required.
func (q *Queries) ListResourceCounts(ctx context.Context, lastID, limit int) (result []ResourceCount, err error) {
ctx, span := tracing.NewSpan(ctx)
defer func() { span.EndWithError(err) }()
err = q.client.QueryContext(ctx, func(rows *sql.Rows) error {
for rows.Next() {
var count ResourceCount
err := rows.Scan(
&count.ID,
&count.InstanceID,
&count.TableName,
&count.ParentType,
&count.ParentID,
&count.Resource,
&count.UpdatedAt,
&count.Amount)
if err != nil {
return zerrors.ThrowInternal(err, "QUERY-2f4g5", "Errors.Internal")
}
result = append(result, count)
}
return nil
}, resourceCountsListQuery, lastID, limit)
if err != nil {
return nil, zerrors.ThrowInternal(err, "QUERY-3f4g5", "Errors.Internal")
}
return result, nil
}