feat: label policy (#1708)

* feat: label policy proto extension * feat: label policy and activate event * feat: label policy asset events * feat: label policy asset commands * feat: add storage key * feat: storage key validation * feat: label policy asset tests * feat: label policy query side * feat: avatar * feat: avatar event * feat: human avatar * feat: avatar read side * feat: font on iam label policy * feat: label policy font * feat: possiblity to create bucket on put file * uplaoder * login policy logo * set bucket prefix * feat: avatar upload * feat: avatar upload * feat: use assets on command side * feat: fix human avatar removed event * feat: remove human avatar * feat: mock asset storage * feat: remove human avatar * fix(operator): add configuration of asset storage to zitadel operator * feat(console): private labeling policy (#1697) * private labeling component, routing, preview * font, colors, upload, i18n * show logo * fix: uniqueness (#1710) * fix: uniqueconstraint to lower * feat: change org * feat: org change test * feat: change org * fix: tests * fix: handle domain claims correctly * feat: update org Co-authored-by: fabi <fabienne.gerschwiler@gmail.com> * fix: handle domain claimed event correctly for service users (#1711) * fix: handle domain claimed event correctly on user view * fix: ignore domain claimed events for email notifications * fix: change org * handle org changed in read models correctly * fix: change org in user grant handler Co-authored-by: fabi <fabienne.gerschwiler@gmail.com> * fix: correct value (#1695) * docs(api): correct link (#1712) * upload service Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: fabi <fabienne.gerschwiler@gmail.com> Co-authored-by: Florian Forster <florian@caos.ch> * feat: fix tests, * feat: remove assets from label policy * fix npm, set environment * lint ts * remove stylelinting * fix(operator): add mapping for console with changed unit tests * fix(operator): add secrets as env variables to pod * feat: remove human avatar * fix(operator): add secrets as env variables to pod * feat: map label policy * feat: labelpolicy, admin, mgmt, adv settings (#1715) * fetch label policy, mgmt, admin service * feat: advanced beh, links, add, update * lint ts * feat: watermark * feat: remove human avatar * feat: remove human avatar * feat: remove human avatar * feat: remove human avatar * feat: remove human avatar * feat: remove human avatar * feat: remove human avatar * feat: custom css * css * css * css * css * css * getobject * feat: dynamic handler * feat: varibale css * content info * css overwrite * feat: variablen css * feat: generate css file * feat: dark mode * feat: dark mode * fix logo css * feat: upload logos * dark mode with cookie * feat: handle images in login * avatar css and begin font * feat: avatar * feat: user avatar * caching of static assets in login * add avatar.js to main.html * feat: header dont show logo if no url * feat: label policy colors * feat: mock asset storage * feat: mock asset storage * feat: fix tests * feat: user avatar * feat: header logo * avatar * avatar * make it compatible with go 1.15 * feat: remove unused logos * fix handler * fix: styling error handling * fonts * fix: download func * switch to mux * fix: change upload api to assets * fix build * fix: download avatar * fix: download logos * fix: my avatar * font * fix: remove error msg popup possibility * fix: docs * fix: svalidate colors * rem msg popup from frontend * fix: email with private labeling * fix: tests * fix: email templates * fix: change migration version * fix: fix duplicate imports * fix(console): assets, service url, upload, policy current and preview (#1781) * upload endpoint, layout * fetch current, preview, fix upload * cleanup private labeling * fix linting * begin generated asset handler * generate asset api in dockerfile * features for label policy * features for label policy * features * flag for asset generator * change asset generator flag * fix label policy view in grpc * fix: layout, activate policy (#1786) * theme switcher up on top * change layout * activate policy * feat(console): label policy back color, layout (#1788) * theme switcher up on top * change layout * activate policy * fix overwrite value fc * reset policy, reset service * autosave policy, preview desc, layout impv * layout, i18n * background colors, inject material styles * load images * clean, lint * fix layout * set custom hex * fix content size conversion * remove font format in generated css * fix features for assets * fix(console): label policy colors, image downloads, preview (#1804) * load images * colors, images binding * lint * refresh emitter * lint * propagate font colors * upload error handling * label policy feature check * add blob in csp for console * log * fix: feature edits for label policy, refresh state on upload (#1807) * show error on load image, stop spinner * fix merge * fix migration versions * fix assets * fix csp * fix background color * scss * fix build * lint scss * fix statik for console * fix features check for label policy * cleanup * lint * public links * fix notifications * public links * feat: merge main * feat: fix translation files * fix migration * set api domain * fix logo in email * font face in email * font face in email * validate assets on upload * cleanup * add missing translations * add missing translations Co-authored-by: Livio Amstutz <livio.a@gmail.com> Co-authored-by: Stefan Benz <stefan@caos.ch> Co-authored-by: Max Peintner <max@caos.ch> Co-authored-by: Florian Forster <florian@caos.ch>
2025-08-11 21:17:32 +00:00 · 2021-06-04 14:53:51 +02:00
parent c0d9d86b09
commit 73d37459bb
257 changed files with 18248 additions and 7178 deletions
--- a/internal/static/s3/minio.go
+++ b/internal/static/s3/minio.go
@@ -2,46 +2,34 @@ package s3

 import (
 	"context"
-	"fmt"
 	"io"
+	"net/http"
 	"net/url"
+	"strings"
 	"time"

 	"github.com/caos/logging"
 	"github.com/minio/minio-go/v7"
-	"github.com/minio/minio-go/v7/pkg/credentials"

 	"github.com/caos/zitadel/internal/domain"
 	caos_errs "github.com/caos/zitadel/internal/errors"
 )

 type Minio struct {
-	Client   *minio.Client
-	Location string
-}
-
-func NewMinio(config S3Config) (*Minio, error) {
-	minioClient, err := minio.New(config.Endpoint, &minio.Options{
-		Creds:  credentials.NewStaticV4(config.AccessKeyID, config.SecretAccessKey, ""),
-		Secure: config.SSL,
-		Region: config.Location,
-	})
-	if err != nil {
-		return nil, caos_errs.ThrowInternal(err, "MINIO-4m90d", "Errors.Assets.Store.NotInitialized")
-	}
-	return &Minio{
-		Client:   minioClient,
-		Location: config.Location,
-	}, nil
+	Client       *minio.Client
+	Location     string
+	BucketPrefix string
 }

 func (m *Minio) CreateBucket(ctx context.Context, name, location string) error {
 	if location == "" {
 		location = m.Location
 	}
+	name = m.prefixBucketName(name)
 	exists, err := m.Client.BucketExists(ctx, name)
 	if err != nil {
-		return caos_errs.ThrowInternal(err, "MINIO-4m90d", "Errors.Assets.Bucket.Internal")
+		logging.LogWithFields("MINIO-ADvf3", "bucketname", name).WithError(err).Error("cannot check if bucket exists")
+		return caos_errs.ThrowInternal(err, "MINIO-1b8fs", "Errors.Assets.Bucket.Internal")
 	}
 	if exists {
 		return caos_errs.ThrowAlreadyExists(nil, "MINIO-9n3MK", "Errors.Assets.Bucket.AlreadyExists")
@@ -69,6 +57,7 @@ func (m *Minio) ListBuckets(ctx context.Context) ([]*domain.BucketInfo, error) {
 }

 func (m *Minio) RemoveBucket(ctx context.Context, name string) error {
+	name = m.prefixBucketName(name)
 	err := m.Client.RemoveBucket(ctx, name)
 	if err != nil {
 		return caos_errs.ThrowInternal(err, "MINIO-338Hs", "Errors.Assets.Bucket.RemoveFailed")
@@ -76,7 +65,14 @@ func (m *Minio) RemoveBucket(ctx context.Context, name string) error {
 	return nil
 }

-func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64) (*domain.AssetInfo, error) {
+func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64, createBucketIfNotExisting bool) (*domain.AssetInfo, error) {
+	if createBucketIfNotExisting {
+		err := m.CreateBucket(ctx, bucketName, "")
+		if err != nil && !caos_errs.IsErrorAlreadyExists(err) {
+			return nil, err
+		}
+	}
+	bucketName = m.prefixBucketName(bucketName)
 	info, err := m.Client.PutObject(ctx, bucketName, objectName, object, objectSize, minio.PutObjectOptions{ContentType: contentType})
 	if err != nil {
 		return nil, caos_errs.ThrowInternal(err, "MINIO-590sw", "Errors.Assets.Object.PutFailed")
@@ -93,20 +89,36 @@ func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentTy
 }

 func (m *Minio) GetObjectInfo(ctx context.Context, bucketName, objectName string) (*domain.AssetInfo, error) {
-	object, err := m.Client.GetObject(ctx, bucketName, objectName, minio.GetObjectOptions{})
+	bucketName = m.prefixBucketName(bucketName)
+	objectinfo, err := m.Client.StatObject(ctx, bucketName, objectName, minio.StatObjectOptions{})
 	if err != nil {
+		if errResp := minio.ToErrorResponse(err); errResp.StatusCode == http.StatusNotFound {
+			return nil, caos_errs.ThrowNotFound(err, "MINIO-Gdfh4", "Errors.Assets.Object.GetFailed")
+		}
 		return nil, caos_errs.ThrowInternal(err, "MINIO-1vySX", "Errors.Assets.Object.GetFailed")
 	}
-	info, err := object.Stat()
+	return m.objectToAssetInfo(bucketName, objectinfo), nil
+}
+
+func (m *Minio) GetObject(ctx context.Context, bucketName, objectName string) (io.Reader, func() (*domain.AssetInfo, error), error) {
+	bucketName = m.prefixBucketName(bucketName)
+	object, err := m.Client.GetObject(ctx, bucketName, objectName, minio.GetObjectOptions{})
 	if err != nil {
-		return nil, caos_errs.ThrowInternal(err, "MINIO-F96xF", "Errors.Assets.Object.GetFailed")
+		return nil, nil, caos_errs.ThrowInternal(err, "MINIO-VGDgv", "Errors.Assets.Object.GetFailed")
 	}
-	return m.objectToAssetInfo(bucketName, info), nil
+	info := func() (*domain.AssetInfo, error) {
+		info, err := object.Stat()
+		if err != nil {
+			return nil, caos_errs.ThrowInternal(err, "MINIO-F96xF", "Errors.Assets.Object.GetFailed")
+		}
+		return m.objectToAssetInfo(bucketName, info), nil
+	}
+	return object, info, nil
 }

 func (m *Minio) GetObjectPresignedURL(ctx context.Context, bucketName, objectName string, expiration time.Duration) (*url.URL, error) {
+	bucketName = m.prefixBucketName(bucketName)
 	reqParams := make(url.Values)
-	reqParams.Set("response-content-disposition", fmt.Sprintf("attachment; filename=\"%s\"", objectName))
 	presignedURL, err := m.Client.PresignedGetObject(ctx, bucketName, objectName, expiration, reqParams)
 	if err != nil {
 		return nil, caos_errs.ThrowInternal(err, "MINIO-19Mp0", "Errors.Assets.Object.PresignedTokenFailed")
@@ -115,6 +127,7 @@ func (m *Minio) GetObjectPresignedURL(ctx context.Context, bucketName, objectNam
 }

 func (m *Minio) ListObjectInfos(ctx context.Context, bucketName, prefix string, recursive bool) ([]*domain.AssetInfo, error) {
+	bucketName = m.prefixBucketName(bucketName)
 	ctx, cancel := context.WithCancel(ctx)
 	defer cancel()

@@ -134,6 +147,7 @@ func (m *Minio) ListObjectInfos(ctx context.Context, bucketName, prefix string,
 }

 func (m *Minio) RemoveObject(ctx context.Context, bucketName, objectName string) error {
+	bucketName = m.prefixBucketName(bucketName)
 	err := m.Client.RemoveObject(ctx, bucketName, objectName, minio.RemoveObjectOptions{})
 	if err != nil {
 		return caos_errs.ThrowInternal(err, "MINIO-x85RT", "Errors.Assets.Object.RemoveFailed")
@@ -149,7 +163,12 @@ func (m *Minio) objectToAssetInfo(bucketName string, object minio.ObjectInfo) *d
 		Size:            object.Size,
 		LastModified:    object.LastModified,
 		VersionID:       object.VersionID,
-		Expiration:      object.Expiration,
-		AutheticatedURL: m.Client.EndpointURL().String() + "/" + object.Key,
+		Expiration:      object.Expires,
+		ContentType:     object.ContentType,
+		AutheticatedURL: m.Client.EndpointURL().String() + "/" + bucketName + "/" + object.Key,
 	}
 }
+
+func (m *Minio) prefixBucketName(name string) string {
+	return strings.ToLower(m.BucketPrefix + "-" + name)
+}