feat: label policy (#1708)

* feat: label policy proto extension

* feat: label policy and activate event

* feat: label policy asset events

* feat: label policy asset commands

* feat: add storage key

* feat: storage key validation

* feat: label policy asset tests

* feat: label policy query side

* feat: avatar

* feat: avatar event

* feat: human avatar

* feat: avatar read side

* feat: font on iam label policy

* feat: label policy font

* feat: possiblity to create bucket on put file

* uplaoder

* login policy logo

* set bucket prefix

* feat: avatar upload

* feat: avatar upload

* feat: use assets on command side

* feat: fix human avatar removed event

* feat: remove human avatar

* feat: mock asset storage

* feat: remove human avatar

* fix(operator): add configuration of asset storage to zitadel operator

* feat(console): private labeling policy (#1697)

* private labeling component, routing, preview

* font, colors, upload, i18n

* show logo

* fix: uniqueness (#1710)

* fix: uniqueconstraint to lower

* feat: change org

* feat: org change test

* feat: change org

* fix: tests

* fix: handle domain claims correctly

* feat: update org

Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>

* fix: handle domain claimed event correctly for service users (#1711)

* fix: handle domain claimed event correctly on user view

* fix: ignore domain claimed events for email notifications

* fix: change org

* handle org changed in read models correctly

* fix: change org in user grant handler

Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>

* fix: correct value (#1695)

* docs(api): correct link (#1712)

* upload service

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: fabi <fabienne.gerschwiler@gmail.com>
Co-authored-by: Florian Forster <florian@caos.ch>

* feat: fix tests,

* feat: remove assets from label policy

* fix npm, set environment

* lint ts

* remove stylelinting

* fix(operator): add mapping for console with changed unit tests

* fix(operator): add secrets as env variables to pod

* feat: remove human avatar

* fix(operator): add secrets as env variables to pod

* feat: map label policy

* feat: labelpolicy, admin, mgmt, adv settings (#1715)

* fetch label policy, mgmt, admin service

* feat: advanced beh, links, add, update

* lint ts

* feat: watermark

* feat: remove human avatar

* feat: remove human avatar

* feat: remove human avatar

* feat: remove human avatar

* feat: remove human avatar

* feat: remove human avatar

* feat: remove human avatar

* feat: custom css

* css

* css

* css

* css

* css

* getobject

* feat: dynamic handler

* feat: varibale css

* content info

* css overwrite

* feat: variablen css

* feat: generate css file

* feat: dark mode

* feat: dark mode

* fix logo css

* feat: upload logos

* dark mode with cookie

* feat: handle images in login

* avatar css and begin font

* feat: avatar

* feat: user avatar

* caching of static assets in login

* add avatar.js to main.html

* feat: header dont show logo if no url

* feat: label policy colors

* feat: mock asset storage

* feat: mock asset storage

* feat: fix tests

* feat: user avatar

* feat: header logo

* avatar

* avatar

* make it compatible with go 1.15

* feat: remove unused logos

* fix handler

* fix: styling error handling

* fonts

* fix: download func

* switch to mux

* fix: change upload api to assets

* fix build

* fix: download avatar

* fix: download logos

* fix: my avatar

* font

* fix: remove error msg popup possibility

* fix: docs

* fix: svalidate colors

* rem msg popup from frontend

* fix: email with private labeling

* fix: tests

* fix: email templates

* fix: change migration version

* fix: fix duplicate imports

* fix(console): assets, service url, upload, policy current and preview  (#1781)

* upload endpoint, layout

* fetch current, preview, fix upload

* cleanup private labeling

* fix linting

* begin generated asset handler

* generate asset api in dockerfile

* features for label policy

* features for label policy

* features

* flag for asset generator

* change asset generator flag

* fix label policy view in grpc

* fix: layout, activate policy (#1786)

* theme switcher up on top

* change layout

* activate policy

* feat(console): label policy back color, layout (#1788)

* theme switcher up on top

* change layout

* activate policy

* fix overwrite value fc

* reset policy, reset service

* autosave policy, preview desc, layout impv

* layout, i18n

* background colors, inject material styles

* load images

* clean, lint

* fix layout

* set custom hex

* fix content size conversion

* remove font format in generated css

* fix features for assets

* fix(console): label policy colors, image downloads, preview (#1804)

* load images

* colors, images binding

* lint

* refresh emitter

* lint

* propagate font colors

* upload error handling

* label policy feature check

* add blob in csp for console

* log

* fix: feature edits for label policy, refresh state on upload (#1807)

* show error on load image, stop spinner

* fix merge

* fix migration versions

* fix assets

* fix csp

* fix background color

* scss

* fix build

* lint scss

* fix statik for console

* fix features check for label policy

* cleanup

* lint

* public links

* fix notifications

* public links

* feat: merge main

* feat: fix translation files

* fix migration

* set api domain

* fix logo in email

* font face in email

* font face in email

* validate assets on upload

* cleanup

* add missing translations

* add missing translations

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
Co-authored-by: Stefan Benz <stefan@caos.ch>
Co-authored-by: Max Peintner <max@caos.ch>
Co-authored-by: Florian Forster <florian@caos.ch>
This commit is contained in:
Fabi
2021-06-04 14:53:51 +02:00
committed by GitHub
parent c0d9d86b09
commit 73d37459bb
257 changed files with 18248 additions and 7178 deletions

View File

@@ -0,0 +1,57 @@
package config
import (
"encoding/json"
"github.com/caos/zitadel/internal/errors"
"github.com/caos/zitadel/internal/static"
"github.com/caos/zitadel/internal/static/s3"
)
type AssetStorageConfig struct {
Type string
Config static.Config
}
var storage = map[string]func() static.Config{
"s3": func() static.Config { return &s3.Config{} },
}
func (c *AssetStorageConfig) UnmarshalJSON(data []byte) error {
var rc struct {
Type string
Config json.RawMessage
}
if err := json.Unmarshal(data, &rc); err != nil {
return errors.ThrowInternal(err, "STATIC-Bfn5r", "error parsing config")
}
c.Type = rc.Type
var err error
c.Config, err = newStorageConfig(c.Type, rc.Config)
if err != nil {
return err
}
return nil
}
func newStorageConfig(storageType string, configData []byte) (static.Config, error) {
t, ok := storage[storageType]
if !ok {
return nil, errors.ThrowInternalf(nil, "STATIC-dsbjh", "config type %s not supported", storageType)
}
staticConfig := t()
if len(configData) == 0 {
return staticConfig, nil
}
if err := json.Unmarshal(configData, staticConfig); err != nil {
return nil, errors.ThrowInternal(err, "STATIC-GB4nw", "Could not read config: %v")
}
return staticConfig, nil
}

View File

@@ -0,0 +1,3 @@
package static
//go:generate mockgen -source storage.go -destination ./mock/storage_mock.go -package mock

View File

@@ -5,14 +5,17 @@ Errors:
IDMissing: ID fehlt
ResourceOwnerMissing: Organisation fehlt
Assets:
EmptyKey: Asset Key ist leer
Store:
NotInitialized: Assets Speicher konnte nicht initialisiert werden
NotConfigured: Assets Speicher wurde nicht konfiguriert
Bucket:
Internal: Interner Fehler beim erstellen eines Buckets
AlreadyExists: Bucket existiert bereits
CreateFailed: Bucket konnte nicht erstellt werden
ListFailed: Buckets konnten nicht gelesen werden
RemoveFailed: Bucket konnte nicht gelöscht werden
SetPublicFailed: Bucket konnte nicht auf public gesetzt werden
Object:
PutFailed: Objekt konnte nicht erstellt werden
GetFaieled: Objekt konnte nicht gelesen werden
@@ -247,6 +250,7 @@ Errors:
IdpIsNotOIDC: IDP Konfiguration ist nicht vom Typ OIDC
LoginPolicyInvalid: Login Policy ist ungültig
LoginPolicyNotExisting: Login Policy nicht vorhanden
IdpProviderInvalid: IDP Provider ist ungültig
LoginPolicy:
NotFound: Default Login Policy konnte nicht gefunden
NotChanged: Default Login Policy wurde nicht verändert
@@ -303,6 +307,16 @@ Errors:
NotChanged: Default Org IAM Policy wurde nicht verändert
Policy:
AlreadyExists: Policy existiert bereits
Label:
Invalid:
PrimaryColor: Primäre Farbe ist kein gültiger Hex Farbwert
BackgroundColor: Hintergrund Farbe ist kein gültiger Hex Farbwert
WarnColor: Warn Farbe ist kein gültiger Hex Farbwert
FontColor: Schrift Farbe ist kein gültiger Hex Farbwert
PrimaryColorDark: Primäre Farbe (dunkler Modus) ist kein gültiger Hex Farbwert
BackgroundColorDark: Hintergrund Farbe (dunkler Modus) ist kein gültiger Hex Farbwert
WarnColorDark: Warn Farbe (dunkler Modus) ist kein gültiger Hex Farbwert
FontColorDark: Schrift Farbe (dunkler Modus) ist kein gültiger Hex Farbwert
UserGrant:
AlreadyExists: Benutzer Berechtigung existiert bereits
NotFound: Benutzer Berechtigung konnte nicht gefunden werden
@@ -372,6 +386,9 @@ EventTypes:
human:
added: Benutzer hinzugefügt
selfregistered: Benutzer hat sich selbst registriert
avatar:
added: Avatar hinzugefügt
removed: Avatar entfernt
initialization:
code:
added: Initialisierungscode generiert
@@ -580,7 +597,25 @@ EventTypes:
label:
added: Label Richtline hinzugefügt
changed: Label Richtline geändert
activated: Label Richtline aktiviert
removed: Label Richtline entfernt
logo:
added: Logo zu Label Richtlinie hinzugefügt
removed: Logo von Label Richtlinie entfernt
dark:
added: Logo (dunkler Modus) zu Label Richtlinie hinzugefügt
removed: Logo (dunkler Modus) von Label Richtlinie entfernt
icon:
added: Icon zu Label Richtlinie hinzugefügt
removed: Icon von Label Richtlinie entfernt
dark:
added: Icon (dunkler Modus) zu Label Richtlinie hinzugefügt
removed: Icon (dunkler Modus) von Label Richtlinie entfernt
font:
added: Schrift zu Label Richtlinie hinzugefügt
removed: Schrift von Label Richtlinie entfernt
assets:
removed: Dateien von Label Richtlinie entfernt
project:
added: Projekt hinzugefügt
changed: Project geändert
@@ -679,6 +714,27 @@ EventTypes:
idpprovider:
added: Idp Provider zu Default Login Policy hinzugefügt
removed: Idp Provider aus Default Login Policy gelöscht
label:
added: Label Richtline hinzugefügt
changed: Label Richtline geändert
activated: Label Richtline aktiviert
logo:
added: Logo zu Label Richtlinie hinzugefügt
removed: Logo von Label Richtlinie entfernt
dark:
added: Logo (dunkler Modus) zu Label Richtlinie hinzugefügt
removed: Logo (dunkler Modus) von Label Richtlinie entfernt
icon:
added: Icon zu Label Richtlinie hinzugefügt
removed: Icon von Label Richtlinie entfernt
dark:
added: Icon (dunkler Modus) zu Label Richtlinie hinzugefügt
removed: Icon (dunkler Modus) von Label Richtlinie entfernt
font:
added: Schrift zu Label Richtlinie hinzugefügt
removed: Schrift von Label Richtlinie entfernt
assets:
removed: Bilder und Schrift von Label Richtlinie entfernt
key_pair:
added: Schlüsselpaar hinzugefügt
Application:

View File

@@ -5,14 +5,17 @@ Errors:
IDMissing: ID missing
ResourceOwnerMissing: Resource Owner Organisation missing
Assets:
EmptyKey: Asset key is empty
Store:
NotInitialized: Assets storage not initialized
NotConfigured: Assets storage not configured
Bucket:
Internal: Internal error on create bucket
AlreadyExists: Bucket already exists
CreateFailed: Bucket not created
ListFailed: Buckets could not be read
RemoveFailed: Bucket not deleted
SetPublicFailed: Could not set bucket to public
Object:
PutFailed: Objekt not created
GetFaieled: Objekt could not be read
@@ -304,6 +307,16 @@ Errors:
NotChanged: Org IAM Policy has not been changed
Policy:
AlreadyExists: Policy already exists
Label:
Invalid:
PrimaryColor: Primary color is no valid Hex color value
BackgroundColor: Background color is no valid Hex color value
WarnColor: Warn color is no valid Hex color value
FontColor: Font color is no valid Hex color value
PrimaryColorDark: Primary color (dark mode) is no valid Hex color value
BackgroundColorDark: Background color (dark mode) is no valid Hex color value
WarnColorDark: Warn color (dark mode) is no valid Hex color value
FontColorDark: Font color (dark mode) is no valid Hex color value
UserGrant:
AlreadyExists: User grant already exists
NotFound: User grant not found
@@ -373,6 +386,9 @@ EventTypes:
human:
added: Person added
selfregistered: Person registered himself
avatar:
added: Avatar added
removed: Avatar removed
initialization:
code:
added: Initialisation code generated
@@ -581,7 +597,25 @@ EventTypes:
label:
added: Label Policy added
changed: Label Policy changed
activated: Label Policy activated
removed: Label Policy removed
logo:
added: Logo added to Label Policy
removed: Logo removed from Label Policy
dark:
added: Logo (dark mode) added to Label Policy
removed: Logo (dark mode) removed from Label Policy
icon:
added: Icon added to Label Policy
removed: Icon removed from Label Policy
dark:
added: Icon (dark mode) added to Label Policy
removed: Icon (dark mode) removed from Label Policy
font:
added: Font added to Label Policy
removed: Font removed from Label Policy
assets:
removed: Assets removed from Label Policy
project:
added: Project added
changed: Project changed
@@ -680,6 +714,27 @@ EventTypes:
idpprovider:
added: Idp Provider added to Default Login Policy
removed: Idp Provider removed from Default Login Policy
label:
added: Label Policy added
changed: Label Policy changed
activated: Label Policy activated
logo:
added: Logo added to Label Policy
removed: Logo removed from Label Policy
dark:
added: Logo (dark mode) added to Label Policy
removed: Logo (dark mode) removed from Label Policy
icon:
added: Icon added to Label Policy
removed: Icon removed from Label Policy
dark:
added: Icon (dark mode) added to Label Policy
removed: Icon (dark mode) removed from Label Policy
font:
added: Font added to Label Policy
removed: Font removed from Label Policy
assets:
removed: Assets removed from Label Policy
key_pair:
added: Key pair added
Application:

View File

@@ -0,0 +1,210 @@
// Code generated by MockGen. DO NOT EDIT.
// Source: storage.go
// Package mock is a generated GoMock package.
package mock
import (
context "context"
domain "github.com/caos/zitadel/internal/domain"
static "github.com/caos/zitadel/internal/static"
gomock "github.com/golang/mock/gomock"
io "io"
url "net/url"
reflect "reflect"
time "time"
)
// MockStorage is a mock of Storage interface
type MockStorage struct {
ctrl *gomock.Controller
recorder *MockStorageMockRecorder
}
// MockStorageMockRecorder is the mock recorder for MockStorage
type MockStorageMockRecorder struct {
mock *MockStorage
}
// NewMockStorage creates a new mock instance
func NewMockStorage(ctrl *gomock.Controller) *MockStorage {
mock := &MockStorage{ctrl: ctrl}
mock.recorder = &MockStorageMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockStorage) EXPECT() *MockStorageMockRecorder {
return m.recorder
}
// CreateBucket mocks base method
func (m *MockStorage) CreateBucket(ctx context.Context, name, location string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "CreateBucket", ctx, name, location)
ret0, _ := ret[0].(error)
return ret0
}
// CreateBucket indicates an expected call of CreateBucket
func (mr *MockStorageMockRecorder) CreateBucket(ctx, name, location interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CreateBucket", reflect.TypeOf((*MockStorage)(nil).CreateBucket), ctx, name, location)
}
// RemoveBucket mocks base method
func (m *MockStorage) RemoveBucket(ctx context.Context, name string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "RemoveBucket", ctx, name)
ret0, _ := ret[0].(error)
return ret0
}
// RemoveBucket indicates an expected call of RemoveBucket
func (mr *MockStorageMockRecorder) RemoveBucket(ctx, name interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemoveBucket", reflect.TypeOf((*MockStorage)(nil).RemoveBucket), ctx, name)
}
// ListBuckets mocks base method
func (m *MockStorage) ListBuckets(ctx context.Context) ([]*domain.BucketInfo, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ListBuckets", ctx)
ret0, _ := ret[0].([]*domain.BucketInfo)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// ListBuckets indicates an expected call of ListBuckets
func (mr *MockStorageMockRecorder) ListBuckets(ctx interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListBuckets", reflect.TypeOf((*MockStorage)(nil).ListBuckets), ctx)
}
// PutObject mocks base method
func (m *MockStorage) PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64, createBucketIfNotExisting bool) (*domain.AssetInfo, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "PutObject", ctx, bucketName, objectName, contentType, object, objectSize, createBucketIfNotExisting)
ret0, _ := ret[0].(*domain.AssetInfo)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// PutObject indicates an expected call of PutObject
func (mr *MockStorageMockRecorder) PutObject(ctx, bucketName, objectName, contentType, object, objectSize, createBucketIfNotExisting interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "PutObject", reflect.TypeOf((*MockStorage)(nil).PutObject), ctx, bucketName, objectName, contentType, object, objectSize, createBucketIfNotExisting)
}
// GetObjectInfo mocks base method
func (m *MockStorage) GetObjectInfo(ctx context.Context, bucketName, objectName string) (*domain.AssetInfo, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetObjectInfo", ctx, bucketName, objectName)
ret0, _ := ret[0].(*domain.AssetInfo)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetObjectInfo indicates an expected call of GetObjectInfo
func (mr *MockStorageMockRecorder) GetObjectInfo(ctx, bucketName, objectName interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetObjectInfo", reflect.TypeOf((*MockStorage)(nil).GetObjectInfo), ctx, bucketName, objectName)
}
// GetObject mocks base method
func (m *MockStorage) GetObject(ctx context.Context, bucketName, objectName string) (io.Reader, func() (*domain.AssetInfo, error), error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetObject", ctx, bucketName, objectName)
ret0, _ := ret[0].(io.Reader)
ret1, _ := ret[1].(func() (*domain.AssetInfo, error))
ret2, _ := ret[2].(error)
return ret0, ret1, ret2
}
// GetObject indicates an expected call of GetObject
func (mr *MockStorageMockRecorder) GetObject(ctx, bucketName, objectName interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetObject", reflect.TypeOf((*MockStorage)(nil).GetObject), ctx, bucketName, objectName)
}
// ListObjectInfos mocks base method
func (m *MockStorage) ListObjectInfos(ctx context.Context, bucketName, prefix string, recursive bool) ([]*domain.AssetInfo, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ListObjectInfos", ctx, bucketName, prefix, recursive)
ret0, _ := ret[0].([]*domain.AssetInfo)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// ListObjectInfos indicates an expected call of ListObjectInfos
func (mr *MockStorageMockRecorder) ListObjectInfos(ctx, bucketName, prefix, recursive interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ListObjectInfos", reflect.TypeOf((*MockStorage)(nil).ListObjectInfos), ctx, bucketName, prefix, recursive)
}
// GetObjectPresignedURL mocks base method
func (m *MockStorage) GetObjectPresignedURL(ctx context.Context, bucketName, objectName string, expiration time.Duration) (*url.URL, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "GetObjectPresignedURL", ctx, bucketName, objectName, expiration)
ret0, _ := ret[0].(*url.URL)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// GetObjectPresignedURL indicates an expected call of GetObjectPresignedURL
func (mr *MockStorageMockRecorder) GetObjectPresignedURL(ctx, bucketName, objectName, expiration interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "GetObjectPresignedURL", reflect.TypeOf((*MockStorage)(nil).GetObjectPresignedURL), ctx, bucketName, objectName, expiration)
}
// RemoveObject mocks base method
func (m *MockStorage) RemoveObject(ctx context.Context, bucketName, objectName string) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "RemoveObject", ctx, bucketName, objectName)
ret0, _ := ret[0].(error)
return ret0
}
// RemoveObject indicates an expected call of RemoveObject
func (mr *MockStorageMockRecorder) RemoveObject(ctx, bucketName, objectName interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "RemoveObject", reflect.TypeOf((*MockStorage)(nil).RemoveObject), ctx, bucketName, objectName)
}
// MockConfig is a mock of Config interface
type MockConfig struct {
ctrl *gomock.Controller
recorder *MockConfigMockRecorder
}
// MockConfigMockRecorder is the mock recorder for MockConfig
type MockConfigMockRecorder struct {
mock *MockConfig
}
// NewMockConfig creates a new mock instance
func NewMockConfig(ctrl *gomock.Controller) *MockConfig {
mock := &MockConfig{ctrl: ctrl}
mock.recorder = &MockConfigMockRecorder{mock}
return mock
}
// EXPECT returns an object that allows the caller to indicate expected use
func (m *MockConfig) EXPECT() *MockConfigMockRecorder {
return m.recorder
}
// NewStorage mocks base method
func (m *MockConfig) NewStorage() (static.Storage, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "NewStorage")
ret0, _ := ret[0].(static.Storage)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// NewStorage indicates an expected call of NewStorage
func (mr *MockConfigMockRecorder) NewStorage() *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "NewStorage", reflect.TypeOf((*MockConfig)(nil).NewStorage))
}

View File

@@ -0,0 +1,41 @@
package mock
import (
"testing"
"github.com/golang/mock/gomock"
caos_errors "github.com/caos/zitadel/internal/errors"
)
func NewStorage(t *testing.T) *MockStorage {
return NewMockStorage(gomock.NewController(t))
}
func (m *MockStorage) ExpectAddObjectNoError() *MockStorage {
m.EXPECT().
PutObject(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).
Return(nil, nil)
return m
}
func (m *MockStorage) ExpectAddObjectError() *MockStorage {
m.EXPECT().
PutObject(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).
Return(nil, caos_errors.ThrowInternal(nil, "", ""))
return m
}
func (m *MockStorage) ExpectRemoveObjectNoError() *MockStorage {
m.EXPECT().
RemoveObject(gomock.Any(), gomock.Any(), gomock.Any()).
Return(nil)
return m
}
func (m *MockStorage) ExpectRemoveObjectError() *MockStorage {
m.EXPECT().
RemoveObject(gomock.Any(), gomock.Any(), gomock.Any()).
Return(caos_errors.ThrowInternal(nil, "", ""))
return m
}

View File

@@ -1,14 +1,34 @@
package s3
type AssetStorage struct {
Type string
Config S3Config
}
import (
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
type S3Config struct {
caos_errs "github.com/caos/zitadel/internal/errors"
"github.com/caos/zitadel/internal/static"
)
type Config struct {
Endpoint string
AccessKeyID string
SecretAccessKey string
SSL bool
Location string
BucketPrefix string
}
func (c *Config) NewStorage() (static.Storage, error) {
minioClient, err := minio.New(c.Endpoint, &minio.Options{
Creds: credentials.NewStaticV4(c.AccessKeyID, c.SecretAccessKey, ""),
Secure: c.SSL,
Region: c.Location,
})
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-2n9fs", "Errors.Assets.Store.NotInitialized")
}
return &Minio{
Client: minioClient,
Location: c.Location,
BucketPrefix: c.BucketPrefix,
}, nil
}

View File

@@ -2,46 +2,34 @@ package s3
import (
"context"
"fmt"
"io"
"net/http"
"net/url"
"strings"
"time"
"github.com/caos/logging"
"github.com/minio/minio-go/v7"
"github.com/minio/minio-go/v7/pkg/credentials"
"github.com/caos/zitadel/internal/domain"
caos_errs "github.com/caos/zitadel/internal/errors"
)
type Minio struct {
Client *minio.Client
Location string
}
func NewMinio(config S3Config) (*Minio, error) {
minioClient, err := minio.New(config.Endpoint, &minio.Options{
Creds: credentials.NewStaticV4(config.AccessKeyID, config.SecretAccessKey, ""),
Secure: config.SSL,
Region: config.Location,
})
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-4m90d", "Errors.Assets.Store.NotInitialized")
}
return &Minio{
Client: minioClient,
Location: config.Location,
}, nil
Client *minio.Client
Location string
BucketPrefix string
}
func (m *Minio) CreateBucket(ctx context.Context, name, location string) error {
if location == "" {
location = m.Location
}
name = m.prefixBucketName(name)
exists, err := m.Client.BucketExists(ctx, name)
if err != nil {
return caos_errs.ThrowInternal(err, "MINIO-4m90d", "Errors.Assets.Bucket.Internal")
logging.LogWithFields("MINIO-ADvf3", "bucketname", name).WithError(err).Error("cannot check if bucket exists")
return caos_errs.ThrowInternal(err, "MINIO-1b8fs", "Errors.Assets.Bucket.Internal")
}
if exists {
return caos_errs.ThrowAlreadyExists(nil, "MINIO-9n3MK", "Errors.Assets.Bucket.AlreadyExists")
@@ -69,6 +57,7 @@ func (m *Minio) ListBuckets(ctx context.Context) ([]*domain.BucketInfo, error) {
}
func (m *Minio) RemoveBucket(ctx context.Context, name string) error {
name = m.prefixBucketName(name)
err := m.Client.RemoveBucket(ctx, name)
if err != nil {
return caos_errs.ThrowInternal(err, "MINIO-338Hs", "Errors.Assets.Bucket.RemoveFailed")
@@ -76,7 +65,14 @@ func (m *Minio) RemoveBucket(ctx context.Context, name string) error {
return nil
}
func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64) (*domain.AssetInfo, error) {
func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64, createBucketIfNotExisting bool) (*domain.AssetInfo, error) {
if createBucketIfNotExisting {
err := m.CreateBucket(ctx, bucketName, "")
if err != nil && !caos_errs.IsErrorAlreadyExists(err) {
return nil, err
}
}
bucketName = m.prefixBucketName(bucketName)
info, err := m.Client.PutObject(ctx, bucketName, objectName, object, objectSize, minio.PutObjectOptions{ContentType: contentType})
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-590sw", "Errors.Assets.Object.PutFailed")
@@ -93,20 +89,36 @@ func (m *Minio) PutObject(ctx context.Context, bucketName, objectName, contentTy
}
func (m *Minio) GetObjectInfo(ctx context.Context, bucketName, objectName string) (*domain.AssetInfo, error) {
object, err := m.Client.GetObject(ctx, bucketName, objectName, minio.GetObjectOptions{})
bucketName = m.prefixBucketName(bucketName)
objectinfo, err := m.Client.StatObject(ctx, bucketName, objectName, minio.StatObjectOptions{})
if err != nil {
if errResp := minio.ToErrorResponse(err); errResp.StatusCode == http.StatusNotFound {
return nil, caos_errs.ThrowNotFound(err, "MINIO-Gdfh4", "Errors.Assets.Object.GetFailed")
}
return nil, caos_errs.ThrowInternal(err, "MINIO-1vySX", "Errors.Assets.Object.GetFailed")
}
info, err := object.Stat()
return m.objectToAssetInfo(bucketName, objectinfo), nil
}
func (m *Minio) GetObject(ctx context.Context, bucketName, objectName string) (io.Reader, func() (*domain.AssetInfo, error), error) {
bucketName = m.prefixBucketName(bucketName)
object, err := m.Client.GetObject(ctx, bucketName, objectName, minio.GetObjectOptions{})
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-F96xF", "Errors.Assets.Object.GetFailed")
return nil, nil, caos_errs.ThrowInternal(err, "MINIO-VGDgv", "Errors.Assets.Object.GetFailed")
}
return m.objectToAssetInfo(bucketName, info), nil
info := func() (*domain.AssetInfo, error) {
info, err := object.Stat()
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-F96xF", "Errors.Assets.Object.GetFailed")
}
return m.objectToAssetInfo(bucketName, info), nil
}
return object, info, nil
}
func (m *Minio) GetObjectPresignedURL(ctx context.Context, bucketName, objectName string, expiration time.Duration) (*url.URL, error) {
bucketName = m.prefixBucketName(bucketName)
reqParams := make(url.Values)
reqParams.Set("response-content-disposition", fmt.Sprintf("attachment; filename=\"%s\"", objectName))
presignedURL, err := m.Client.PresignedGetObject(ctx, bucketName, objectName, expiration, reqParams)
if err != nil {
return nil, caos_errs.ThrowInternal(err, "MINIO-19Mp0", "Errors.Assets.Object.PresignedTokenFailed")
@@ -115,6 +127,7 @@ func (m *Minio) GetObjectPresignedURL(ctx context.Context, bucketName, objectNam
}
func (m *Minio) ListObjectInfos(ctx context.Context, bucketName, prefix string, recursive bool) ([]*domain.AssetInfo, error) {
bucketName = m.prefixBucketName(bucketName)
ctx, cancel := context.WithCancel(ctx)
defer cancel()
@@ -134,6 +147,7 @@ func (m *Minio) ListObjectInfos(ctx context.Context, bucketName, prefix string,
}
func (m *Minio) RemoveObject(ctx context.Context, bucketName, objectName string) error {
bucketName = m.prefixBucketName(bucketName)
err := m.Client.RemoveObject(ctx, bucketName, objectName, minio.RemoveObjectOptions{})
if err != nil {
return caos_errs.ThrowInternal(err, "MINIO-x85RT", "Errors.Assets.Object.RemoveFailed")
@@ -149,7 +163,12 @@ func (m *Minio) objectToAssetInfo(bucketName string, object minio.ObjectInfo) *d
Size: object.Size,
LastModified: object.LastModified,
VersionID: object.VersionID,
Expiration: object.Expiration,
AutheticatedURL: m.Client.EndpointURL().String() + "/" + object.Key,
Expiration: object.Expires,
ContentType: object.ContentType,
AutheticatedURL: m.Client.EndpointURL().String() + "/" + bucketName + "/" + object.Key,
}
}
func (m *Minio) prefixBucketName(name string) string {
return strings.ToLower(m.BucketPrefix + "-" + name)
}

View File

@@ -1,4 +1,4 @@
package s3
package static
import (
"context"
@@ -9,13 +9,17 @@ import (
"github.com/caos/zitadel/internal/domain"
)
type Client interface {
type Storage interface {
CreateBucket(ctx context.Context, name, location string) error
RemoveBucket(ctx context.Context, name string) error
ListBuckets(ctx context.Context) ([]*domain.BucketInfo, error)
PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64) (*domain.AssetInfo, error)
PutObject(ctx context.Context, bucketName, objectName, contentType string, object io.Reader, objectSize int64, createBucketIfNotExisting bool) (*domain.AssetInfo, error)
GetObjectInfo(ctx context.Context, bucketName, objectName string) (*domain.AssetInfo, error)
ListObjectInfos(ctx context.Context, bucketName, prefix string) ([]*domain.AssetInfo, error)
GetObject(ctx context.Context, bucketName, objectName string) (io.Reader, func() (*domain.AssetInfo, error), error)
ListObjectInfos(ctx context.Context, bucketName, prefix string, recursive bool) ([]*domain.AssetInfo, error)
GetObjectPresignedURL(ctx context.Context, bucketName, objectName string, expiration time.Duration) (*url.URL, error)
RemoveObject(ctx context.Context, bucketName, objectName string) error
}
type Config interface {
NewStorage() (Storage, error)
}