TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2025-01-05 14:37:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore(contribute): add step by step guide (#1754)

Browse Source 
* chore(contributing): add startup

* init

* cleanup docker file

* local

* compose works

* markdowns

* add gateway start on readme

* readme done

* finish mds

* rename/delete compose files

* correct docker compose file name

* fix links,
update contribute,
split build/readme into separate files in /guides,
add zitadel startup

* fix(docker compose): allow .keys folder to not exist

* update md's

* use docker-compose instead of docker compose as --profile gets ignored

* write a message if create key

* copy openapi statik.go

* explain how to connect in quickstart

* Apply suggestions from code review

Co-authored-by: Livio Amstutz <livio.a@gmail.com>

* sremove subscription service from env.json

* Delete caos_local.sh

moved to build/local/local.env

Co-authored-by: Livio Amstutz <livio.a@gmail.com>
This commit is contained in:
Silvan 2021-06-01 09:55:08 +02:00 committed by GitHub
parent d61baadfd9
commit a6e4b537fe
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
28 changed files with 670 additions and 299 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
CONTRIBUTING.md
View File

@ -1,12 +1,36 @@
# How to contribute to ZITADEL
## **Did you find a bug?**
## Did you find a bug?
## **Want to contribute code?**
Please file an issue [here](https://github.com/caos/zitadel/issues/new?assignees=&labels=bug&template=bug_report.md&title=).
* Check out our [Dev Build Guide](build/README.md).
Bugs are evaluated every day as soon as possible.
## Enhancement
Do you miss a feature? Please file an issue [here](https://github.com/caos/zitadel/issues/new?assignees=&labels=enhancement&template=feature_request.md&title=)
Enhancements are discussed and evaluated every Wednesday by the ZITADEL core team.
## Grab an Issues
We add the label "good first issue" for problems we think are a good starting point to contribute to ZITADEL.
* [Issues for first time contributors](https://github.com/caos/zitadel/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22)
* [All issues](https://github.com/caos/zitadel/issues)
## Want to start ZITADEL?
Checkout the following chapters to start ZITADEL locally.
### Build for local development
* Check out our [development guide](guides/development.md).
### Quick start
* Check out our [quick start guide](guides/quickstart.md).
## **Did you find a security flaw?**
* Please read [Security Policy](SECURITY.md).

60
build/README.md
View File

@ -1,60 +0,0 @@
# Development
## Prerequisite
- Buildkit compatible docker installation
## Generate Proto Clients
### Angular
This command generates the grpc stub for angular into the folder console/src/app/proto/generated for local development
```Bash
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:local --target npm-copy -o .
```
### Go
With this command you can generate the stub for golang into the zitadel dir
```Bash
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:local --target go-copy -o .
```
## Run
### Run Angular
```Bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f build/docker-compose-dev.yml up --build angular
```
### Run Go
```Bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f build/docker-compose-dev.yml up --build go
```
### Fullstack including database
```Bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f build/docker-compose-dev.yml up --build
```
## Debug
### Debug Go
```Bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f build/docker-compose-debug.yml up --build go
```
## Production Build
This can also be run locally!
```Bash
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:local --build-arg ENV=prod
```

5
build/console/generate-static.sh
View File

@ -1,5 +0,0 @@
#! /bin/sh
set -eux
go generate internal/ui/console/statik/generate.go

30
build/docker-compose-debug.yml
View File

@ -1,30 +0,0 @@
version: "3.8"
services:
angular:
build:
context: ..
dockerfile: dockerfile
target: dev-angular-build
args:
ENV: dev
command: sh -c "ng serve --host 0.0.0.0"
ports:
- 4200:4200
go:
build:
context: ..
dockerfile: dockerfile
target: dev-go-build
args:
ENV: dev
command: dlv --listen=:2345 --headless=true --log=true --log-output=debugger,debuglineerr,gdbwire,lldbout,rpc --accept-multiclient --api-version=2 debug cmd/zitadel/main.go
ports:
- 2345:2345
- 50000:50000
db:
image: cockroachdb/cockroach:v20.2.0
command: start-single-node --insecure
ports:
- 8080:8080
- 26257:26257

31
build/docker-compose-dev.yml
View File

@ -1,31 +0,0 @@
version: "3.8"
services:
angular:
build:
context: ..
dockerfile: dockerfile
target: dev-angular-build
args:
ENV: dev
command: sh -c "ng serve --host 0.0.0.0"
ports:
- 4200:4200
go:
build:
context: ..
dockerfile: dockerfile
target: dev-go-build
args:
ENV: dev
command: go run cmd/zitadel/main.go
ports:
- 50000:50000
db:
image: cockroachdb/cockroach:v20.2.0
command: start-single-node --insecure
ports:
- 8080:8080
- 26257:26257
volumes:
- "../cockroach-data/zitadel1:/cockroach/cockroach-data"

81
build/dockerfile
View File

@ -11,6 +11,7 @@ ENV PROTOC_ARCH aarch_64
FROM alpine AS amd64-base
ENV PROTOC_ARCH x86_64
#######################
## This step sets up the folder structure,
## initalices go mods,
@ -24,7 +25,6 @@ ARG GRPC_WEB_VERSION=1.2.1
# no arm specific version available and x86 works fine at the moment:
ARG GRPC_WEB=protoc-gen-grpc-web-${GRPC_WEB_VERSION}-linux-x86_64
RUN apk add tar curl
WORKDIR /proto
@ -51,7 +51,6 @@ RUN curl https://raw.githubusercontent.com/envoyproxy/protoc-gen-validate/v0.4.1
COPY proto/ include/.
#######################
## With this step we prepare all node_modules, this helps caching the build
## Speed up this step by mounting your local node_modules directory
@ -75,12 +74,14 @@ RUN build/console/generate-grpc.sh
FROM scratch as npm-copy
COPY --from=npm-base /console/src/app/proto/generated ./console/src/app/proto/generated
#######################
## angular dev build
#######################
FROM npm-base as dev-angular-build
RUN npm install -g @angular/cli
#######################
## angular lint workspace and prod build
#######################
@ -88,44 +89,79 @@ FROM npm-base as prod-angular-build
RUN npm run lint
RUN npm run prodbuild
#######################
## Go dependencies
## Speed up this step by mounting your local go mod pkg directory
#######################
FROM golang:${GO_VERSION} as go-dep
RUN mkdir -p src/github.com/caos/zitadel
# copy mod definitions
COPY tools src/github.com/caos/zitadel/tools
COPY ./go.* src/github.com/caos/zitadel
# install all dependencies
WORKDIR /go/src/github.com/caos/zitadel
#download modules
COPY ./go.* .
RUN go mod download
# install tools
COPY tools ./tools
RUN ./tools/install.sh
FROM go-dep AS go-gen
#######################
## generates static files
#######################
FROM go-dep AS go-static
COPY internal/ui/login/static internal/ui/login/static
COPY internal/ui/login/statik internal/ui/login/statik
COPY internal/notification/static internal/notification/static
COPY internal/notification/statik internal/notification/statik
COPY internal/static internal/static
COPY internal/statik internal/statik
RUN go generate internal/ui/login/statik/generate.go \
&& go generate internal/ui/login/static/generate.go \
&& go generate internal/notification/statik/generate.go \
&& go generate internal/statik/generate.go
#######################
## generates grpc stub
#######################
FROM go-static AS go-stub
COPY --from=base /proto /proto
COPY --from=base /usr/local/bin /usr/local/bin/.
COPY build/zitadel/generate-grpc.sh build/zitadel/generate-grpc.sh
COPY internal/protoc internal/protoc
RUN build/zitadel/generate-grpc.sh
COPY openapi/statik openapi/statik
RUN build/zitadel/generate-grpc.sh \
&& go generate openapi/statik/generate.go
#######################
## Go base build
#######################
FROM go-gen as go-base
# copy all zitadel files
FROM go-stub as go-base
# copy remaining zitadel files
COPY . .
#######################
## copy for local dev
#######################
FROM scratch as go-copy
COPY --from=go-gen /go/src/github.com/caos/zitadel/pkg/grpc ./pkg/grpc
COPY --from=go-gen /go/src/github.com/caos/zitadel/openapi/v2/zitadel ./openapi/v2/zitadel
COPY --from=go-gen /go/src/github.com/caos/zitadel/internal/protoc/protoc-gen-authoption/templates.gen.go ./internal/protoc/protoc-gen-authoption/templates.gen.go
COPY --from=go-gen /go/src/github.com/caos/zitadel/internal/protoc/protoc-gen-authoption/authoption/options.pb.go ./internal/protoc/protoc-gen-authoption/authoption/options.pb.go
COPY --from=go-gen /go/src/github.com/caos/zitadel/docs/apis/proto ./docs/docs/apis/proto
COPY --from=go-static /go/src/github.com/caos/zitadel/internal/ui/login/statik/statik.go internal/ui/login/statik/statik.go
COPY --from=go-static /go/src/github.com/caos/zitadel/internal/notification/statik/statik.go internal/notification/statik/statik.go
COPY --from=go-static /go/src/github.com/caos/zitadel/internal/statik/statik.go internal/statik/statik.go
COPY --from=go-static /go/src/github.com/caos/zitadel/openapi/statik/statik.go openapi/statik/statik.go
COPY --from=go-stub /go/src/github.com/caos/zitadel/pkg/grpc pkg/grpc
COPY --from=go-stub /go/src/github.com/caos/zitadel/openapi/v2/zitadel openapi/v2/zitadel
COPY --from=go-stub /go/src/github.com/caos/zitadel/openapi/statik/statik.go openapi/statik/statik.go
COPY --from=go-stub /go/src/github.com/caos/zitadel/internal/protoc/protoc-gen-authoption/templates.gen.go internal/protoc/protoc-gen-authoption/templates.gen.go
COPY --from=go-stub /go/src/github.com/caos/zitadel/internal/protoc/protoc-gen-authoption/authoption/options.pb.go internal/protoc/protoc-gen-authoption/authoption/options.pb.go
COPY --from=go-stub /go/src/github.com/caos/zitadel/docs/apis/proto docs/docs/apis/proto
#######################
@ -151,13 +187,11 @@ COPY --from=go-test /go/src/github.com/caos/zitadel/profile.cov profile.cov
#######################
FROM go-test as prod-go-build
ARG BUILDARCH
#generate statik code for console
COPY --from=prod-angular-build console/dist/console console/dist/console/
RUN go get github.com/rakyll/statik \
&& ./build/console/generate-static.sh \
&& ./build/login/generate-static.sh \
&& ./build/notification/generate-static.sh \
&& ./build/zitadel/generate-static.sh \
&& ./build/zitadel/generate-openapi-static.sh
RUN go generate internal/statik/generate.go
RUN CGO_ENABLED=0 GOOS=linux GOARCH=${BUILDARCH} go build -a -installsuffix cgo -ldflags '-extldflags "-static"' -o zitadel-linux-${BUILDARCH} cmd/zitadel/main.go
@ -165,8 +199,7 @@ RUN CGO_ENABLED=0 GOOS=linux GOARCH=${BUILDARCH} go build -a -installsuffix cgo
## Go dev build
#######################
FROM go-base as dev-go-build
RUN go get github.com/go-delve/delve/cmd/dlv
ENTRYPOINT [ "go", "run", "cmd/zitadel/main.go" ]
#######################
## Final Production Image

11
build/local/Dockerfile.clientid Normal file
View File

@ -0,0 +1,11 @@
FROM ubuntu:latest AS client-id
#install dependencies
RUN apt-get update \
&& apt-get install curl -y \
&& apt-get install jq -y
#prepare script
COPY build/local/clientid.sh clientid.sh
RUN chmod +x /clientid.sh
ENTRYPOINT [ "/clientid.sh" ]

36
build/local/Dockerfile.gateway Normal file
View File

@ -0,0 +1,36 @@
# copy from https://raw.githubusercontent.com/grpc/grpc-web/master/net/grpc/gateway/docker/grpcwebproxy/Dockerfile
FROM golang:1.16-alpine3.13
RUN apk add --no-cache curl git ca-certificates && \
rm -rf /var/lib/apt/lists/*
ARG VERSION=0.14.0
WORKDIR /tmp
RUN curl -sS https://raw.githubusercontent.com/golang/dep/master/install.sh | sh
RUN wget https://github.com/improbable-eng/grpc-web/archive/v$VERSION.tar.gz
WORKDIR /go/src/github.com/improbable-eng/
RUN tar -zxf /tmp/v$VERSION.tar.gz -C .
RUN mv grpc-web-$VERSION grpc-web
WORKDIR /go/src/github.com/improbable-eng/grpc-web
RUN dep ensure && \
go env -w GO111MODULE=auto && \
go install ./go/grpcwebproxy
# ADD ./etc/localhost.crt /etc
# ADD ./etc/localhost.key /etc
ENV BKD_HOST=backend-run
ENV BKD_PORT=50001
ENTRYPOINT [ "/bin/sh", "-c", "exec /go/bin/grpcwebproxy \
--backend_addr=${BKD_HOST}:${BKD_PORT} \
--run_tls_server=false \
--use_websockets \
--allow_all_origins " ]

7
build/local/Dockerfile.keys Normal file
View File

@ -0,0 +1,7 @@
FROM alpine:latest AS gen-keys
COPY build/local/keys.sh keys.sh
RUN chmod +x /keys.sh
ENTRYPOINT [ "/keys.sh" ]
FROM scratch AS copy-keys
COPY --from=gen-keys /.keys /.keys

10
build/local/Dockerfile.started Normal file
View File

@ -0,0 +1,10 @@
FROM ubuntu:latest AS started
#install dependencies
RUN apt-get update \
&& apt-get install curl -y
#prepare script
COPY build/local/zitadel-started.sh zitadel-started.sh
RUN chmod +x /zitadel-started.sh
ENTRYPOINT [ "/zitadel-started.sh" ]

18
build/local/clientid.sh Normal file
View File

@ -0,0 +1,18 @@
#!/bin/bash
# ------------------------------
# sets the client id in environment.json
# ------------------------------
clientid=""
while [ -z $clientid ]; do
echo "no from zitadel ==> retry"
sleep 2
clientid=$(curl -s http://${HOST}:${PORT}/clientID)
if [[ "$clientid" != *@zitadel* ]]; then
echo "invalid response from zitadel ==> retry"
clientid=""
fi
done
echo "$(jq ".clientid = $clientid" /environment.json)" > environment.json

157
build/local/docker-compose-local.yml Normal file
View File

@ -0,0 +1,157 @@
version: "3.8"
services:
db:
profiles: ["database"]
restart: always
networks:
- zitadel
image: cockroachdb/cockroach:v21.1.0
command: start-single-node --insecure --listen-addr=0.0.0.0
ports:
- 8080:8080
- 26257:26257
db-migrations:
profiles: ["database"]
restart: on-failure
networks:
- zitadel
depends_on:
- db
image: flyway/flyway:latest
volumes:
- ../../migrations/cockroach:/flyway/sql
environment:
- FLYWAY_PLACEHOLDERS_eventstorepassword=NULL
- FLYWAY_PLACEHOLDERS_managementpassword=NULL
- FLYWAY_PLACEHOLDERS_adminapipassword=NULL
- FLYWAY_PLACEHOLDERS_authpassword=NULL
- FLYWAY_PLACEHOLDERS_notificationpassword=NULL
- FLYWAY_PLACEHOLDERS_authzpassword=NULL
- FLYWAY_PLACEHOLDERS_queriespassword=NULL
command: -url=jdbc:postgresql://db:26257/defaultdb -user=root -password= -connectRetries=5 migrate
keys:
profiles: ["init-backend"]
restart: on-failure
networks:
- zitadel
build:
context: ../..
dockerfile: build/local/Dockerfile.keys
target: gen-keys
volumes:
- ../../.:/zitadel
env_file:
- ./local.env
backend-setup:
profiles: ["init-backend"]
restart: on-failure
networks:
- zitadel
depends_on:
- keys
build:
context: ../..
dockerfile: build/dockerfile
target: dev-go-build
args:
ENV: dev
volumes:
- ../../.keys:/go/src/github.com/caos/zitadel/.keys
env_file:
- ./local.env
environment:
- ZITADEL_EVENTSTORE_HOST=db
command: [ "-setup-files=cmd/zitadel/setup.yaml", "-setup-files=cmd/zitadel/system-defaults.yaml", "-setup-files=cmd/zitadel/authz.yaml", "setup" ]
backend-run:
profiles: ["backend"]
restart: on-failure
networks:
- zitadel
depends_on:
- db
build:
context: ../..
dockerfile: build/dockerfile
target: dev-go-build
args:
ENV: dev
volumes:
- ../../.keys:/go/src/github.com/caos/zitadel/.keys
env_file:
- ./local.env
environment:
- ZITADEL_EVENTSTORE_HOST=db
ports:
- 50002:50002
- 50003:50003
command: [ "-console=false", "-localDevMode=true", "-config-files=cmd/zitadel/startup.yaml", "-config-files=cmd/zitadel/system-defaults.yaml", "-config-files=cmd/zitadel/authz.yaml", "start" ]
zitadel-setted-up:
profiles: ["setup"]
networks:
- zitadel
build:
context: ../..
dockerfile: build/local/Dockerfile.started
volumes:
- ./environment.json:/environment.json
environment:
- BE_PORT=50002
- FE_PORT=4200
grpc-web-gateway:
profiles: ["frontend"]
restart: on-failure
logging:
driver: none
networks:
- zitadel
build:
context: ../..
dockerfile: build/local/Dockerfile.gateway
image: grpcweb/grpcwebproxy
ports:
- "50000:8080"
environment:
- BKD_HOST=backend-run
- BKD_PORT=50001
frontend-local-run:
profiles: ["frontend"]
networks:
- zitadel
depends_on:
- grpc-web-gateway
build:
context: ../..
dockerfile: build/dockerfile
target: dev-angular-build
args:
ENV: dev
volumes:
- ./environment.json:/console/src/assets/environment.json
command: sh -c "ng serve --host 0.0.0.0"
ports:
- 4200:4200
client-id:
profiles: ["init-frontend"]
networks:
- zitadel
build:
context: ../..
dockerfile: build/local/Dockerfile.clientid
target: client-id
volumes:
- ./environment.json:/environment.json
environment:
- HOST=backend-run
- PORT=50002
networks:
zitadel: {}

7
build/local/environment.json Normal file
View File

@ -0,0 +1,7 @@
{
"authServiceUrl": "http://localhost:50000",
"mgmtServiceUrl": "http://localhost:50000",
"adminServiceUrl": "http://localhost:50000",
"issuer": "http://localhost:50002/oauth/v2",
"clientid": "@zitadel"
}

23
build/local/keys.sh Executable file
View File

@ -0,0 +1,23 @@
#!/bin/sh
# ----------------------------------------------------------------
# generates necessary ZITADEL keys
# ----------------------------------------------------------------
set -e
KEY_PATH=$(echo "/zitadel/$(dirname ${ZITADEL_KEY_PATH})")
KEY_FILE=${KEY_PATH}/local_keys.yaml
mkdir -p ${KEY_PATH}
if [ ! -f ${KEY_FILE} ]; then
touch ${KEY_FILE}
fi
for key in $(env | grep "ZITADEL_.*_KEY" | cut -d'=' -f2); do
if [ $(grep -L ${key} ${KEY_FILE}) ]; then
echo "create key for ${key} in ${KEY_FILE}"
echo -e "${key}: $(head -c22 /dev/urandom | base64)" >> ${KEY_FILE}
fi
done

60
build/local/local.env Normal file
View File

@ -0,0 +1,60 @@
#tracing is disabled locally
ZITADEL_TRACING_TYPE=none
#metrics is disabled locally
ZITADEL_METRICS_TYPE=none
#recommended log level for local is debug
ZITADEL_LOG_LEVEL=debug
#database connection (cockroach insecure)
ZITADEL_EVENTSTORE_HOST=localhost
ZITADEL_EVENTSTORE_PORT=26257
CR_SSL_MODE=disable
#keys for cryptography
ZITADEL_KEY_PATH=.keys/local_keys.yaml
ZITADEL_USER_VERIFICATION_KEY=userverificationkey_1
ZITADEL_OTP_VERIFICATION_KEY=OTPVerificationKey_1
ZITADEL_OIDC_KEYS_ID=oidckey_1
ZITADEL_COOKIE_KEY=cookiekey_1
ZITADEL_CSRF_KEY=cookiekey_1
ZITADEL_IDP_CONFIG_VERIFICATION_KEY=idpconfigverificationkey_1
ZITADEL_DOMAIN_VERIFICATION_KEY=domainverificationkey_1
#debug mode is used for notifications
DEBUG_MODE=true
#used in the oidc library
#true enables usage of (insecure) http for localhost as issuer
CAOS_OIDC_DEV=true
#sets the cookies insecure in login (never use this in production!)
ZITADEL_CSRF_DEV=true
#currently needed
TWILIO_SENDER_NAME=ZITADEL developer
SMTP_HOST=smtp.gmail.com:465
SMTP_USER=zitadel-dev@caos.ch
EMAIL_SENDER_ADDRESS=noreply@caos.ch
EMAIL_SENDER_NAME=CAOS AG
SMTP_TLS=true
#configuration for api/browser calls
ZITADEL_DEFAULT_DOMAIN=zitadel.ch
ZITADEL_ISSUER=http://localhost:50002/oauth/v2
ZITADEL_ACCOUNTS=http://localhost:50003/login
ZITADEL_AUTHORIZE=http://localhost:50002/oauth/v2
ZITADEL_OAUTH=http://localhost:50002/oauth/v2
ZITADEL_CONSOLE=http://localhost:4200
ZITADEL_COOKIE_DOMAIN=localhost
#caching is used in UI's and API's
ZITADEL_CACHE_MAXAGE=12h
ZITADEL_CACHE_SHARED_MAXAGE=168h
ZITADEL_SHORT_CACHE_MAXAGE=5m
ZITADEL_SHORT_CACHE_SHARED_MAXAGE=15m
#console authorization configuration
ZITADEL_CONSOLE_RESPONSE_TYPE=CODE
ZITADEL_CONSOLE_GRANT_TYPE=AUTHORIZATION_CODE
ZITADEL_CONSOLE_DEV_MODE=true
ZITADEL_CONSOLE_ENV_DIR=console/src/assets/

37
build/local/zitadel-started.sh Executable file
View File

@ -0,0 +1,37 @@
#!/bin/bash
# ------------------------------
# prints a message as soon as
# ZITADEL is ready
# ------------------------------
be_status=""
fe_status=""
while [[ $be_status -ne 200 || $fe_status -ne 200 ]]; do
sleep 5
be_status=$(curl -s -o /dev/null -I -w "%{http_code}" host.docker.internal:${BE_PORT}/clientID)
fe_status=$(curl -s -o /dev/null -I -w "%{http_code}" host.docker.internal:${FE_PORT}/assets/environment.json)
echo "backend (${be_status}) or frontend (${fe_status}) not ready yet"
done
echo -e "++=======================================================================================++
|| ||
|| ZZZZZZZZZZZZ II TTTTTTTTTTTT AAAA DDDDDD EEEEEEEEEE LL ||
|| ZZ II TT AA AA DD DD EE LL ||
|| ZZ II TT AA AA DD DD EE LL ||
|| ZZ II TT AA AA DD DD EEEEEEEE LL ||
|| ZZ II TT AAAAAAAAAAAA DD DD EE LL ||
|| ZZ II TT AA AA DD DD EE LL ||
|| ZZZZZZZZZZZZ II TT AA AA DDDDDD EEEEEEEEEE LLLLLLLLLL ||
|| ||
|| ||
|| SSSSSSSSSS TTTTTTTTTTTT AAAA RRRRRRRR TTTTTTTTTTTT EEEEEEEEEE DDDDDD ||
|| SS TT AA AA RR RR TT EE DD DD ||
|| SS TT AA AA RR RR TT EE DD DD ||
|| SSSSSS TT AA AA RRRRRRRR TT EEEEEEEE DD DD ||
|| SS TT AAAAAAAAAAAA RRRR TT EE DD DD ||
|| SS TT AA AA RR RR TT EE DD DD ||
|| SSSSSSSSSS TT AA AA RR RR TT EEEEEEEEEE DDDDDD ||
|| ||
++=======================================================================================++"

5
build/login/generate-static.sh
View File

@ -1,5 +0,0 @@
#! /bin/sh
set -eux
go generate internal/ui/login/statik/generate.go

5
build/notification/generate-static.sh
View File

@ -1,5 +0,0 @@
#! /bin/sh
set -eux
go generate internal/notification/statik/generate.go

5
build/zitadel/generate-openapi-static.sh
View File

@ -1,5 +0,0 @@
#! /bin/sh
set -eux
go generate openapi/statik/generate.go

5
build/zitadel/generate-static.sh
View File

@ -1,5 +0,0 @@
#! /bin/sh
set -eux
go generate internal/statik/generate.go

84
cmd/zitadel/caos_local.sh
View File

@ -1,84 +0,0 @@
BASEDIR=$(dirname "$0")
gopass sync --store zitadel-secrets
# Tracing
gopass zitadel-secrets/zitadel/developer/default/zitadel-svc-account-zitadel-local | base64 -D > "$BASEDIR/local_svc-account-tracing.json"
export GOOGLE_APPLICATION_CREDENTIALS="$BASEDIR/local_svc-account-tracing.json"
export ZITADEL_TRACING_PROJECT_ID=zitadel-dev
export ZITADEL_TRACING_FRACTION=0.1
export ZITADEL_TRACING_ENDPOINT=localhost:9096
export ZITADEL_TRACING_TYPE=google
export ZITADEL_METRICS_TYPE=otel
# S3 Storage
export ZITADEL_ASSET_STORAGE_TYPE=minio
export ZITADEL_ASSET_STORAGE_ENDPOINT=storage.googleapis.com
export ZITADEL_ASSET_STORAGE_ACCESS_KEY_ID=
export ZITADEL_ASSET_STORAGE_SECRET_ACCESS_KEY=
export ZITADEL_ASSET_STORAGE_SSL=
export ZITADEL_ASSET_STORAGE_LOCATION=
# Log
export ZITADEL_LOG_LEVEL=debug
# Cockroach
export ZITADEL_EVENTSTORE_HOST=localhost
export ZITADEL_EVENTSTORE_PORT=26257
# Keys
gopass zitadel-secrets/zitadel/developer/default/keys.yaml > "$BASEDIR/local_keys.yaml"
export ZITADEL_KEY_PATH="$BASEDIR/local_keys.yaml"
export ZITADEL_USER_VERIFICATION_KEY=UserVerificationKey_1
export ZITADEL_IDP_CONFIG_VERIFICATION_KEY=IdpConfigVerificationKey_1
export ZITADEL_OTP_VERIFICATION_KEY=OTPVerificationKey_1
export ZITADEL_OIDC_KEYS_ID=OIDCKey_1
export ZITADEL_COOKIE_KEY=CookieKey_1
export ZITADEL_CSRF_KEY=CookieKey_1
export ZITADEL_DOMAIN_VERIFICATION_KEY=DomainVerificationKey_1
# Notifications
export DEBUG_MODE=TRUE
export TWILIO_SERVICE_SID=$(gopass zitadel-secrets/zitadel/dev/twilio-sid)
export TWILIO_TOKEN=$(gopass zitadel-secrets/zitadel/dev/twilio-auth-token)
export TWILIO_SENDER_NAME=CAOS AG
export SMTP_HOST=smtp.gmail.com:465
export SMTP_USER=zitadel@caos.ch
export SMTP_PASSWORD=$(gopass zitadel-secrets/zitadel/google/emailappkey)
export EMAIL_SENDER_ADDRESS=noreply@caos.ch
export EMAIL_SENDER_NAME=CAOS AG
export SMTP_TLS=TRUE
export CHAT_URL=$(gopass zitadel-secrets/zitadel/dev/google-chat-url)
#OIDC
export ZITADEL_ISSUER=http://localhost:50002/oauth/v2
export ZITADEL_ACCOUNTS=http://localhost:50003/login
export ZITADEL_AUTHORIZE=http://localhost:50002/oauth/v2
export ZITADEL_OAUTH=http://localhost:50002/oauth/v2
export ZITADEL_CONSOLE=http://localhost:4200
export CAOS_OIDC_DEV=true
export ZITADEL_COOKIE_DOMAIN=localhost
#CSRF
export ZITADEL_CSRF_DEV=true
#CACHE
export ZITADEL_CACHE_MAXAGE=12h
export ZITADEL_CACHE_SHARED_MAXAGE=168h
export ZITADEL_SHORT_CACHE_MAXAGE=5m
export ZITADEL_SHORT_CACHE_SHARED_MAXAGE=15m
#Console
export ZITADEL_CONSOLE_ENV_DIR=../../console/src/assets/
#Org
export ZITADEL_DEFAULT_DOMAIN=localhost
#Setup
export ZITADEL_CONSOLE_RESPONSE_TYPE='AUTHORIZATION_CODE'
export ZITADEL_CONSOLE_GRANT_TYPE='CODE'
export ZITADEL_CONSOLE_DEV_MODE=true

28
cmd/zitadel/main.go
View File

@ -4,16 +4,7 @@ import (
"context"
"flag"
"github.com/caos/zitadel/internal/command"
"github.com/caos/zitadel/internal/config/types"
"github.com/caos/zitadel/internal/eventstore"
"github.com/caos/zitadel/internal/query"
"github.com/caos/zitadel/internal/static/s3"
metrics "github.com/caos/zitadel/internal/telemetry/metrics/config"
"github.com/caos/zitadel/openapi"
"github.com/caos/logging"
admin_es "github.com/caos/zitadel/internal/admin/repository/eventsourcing"
"github.com/caos/zitadel/internal/api"
internal_authz "github.com/caos/zitadel/internal/api/authz"
@ -24,15 +15,22 @@ import (
auth_es "github.com/caos/zitadel/internal/auth/repository/eventsourcing"
"github.com/caos/zitadel/internal/authz"
authz_repo "github.com/caos/zitadel/internal/authz/repository/eventsourcing"
"github.com/caos/zitadel/internal/command"
"github.com/caos/zitadel/internal/config"
sd "github.com/caos/zitadel/internal/config/systemdefaults"
"github.com/caos/zitadel/internal/config/types"
"github.com/caos/zitadel/internal/eventstore"
mgmt_es "github.com/caos/zitadel/internal/management/repository/eventsourcing"
"github.com/caos/zitadel/internal/notification"
"github.com/caos/zitadel/internal/query"
"github.com/caos/zitadel/internal/setup"
"github.com/caos/zitadel/internal/static/s3"
metrics "github.com/caos/zitadel/internal/telemetry/metrics/config"
tracing "github.com/caos/zitadel/internal/telemetry/tracing/config"
"github.com/caos/zitadel/internal/ui"
"github.com/caos/zitadel/internal/ui/console"
"github.com/caos/zitadel/internal/ui/login"
"github.com/caos/zitadel/openapi"
)
type Config struct {
@ -94,7 +92,7 @@ func main() {
case cmdStart:
startZitadel(configPaths.Values())
case cmdSetup:
startSetup(setupPaths.Values(), *localDevMode)
startSetup(setupPaths.Values())
default:
logging.Log("MAIN-afEQ2").Fatal("please provide an valid argument [start, setup]")
}
@ -103,7 +101,7 @@ func main() {
func startZitadel(configPaths []string) {
conf := new(Config)
err := config.Read(conf, configPaths...)
logging.Log("MAIN-FaF2r").OnError(err).Fatal("cannot read config")
logging.Log("ZITAD-EDz31").OnError(err).Fatal("cannot read config")
ctx := context.Background()
esQueries, err := eventstore.StartWithUser(conf.EventstoreBase, conf.Queries.Eventstore)
@ -112,17 +110,17 @@ func startZitadel(configPaths []string) {
}
queries, err := query.StartQueries(esQueries, conf.SystemDefaults)
if err != nil {
logging.Log("MAIN-Ddv21").OnError(err).Fatal("cannot start queries")
logging.Log("ZITAD-WpeJY").OnError(err).Fatal("cannot start queries")
}
authZRepo, err := authz.Start(ctx, conf.AuthZ, conf.InternalAuthZ, conf.SystemDefaults, queries)
logging.Log("MAIN-s9KOw").OnError(err).Fatal("error starting authz repo")
esCommands, err := eventstore.StartWithUser(conf.EventstoreBase, conf.Commands.Eventstore)
if err != nil {
logging.Log("MAIN-Ddv21").OnError(err).Fatal("cannot start eventstore for commands")
logging.Log("ZITAD-iRCMm").OnError(err).Fatal("cannot start eventstore for commands")
}
commands, err := command.StartCommands(esCommands, conf.SystemDefaults, conf.InternalAuthZ, authZRepo)
if err != nil {
logging.Log("MAIN-Ddv21").OnError(err).Fatal("cannot start commands")
logging.Log("ZITAD-bmNiJ").OnError(err).Fatal("cannot start commands")
}
var authRepo *auth_es.EsRepository
if *authEnabled || *oidcEnabled || *loginEnabled {
@ -188,7 +186,7 @@ func startAPI(ctx context.Context, conf *Config, authZRepo *authz_repo.EsReposit
apis.Start(ctx)
}
func startSetup(configPaths []string, localDevMode bool) {
func startSetup(configPaths []string) {
conf := new(setupConfig)
err := config.Read(conf, configPaths...)
logging.Log("MAIN-FaF2r").OnError(err).Fatal("cannot read config")

153
guides/development.md Normal file
View File

@ -0,0 +1,153 @@
# Development
You should stay in the ZITADEL root directory to execute the statements in the following chapters.
## Prerequisite
- Buildkit compatible docker installation
### env variables
You can use the default vars provided in [this .env-file](../build/local/local.env) or create your own and update the paths in the [docker compose file](../build/local/docker-compose-local.yml).
## Generate required files
This part is relevant if you start the backend or console without docker compose.
### Console
This command generates the grpc stub for console into the folder console/src/app/proto/generated for local development.
```bash
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:gen-fe --target npm-copy -o .
```
### Backend
With this command you can generate the stub for the backend.
```bash
# generates grpc stub
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:gen-be --target go-copy -o .
# generates keys for cryptography
DOCKER_BUILDKIT=1 docker build --target copy_keys -f build/Dockerfile.dev . -o .keys
```
## Run
### Initialise data
Used if you want to setup the database and load the initial data.
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f ./build/local/docker-compose-local.yml --profile database --profile init-backend -p zitadel up
```
You can stop as soon as db-migrations AND backend-setup returned with exit code 0.
### Initialise frontend
Used to set the client id of the console. This step is for local development. If you don't work with a local backend you have to set the client id manually.
You must [initialise the data](###-Initialise-data)) first.
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f ./build/local/docker-compose-local.yml --profile database --profile backend --profile init-frontend -p zitadel up --exit-code-from client-id
```
The command exists as soon as the client id is set.
### Run database
Used if you want to run the backend/console locally and only need the database. It's recommended to [initialise the data](###-Initialise-data) first.
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f ./build/local/docker-compose-local.yml --profile database -p zitadel up
```
**On apple silicon:**
Restart the command (second terminal `docker restart zitadel_<SERVICE_NAME>_1`) if `db` logs `qemu: uncaught target signal 11 (Segmentation fault) - core dumped` or no logs are written from `db-migrations`.
### Run Console
The console service is configured for hot reloading. You can also use docker compose for local development.
If you don't use the backend from local you have to configure [the environment.json](../build/local/environment.json) manually.
If you use the local backend ensure that you have [set the correct client id](###-Initialise-frontend).
#### Run console in docker compose
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f ./build/local/docker-compose-local.yml --profile frontend -p zitadel up
```
### Run backend
Used if you want to run the backend locally. It's recommended to [initialise the data](###-Initialise-data) first.
#### Run backend in docker compose
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 docker-compose -f ./build/local/docker-compose-local.yml --profile database --profile backend -p zitadel up
```
#### Run backend locally
##### Export environment variables
```bash
# exports all default env variables
while read line; do
if [[ $line != #* ]] && [[ ! -z $line ]]; then
export $line
fi
done < build/local/local.env
```
##### Start command for backend
```bash
# starts zitadel with default config files
go run cmd/zitadel/main.go -console=false -localDevMode=true -config-files=cmd/zitadel/startup.yaml -config-files=cmd/zitadel/system-defaults.yaml -config-files=cmd/zitadel/authz.yaml start
```
If you want to run your backend locally and the frontend by docker compose you have to replace the following variables:
[docker compose yaml](../build/local/docker-compose-local.yml):
```yaml
service:
client-id:
environment:
- HOST=backend-run
grpc-web-gateway:
environment:
- BKD_HOST=backend-run
```
with
```yaml
service:
client-id:
environment:
- HOST=host.docker.internal
grpc-web-gateway:
environment:
- BKD_HOST=host.docker.internal
```
##### Setup ZITADEL
The following command starts the backend of ZITADEL with the default config files:
```bash
go run cmd/zitadel/main.go -setup-files=cmd/zitadel/setup.yaml -setup-files=cmd/zitadel/system-defaults.yaml -setup-files=cmd/zitadel/authz.yaml setup
```
## Initial login credentials
**username**: `zitadel-admin@caos-ag.zitadel.ch`
**password**: `Password1!`

7
guides/production.md Normal file
View File

@ -0,0 +1,7 @@
# Production Build
This can also be run locally!
```bash
DOCKER_BUILDKIT=1 docker build -f build/dockerfile . -t zitadel:local --build-arg ENV=prod
```

25
guides/quickstart.md Normal file
View File

@ -0,0 +1,25 @@
# Quickstart with docker compose
You can start ZITADEL with a simple docker compose up.
The services are configured to restart if an error occurs.
In the following script the basic setup of the database is executed before ZITADEL starts. Execute the statement from the root of ZITADEL.
You can connect to [ZITADEL on localhost:4200](http://localhost:4200) as soon as the following text appears:
```text
++=========++
|| ZITADEL ||
|| STARTED ||
++=========++
```
```bash
COMPOSE_DOCKER_CLI_BUILD=1 DOCKER_BUILDKIT=1 \
&& docker-compose -f ./build/local/docker-compose-local.yml --profile database -p zitadel up --exit-code-from db-migrations \
&& sleep 5 \
&& docker-compose -f ./build/local/docker-compose-local.yml --profile database --profile init-backend --profile init-frontend --profile backend --profile frontend --profile setup -p zitadel up
```
For a more detailed guide take a look at the [development guide](./development.md)

24
internal/notification/repository/eventsourcing/handler/notification.go
View File

@ -3,21 +3,16 @@ package handler
import (
"context"
"encoding/json"
"github.com/caos/zitadel/internal/command"
"github.com/caos/zitadel/internal/eventstore/v1"
"github.com/caos/zitadel/internal/user/repository/view"
"github.com/caos/zitadel/internal/user/repository/view/model"
view_model "github.com/caos/zitadel/internal/user/repository/view/model"
"golang.org/x/text/language"
"net/http"
"time"
"github.com/caos/logging"
"github.com/caos/zitadel/internal/api/authz"
"github.com/caos/zitadel/internal/command"
sd "github.com/caos/zitadel/internal/config/systemdefaults"
"github.com/caos/zitadel/internal/crypto"
"github.com/caos/zitadel/internal/errors"
caos_errs "github.com/caos/zitadel/internal/errors"
v1 "github.com/caos/zitadel/internal/eventstore/v1"
"github.com/caos/zitadel/internal/eventstore/v1/models"
"github.com/caos/zitadel/internal/eventstore/v1/query"
"github.com/caos/zitadel/internal/eventstore/v1/spooler"
@ -26,6 +21,9 @@ import (
iam_es_model "github.com/caos/zitadel/internal/iam/repository/view/model"
"github.com/caos/zitadel/internal/notification/types"
es_model "github.com/caos/zitadel/internal/user/repository/eventsourcing/model"
"github.com/caos/zitadel/internal/user/repository/view"
"github.com/caos/zitadel/internal/user/repository/view/model"
"golang.org/x/text/language"
)
const (
@ -278,7 +276,7 @@ func (n *Notification) handleDomainClaimed(event *models.Event) (err error) {
data := make(map[string]string)
if err := json.Unmarshal(event.Data, &data); err != nil {
logging.Log("HANDLE-Gghq2").WithError(err).Error("could not unmarshal event data")
return caos_errs.ThrowInternal(err, "HANDLE-7hgj3", "could not unmarshal event")
return errors.ThrowInternal(err, "HANDLE-7hgj3", "could not unmarshal event")
}
user, err := n.getUserByID(event.AggregateID)
if err != nil {
@ -356,7 +354,7 @@ func getSetNotifyContextData(orgID string) context.Context {
func (n *Notification) getLabelPolicy(ctx context.Context) (*iam_model.LabelPolicyView, error) {
// read from Org
policy, err := n.view.LabelPolicyByAggregateID(authz.GetCtxData(ctx).OrgID, labelPolicyTableOrg)
if caos_errs.IsNotFound(err) {
if errors.IsNotFound(err) {
// read from default
policy, err = n.view.LabelPolicyByAggregateID(n.systemDefaults.IamID, labelPolicyTableDef)
if err != nil {
@ -411,13 +409,13 @@ func (n *Notification) getMailText(ctx context.Context, textType string, lang st
return iam_es_model.MailTextViewToModel(mailText), err
}
func (n *Notification) getUserByID(userID string) (*view_model.NotifyUser, error) {
func (n *Notification) getUserByID(userID string) (*model.NotifyUser, error) {
user, usrErr := n.view.NotifyUserByID(userID)
if usrErr != nil && !caos_errs.IsNotFound(usrErr) {
if usrErr != nil && !errors.IsNotFound(usrErr) {
return nil, usrErr
}
if user == nil {
user = &view_model.NotifyUser{}
user = &model.NotifyUser{}
}
events, err := n.getUserEvents(userID, user.Sequence)
if err != nil {
@ -430,7 +428,7 @@ func (n *Notification) getUserByID(userID string) (*view_model.NotifyUser, error
}
}
if userCopy.State == int32(model.UserStateDeleted) {
return nil, caos_errs.ThrowNotFound(nil, "HANDLER-3n8fs", "Errors.User.NotFound")
return nil, errors.ThrowNotFound(nil, "HANDLER-3n8fs", "Errors.User.NotFound")
}
return &userCopy, nil
}

7
internal/query/query.go
View File

@ -3,15 +3,14 @@ package query
import (
"context"
"github.com/caos/zitadel/internal/config/types"
"github.com/caos/zitadel/internal/eventstore"
usr_repo "github.com/caos/zitadel/internal/repository/user"
sd "github.com/caos/zitadel/internal/config/systemdefaults"
"github.com/caos/zitadel/internal/config/types"
"github.com/caos/zitadel/internal/crypto"
"github.com/caos/zitadel/internal/eventstore"
iam_model "github.com/caos/zitadel/internal/iam/model"
"github.com/caos/zitadel/internal/id"
iam_repo "github.com/caos/zitadel/internal/repository/iam"
usr_repo "github.com/caos/zitadel/internal/repository/user"
"github.com/caos/zitadel/internal/telemetry/tracing"
)

16
internal/ui/login/handler/login.go
View File

@ -5,26 +5,24 @@ import (
"net"
"net/http"
"github.com/caos/zitadel/internal/command"
"github.com/caos/zitadel/internal/domain"
"github.com/caos/zitadel/internal/query"
usr_model "github.com/caos/zitadel/internal/user/model"
"github.com/caos/logging"
"github.com/gorilla/csrf"
"github.com/rakyll/statik/fs"
"golang.org/x/text/language"
"github.com/caos/zitadel/internal/api/authz"
http_utils "github.com/caos/zitadel/internal/api/http"
"github.com/caos/zitadel/internal/api/http/middleware"
auth_repository "github.com/caos/zitadel/internal/auth/repository"
"github.com/caos/zitadel/internal/auth/repository/eventsourcing"
"github.com/caos/zitadel/internal/command"
"github.com/caos/zitadel/internal/config/systemdefaults"
"github.com/caos/zitadel/internal/crypto"
"github.com/caos/zitadel/internal/domain"
"github.com/caos/zitadel/internal/form"
"github.com/caos/zitadel/internal/id"
"github.com/caos/zitadel/internal/query"
_ "github.com/caos/zitadel/internal/ui/login/statik"
usr_model "github.com/caos/zitadel/internal/user/model"
"github.com/gorilla/csrf"
"github.com/rakyll/statik/fs"
"golang.org/x/text/language"
)
type Login struct {