TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2025-08-11 14:47:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: reproducible pipeline with dev containers (#10305)

Browse Source 
# Which Problems Are Solved

- The previous monorepo in monorepo structure for the login app and its
related packages was fragmented, complicated and buggy.
- The process for building and testing the login container was
inconsistent between local development and CI.
- Lack of clear documentation as well as easy and reliable ways for
non-frontend developers to reproduce and fix failing PR checks locally.

# How the Problems Are Solved

- Consolidated the login app and its related npm packages by moving the
main package to `apps/login/apps/login` and merging
`apps/login/packages/integration` and `apps/login/packages/acceptance`
into the main `apps/login` package.
- Migrated from Docker Compose-based test setups to dev container-based
setups, adding support for multiple dev container configurations:
  - `.devcontainer/base`
  - `.devcontainer/turbo-lint-unit`
  - `.devcontainer/turbo-lint-unit-debug`
  - `.devcontainer/login-integration`
  - `.devcontainer/login-integration-debug`
- Added npm scripts to run the new dev container setups, enabling exact
reproduction of GitHub PR checks locally, and updated the pipeline to
use these containers.
- Cleaned up Dockerfiles and docker-bake.hcl files to only build the
production image for the login app.
- Cleaned up compose files to focus on dev environments in dev
containers.
- Updated `CONTRIBUTING.md` with guidance on running and debugging PR
checks locally using the new dev container approach.
- Introduced separate Dockerfiles for the login app to distinguish
between using published client packages and building clients from local
protos.
- Ensured the login container is always built in the pipeline for use in
integration and acceptance tests.
- Updated Makefile and GitHub Actions workflows to use
`--frozen-lockfile` for installing pnpm packages, ensuring reproducible
installs.
- Disabled GitHub release creation by the changeset action.
- Refactored the `/build` directory structure for clarity and
maintainability.
- Added a `clean` command to `docks/package.json`.
- Experimentally added `knip` to the `zitadel-client` package for
improved linting of dependencies and exports.

# Additional Changes

- Fixed Makefile commands for consistency and reliability.
- Improved the structure and clarity of the `/build` directory to
support seamless integration of the login build.
- Enhanced documentation and developer experience for running and
debugging CI checks locally.

# Additional Context

- See updated `CONTRIBUTING.md` for new local development and debugging
instructions.
- These changes are a prerequisite for further improvements to the CI
pipeline and local development workflow.
- Closes #10276
This commit is contained in:
Elio Bischof
2025-07-24 14:22:32 +02:00
committed by GitHub
parent af66c9844a
commit b10455b51f
430 changed files with 2869 additions and 4108 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
build/Dockerfile.gitignore
View File

@@ -1,3 +0,0 @@
*
!build/entrypoint.sh
!zitadel

37
build/login/.dockerignore Normal file
View File

@@ -0,0 +1,37 @@
*
!apps/login/constants
!apps/login/scripts
!apps/login/src
!apps/login/public
!apps/login/locales
!apps/login/next.config.mjs
!apps/login/next-env-vars.d.ts
!apps/login/next-env.d.ts
!apps/login/tailwind.config.js
!apps/login/tsconfig.json
!apps/login/package.json
!apps/login/turbo.json
!package.json
!pnpm-lock.yaml
!pnpm-workspace.yaml
!turbo.json
!packages/zitadel-proto/package.json
!packages/zitadel-proto/buf.gen.yaml
!packages/zitadel-proto/turbo.json
!packages/zitadel-client/package.json
!packages/zitadel-client/src
!packages/zitadel-client/tsconfig.json
!packages/zitadel-client/tsup.config.ts
!packages/zitadel-client/turbo.json
!proto
*.md
*.png
node_modules
*.test.ts
*.test.tsx

45
build/login/Dockerfile Normal file
View File

@@ -0,0 +1,45 @@
FROM node:20-alpine AS base
FROM base AS build
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME:$PATH"
RUN corepack enable && COREPACK_ENABLE_DOWNLOAD_PROMPT=0 corepack prepare pnpm@9.1.2 --activate && \
apk update && apk add --no-cache && \
rm -rf /var/cache/apk/*
WORKDIR /app
COPY pnpm-lock.yaml pnpm-workspace.yaml ./
RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store pnpm fetch --frozen-lockfile \
--filter @zitadel/login \
--filter @zitadel/client \
--filter @zitadel/proto
COPY package.json ./
COPY apps/login/package.json ./apps/login/package.json
COPY packages/zitadel-proto/package.json ./packages/zitadel-proto/package.json
COPY packages/zitadel-client/package.json ./packages/zitadel-client/package.json
RUN --mount=type=cache,id=pnpm,target=/root/.local/share/pnpm/store pnpm install --frozen-lockfile \
--filter @zitadel/login \
--filter @zitadel/client \
--filter @zitadel/proto
COPY . .
RUN pnpm turbo build:login:standalone
FROM scratch AS build-out
COPY --from=build /app/apps/login/.next/standalone /
COPY --from=build /app/apps/login/.next/static /.next/static
COPY --from=build /app/apps/login/public /public
FROM base AS login-standalone
WORKDIR /runtime
RUN addgroup --system --gid 1001 nodejs && \
adduser --system --uid 1001 nextjs
# If /.env-file/.env is mounted into the container, its variables are made available to the server before it starts up.
RUN mkdir -p /.env-file && touch /.env-file/.env && chown -R nextjs:nodejs /.env-file
COPY apps/login/scripts ./
COPY --chown=nextjs:nodejs --from=build-out . .
USER nextjs
ENV HOSTNAME="0.0.0.0"
ENV PORT=3000
# TODO: Check healthy, not ready
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
CMD ["/bin/sh", "-c", "node ./healthcheck.js http://localhost:${PORT}/ui/v2/login/healthy"]
ENTRYPOINT ["./entrypoint.sh"]

296
build/workflow.Dockerfile
View File

@@ -1,296 +0,0 @@
# ##############################################################################
# core
# ##############################################################################
# #######################################
# download dependencies
# #######################################
FROM golang:buster AS core-deps
WORKDIR /go/src/github.com/zitadel/zitadel
COPY go.mod .
COPY go.sum .
RUN go mod download
# #######################################
# compile custom protoc plugins
# #######################################
FROM golang:buster AS core-api-generator
WORKDIR /go/src/github.com/zitadel/zitadel
COPY go.mod .
COPY go.sum .
COPY internal/protoc internal/protoc
COPY pkg/grpc/protoc/v2 pkg/grpc/protoc/v2
RUN go install internal/protoc/protoc-gen-authoption/main.go \
&& mv $(go env GOPATH)/bin/main $(go env GOPATH)/bin/protoc-gen-authoption \
&& go install internal/protoc/protoc-gen-zitadel/main.go \
&& mv $(go env GOPATH)/bin/main $(go env GOPATH)/bin/protoc-gen-zitadel
# #######################################
# build backend stub
# #######################################
FROM golang:buster AS core-api
WORKDIR /go/src/github.com/zitadel/zitadel
COPY go.mod .
COPY go.sum .
COPY proto proto
COPY buf.*.yaml .
COPY Makefile Makefile
COPY --from=core-api-generator /go/bin /usr/local/bin
RUN make grpc
# #######################################
# generate code for login ui
# #######################################
FROM golang:buster AS core-login
WORKDIR /go/src/github.com/zitadel/zitadel
COPY Makefile Makefile
COPY internal/api/ui/login/static internal/api/ui/login/static
COPY internal/api/ui/login/statik internal/api/ui/login/statik
COPY internal/notification/static internal/notification/static
COPY internal/notification/statik internal/notification/statik
COPY internal/static internal/static
COPY internal/statik internal/statik
RUN make static
# #######################################
# generate code for assets
# #######################################
FROM golang:buster AS core-assets
WORKDIR /go/src/github.com/zitadel/zitadel
COPY go.mod .
COPY go.sum .
COPY Makefile Makefile
COPY internal/api/assets/generator internal/api/assets/generator
COPY internal/config internal/config
COPY internal/errors internal/errors
COPY --from=core-api /go/src/github.com/zitadel/zitadel/openapi/v2 openapi/v2
RUN make assets
# #######################################
# Gather all core files
# #######################################
FROM core-deps AS core-gathered
COPY cmd cmd
COPY internal internal
COPY pkg pkg
COPY proto proto
COPY openapi openapi
COPY statik statik
COPY main.go main.go
COPY --from=core-api /go/src/github.com/zitadel/zitadel .
COPY --from=core-login /go/src/github.com/zitadel/zitadel .
COPY --from=core-assets /go/src/github.com/zitadel/zitadel/internal ./internal
# ##############################################################################
# build console
# ##############################################################################
# #######################################
# download console dependencies
# #######################################
FROM node:20-buster AS console-deps
WORKDIR /zitadel/console
COPY pnpm-lock.yaml .
COPY pnpm-workspace.yaml .
COPY console/package.json console/
RUN corepack enable pnpm && pnpm install --frozen-lockfile --filter=console
# #######################################
# generate console client
# #######################################
FROM node:20-buster AS console-client
WORKDIR /zitadel/console
# install buf
COPY --from=bufbuild/buf:latest /usr/local/bin/* /usr/local/bin/
ENV PATH="/usr/local/bin:${PATH}"
COPY console/package.json .
COPY console/buf.*.yaml .
COPY proto ../proto
RUN pnpm generate
# #######################################
# Gather all console files
# #######################################
FROM console-deps as console-gathered
COPY --from=console-client /zitadel/console/src/app/proto/generated src/app/proto/generated
COPY console/src src
COPY console/angular.json .
COPY console/ngsw-config.json .
COPY console/tsconfig* .
# #######################################
# Build console
# #######################################
FROM console-gathered AS console
RUN pnpm build
# ##############################################################################
# build the executable
# ##############################################################################
# #######################################
# build executable
# #######################################
FROM core-gathered AS compile
ARG GOOS
ARG GOARCH
COPY --from=console /zitadel/console/dist/console internal/api/ui/console/static/
RUN go build -o zitadel -ldflags="-s -w -race" \
&& chmod +x zitadel
ENTRYPOINT [ "./zitadel" ]
# #######################################
# copy executable
# #######################################
FROM scratch AS copy-executable
ARG GOOS
ARG GOARCH
COPY --from=compile /go/src/github.com/zitadel/zitadel/zitadel /.artifacts/zitadel
# ##############################################################################
# tests
# ##############################################################################
FROM ubuntu/postgres:latest AS test-core-base
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends \
gcc \
make \
ca-certificates \
gcc \
&& \
update-ca-certificates; \
rm -rf /var/lib/apt/lists/*
# install go
COPY --from=golang:latest /usr/local/go/ /usr/local/go/
ENV PATH="/go/bin:/usr/local/go/bin:${PATH}"
WORKDIR /go/src/github.com/zitadel/zitadel
# default vars
ENV POSTGRES_USER=zitadel
ENV POSTGRES_DB=zitadel
ENV POSTGRES_PASSWORD=postgres
ENV POSTGRES_HOST_AUTH_METHOD=trust
ENV PGUSER=zitadel
ENV PGDATABASE=zitadel
ENV PGPASSWORD=postgres
ENV CGO_ENABLED=1
# copy zitadel files
COPY --from=core-deps /go/pkg/mod /root/go/pkg/mod
COPY --from=core-gathered /go/src/github.com/zitadel/zitadel .
# #######################################
# unit test core
# #######################################
FROM test-core-base AS test-core-unit
RUN go test -race -v -coverprofile=profile.cov ./...
# #######################################
# coverage output
# #######################################
FROM scratch AS coverage-core-unit
COPY --from=test-core-unit /go/src/github.com/zitadel/zitadel/profile.cov /coverage/
# #######################################
# integration test core
# #######################################
FROM test-core-base AS test-core-integration
ENV ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters
COPY build/core-integration-test.sh /usr/local/bin/run-tests.sh
RUN chmod +x /usr/local/bin/run-tests.sh
RUN run-tests.sh
# #######################################
# coverage output
# #######################################
FROM scratch AS coverage-core-integration
COPY --from=test-core-integration /go/src/github.com/zitadel/zitadel/profile.cov /coverage/
# ##############################################################################
# linting
# ##############################################################################
# #######################################
# api
# #######################################
FROM bufbuild/buf:latest AS lint-api
COPY proto proto
COPY buf.*.yaml .
RUN buf lint
# #######################################
# console
# #######################################
FROM console-gathered AS lint-console
COPY console/.eslintrc.js .
COPY console/.prettier* .
RUN pnpm lint
# #######################################
# core
# #######################################
FROM golangci/golangci-lint:latest AS lint-core
ARG LINT_EXIT_CODE=1
WORKDIR /go/src/github.com/zitadel/zitadel
COPY .golangci.yaml .
COPY .git/ .git/
COPY --from=core-deps /go/pkg/mod /go/pkg/mod
COPY --from=core-gathered /go/src/github.com/zitadel/zitadel .
RUN git fetch https://github.com/zitadel/zitadel main:main
RUN golangci-lint run \
--timeout 10m \
--config ./.golangci.yaml \
--out-format=github-actions:report,colored-line-number \
--issues-exit-code=${LINT_EXIT_CODE} \
--concurrency=$(getconf _NPROCESSORS_ONLN)
# #######################################
# report output
# #######################################
FROM scratch AS lint-core-report
COPY --from=lint-core /go/src/github.com/zitadel/zitadel/report .

3
build/zitadel/.dockerignore Normal file
View File

@@ -0,0 +1,3 @@
*
!build/zitadel/entrypoint.sh
!zitadel

2
build/Dockerfile → build/zitadel/Dockerfile
View File

@@ -4,7 +4,7 @@ ARG TARGETPLATFORM
RUN apt-get update && apt-get install ca-certificates -y
COPY build/entrypoint.sh /app/entrypoint.sh
COPY build/zitadel/entrypoint.sh /app/entrypoint.sh
COPY zitadel /app/zitadel
RUN useradd -s "" --home / zitadel && \

0
build/entrypoint.sh → build/zitadel/entrypoint.sh
View File

0
build/zitadel/generate-grpc.sh
View File