mirror of
https://github.com/zitadel/zitadel.git
synced 2025-08-11 18:33:28 +00:00
312b7b6010
This PR modernizes the ZITADEL monorepo build system by migrating from Yarn to pnpm, introducing Turbo for improved build orchestration, and cleaning up configuration inconsistencies across all apps and packages. ### 🎯 Key Improvements #### 📦 **Package Manager Migration (Yarn → pnpm)** - **Performance**: Faster installs with pnpm's efficient symlink-based node_modules structure - **Disk space**: Significant reduction in disk usage through content-addressable storage - **Lockfile**: More reliable dependency resolution with pnpm-lock.yaml - **Workspace support**: Better monorepo dependency management #### ⚡ **Turbo Integration** - **Build orchestration**: Dependency-aware task execution across the monorepo - **Intelligent caching**: Dramatically faster builds on CI/CD and local development - **Parallel execution**: Optimal task scheduling based on dependency graphs - **Vercel optimization**: Enhanced build performance and caching on Vercel deployments #### 🧹 **Configuration Cleanup & Unification** - **Removed config packages**: Eliminated `@zitadel/*-config` packages and inlined configurations - **Simplified dependencies**: Reduced complexity in package.json files across all apps - **Consistent tooling**: Unified prettier, ESLint, and TypeScript configurations - **Standalone support**: Improved prepare-standalone.js script for subtree deployments ### 📋 Detailed Changes #### **🔧 Build System & Dependencies** - ✅ Updated all package.json scripts to use `pnpm` instead of `yarn` - ✅ Replaced `yarn.lock` with pnpm-lock.yaml and regenerated dependencies - ✅ Added Turbo configuration (turbo.json) to root and individual packages - ✅ Configured proper dependency chains: `@zitadel/proto#generate` → `@zitadel/client#build` → `console#build` - ✅ Added missing `@bufbuild/protobuf` dependency to console app for TypeScript compilation #### **🚀 CI/CD & Workflows** - ✅ Updated all GitHub Actions workflows to use `pnpm/action-setup@v4` - ✅ Migrated build processes to use Turbo with directory-based filters (`--filter=./console`) - ✅ **New**: Added `docs.yml` workflow for building documentation locally (helpful for contributors without Vercel access) - ✅ Fixed dependency resolution issues in lint workflows - ✅ Ensured proto generation always runs before builds and linting #### **📚 Documentation & Proto Generation** - ✅ **Robust plugin management**: Enhanced plugin-download.sh with retry logic and error handling - ✅ **Vercel compatibility**: Fixed protoc-gen-connect-openapi plugin availability in Vercel builds - ✅ **API docs generation**: Resolved Docusaurus build errors with OpenAPI plugin configuration - ✅ **Type safety**: Improved TypeScript type extraction patterns in Angular components #### **🛠️ Developer Experience** - ✅ Updated all README files to reference pnpm commands - ✅ Improved Makefile targets to use Turbo for consistent builds - ✅ Enhanced standalone build process for login app subtree deployments - ✅ Added debug utilities for troubleshooting build issues #### **🗂️ File Structure & Cleanup** - ✅ Removed obsolete configuration packages and their references - ✅ Cleaned up Docker files to remove non-existent package copies - ✅ Updated workspace references and import paths - ✅ Streamlined turbo.json configurations across all packages ### 🎉 Benefits 1. **⚡ Faster Builds**: Turbo's caching and parallel execution significantly reduce build times 2. **🔄 Better Caching**: Improved cache hits on Vercel and CI/CD environments 3. **🛠️ Simplified Maintenance**: Unified tooling and configuration management 4. **📈 Developer Productivity**: Faster local development with optimized dependency resolution 5. **🚀 Enhanced CI/CD**: More reliable and faster automated builds and deployments 6. **📖 Better Documentation**: Comprehensive build documentation and troubleshooting guides ### 🧪 Testing - ✅ All apps build successfully with new pnpm + Turbo setup - ✅ Proto generation works correctly across console, login, and docs - ✅ GitHub Actions workflows pass with new configuration - ✅ Vercel deployments work with enhanced plugin management - ✅ Local development workflow verified and documented This migration sets a solid foundation for future development while maintaining backward compatibility and improving the overall developer experience. --------- Co-authored-by: Elio Bischof <elio@zitadel.com>
296 lines
8.2 KiB
Docker
296 lines
8.2 KiB
Docker
# ##############################################################################
|
|
# core
|
|
# ##############################################################################
|
|
|
|
# #######################################
|
|
# download dependencies
|
|
# #######################################
|
|
FROM golang:buster AS core-deps
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY go.mod .
|
|
COPY go.sum .
|
|
|
|
RUN go mod download
|
|
|
|
# #######################################
|
|
# compile custom protoc plugins
|
|
# #######################################
|
|
FROM golang:buster AS core-api-generator
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY go.mod .
|
|
COPY go.sum .
|
|
COPY internal/protoc internal/protoc
|
|
COPY pkg/grpc/protoc/v2 pkg/grpc/protoc/v2
|
|
|
|
RUN go install internal/protoc/protoc-gen-authoption/main.go \
|
|
&& mv $(go env GOPATH)/bin/main $(go env GOPATH)/bin/protoc-gen-authoption \
|
|
&& go install internal/protoc/protoc-gen-zitadel/main.go \
|
|
&& mv $(go env GOPATH)/bin/main $(go env GOPATH)/bin/protoc-gen-zitadel
|
|
|
|
# #######################################
|
|
# build backend stub
|
|
# #######################################
|
|
FROM golang:buster AS core-api
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY go.mod .
|
|
COPY go.sum .
|
|
COPY proto proto
|
|
COPY buf.*.yaml .
|
|
COPY Makefile Makefile
|
|
COPY --from=core-api-generator /go/bin /usr/local/bin
|
|
|
|
RUN make grpc
|
|
|
|
# #######################################
|
|
# generate code for login ui
|
|
# #######################################
|
|
FROM golang:buster AS core-login
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY Makefile Makefile
|
|
COPY internal/api/ui/login/static internal/api/ui/login/static
|
|
COPY internal/api/ui/login/statik internal/api/ui/login/statik
|
|
COPY internal/notification/static internal/notification/static
|
|
COPY internal/notification/statik internal/notification/statik
|
|
COPY internal/static internal/static
|
|
COPY internal/statik internal/statik
|
|
|
|
RUN make static
|
|
|
|
# #######################################
|
|
# generate code for assets
|
|
# #######################################
|
|
FROM golang:buster AS core-assets
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY go.mod .
|
|
COPY go.sum .
|
|
COPY Makefile Makefile
|
|
COPY internal/api/assets/generator internal/api/assets/generator
|
|
COPY internal/config internal/config
|
|
COPY internal/errors internal/errors
|
|
COPY --from=core-api /go/src/github.com/zitadel/zitadel/openapi/v2 openapi/v2
|
|
|
|
RUN make assets
|
|
|
|
# #######################################
|
|
# Gather all core files
|
|
# #######################################
|
|
FROM core-deps AS core-gathered
|
|
|
|
COPY cmd cmd
|
|
COPY internal internal
|
|
COPY pkg pkg
|
|
COPY proto proto
|
|
COPY openapi openapi
|
|
COPY statik statik
|
|
COPY main.go main.go
|
|
COPY --from=core-api /go/src/github.com/zitadel/zitadel .
|
|
COPY --from=core-login /go/src/github.com/zitadel/zitadel .
|
|
COPY --from=core-assets /go/src/github.com/zitadel/zitadel/internal ./internal
|
|
|
|
# ##############################################################################
|
|
# build console
|
|
# ##############################################################################
|
|
|
|
# #######################################
|
|
# download console dependencies
|
|
# #######################################
|
|
FROM node:20-buster AS console-deps
|
|
|
|
WORKDIR /zitadel/console
|
|
|
|
COPY pnpm-lock.yaml .
|
|
COPY pnpm-workspace.yaml .
|
|
COPY console/package.json console/
|
|
|
|
RUN corepack enable pnpm && pnpm install --frozen-lockfile --filter=console
|
|
|
|
# #######################################
|
|
# generate console client
|
|
# #######################################
|
|
FROM node:20-buster AS console-client
|
|
|
|
WORKDIR /zitadel/console
|
|
|
|
# install buf
|
|
COPY --from=bufbuild/buf:latest /usr/local/bin/* /usr/local/bin/
|
|
ENV PATH="/usr/local/bin:${PATH}"
|
|
|
|
COPY console/package.json .
|
|
COPY console/buf.*.yaml .
|
|
COPY proto ../proto
|
|
|
|
RUN pnpm generate
|
|
|
|
# #######################################
|
|
# Gather all console files
|
|
# #######################################
|
|
FROM console-deps as console-gathered
|
|
|
|
COPY --from=console-client /zitadel/console/src/app/proto/generated src/app/proto/generated
|
|
|
|
COPY console/src src
|
|
COPY console/angular.json .
|
|
COPY console/ngsw-config.json .
|
|
COPY console/tsconfig* .
|
|
|
|
# #######################################
|
|
# Build console
|
|
# #######################################
|
|
FROM console-gathered AS console
|
|
RUN pnpm build
|
|
|
|
# ##############################################################################
|
|
# build the executable
|
|
# ##############################################################################
|
|
|
|
# #######################################
|
|
# build executable
|
|
# #######################################
|
|
FROM core-gathered AS compile
|
|
|
|
ARG GOOS
|
|
ARG GOARCH
|
|
|
|
COPY --from=console /zitadel/console/dist/console internal/api/ui/console/static/
|
|
|
|
RUN go build -o zitadel -ldflags="-s -w -race" \
|
|
&& chmod +x zitadel
|
|
|
|
ENTRYPOINT [ "./zitadel" ]
|
|
|
|
# #######################################
|
|
# copy executable
|
|
# #######################################
|
|
FROM scratch AS copy-executable
|
|
ARG GOOS
|
|
ARG GOARCH
|
|
|
|
COPY --from=compile /go/src/github.com/zitadel/zitadel/zitadel /.artifacts/zitadel
|
|
|
|
# ##############################################################################
|
|
# tests
|
|
# ##############################################################################
|
|
FROM ubuntu/postgres:latest AS test-core-base
|
|
|
|
ARG DEBIAN_FRONTEND=noninteractive
|
|
|
|
RUN apt-get update && \
|
|
apt-get install -y --no-install-recommends \
|
|
gcc \
|
|
make \
|
|
ca-certificates \
|
|
gcc \
|
|
&& \
|
|
update-ca-certificates; \
|
|
rm -rf /var/lib/apt/lists/*
|
|
|
|
# install go
|
|
COPY --from=golang:latest /usr/local/go/ /usr/local/go/
|
|
ENV PATH="/go/bin:/usr/local/go/bin:${PATH}"
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
# default vars
|
|
ENV POSTGRES_USER=zitadel
|
|
ENV POSTGRES_DB=zitadel
|
|
ENV POSTGRES_PASSWORD=postgres
|
|
ENV POSTGRES_HOST_AUTH_METHOD=trust
|
|
|
|
ENV PGUSER=zitadel
|
|
ENV PGDATABASE=zitadel
|
|
ENV PGPASSWORD=postgres
|
|
|
|
ENV CGO_ENABLED=1
|
|
|
|
# copy zitadel files
|
|
COPY --from=core-deps /go/pkg/mod /root/go/pkg/mod
|
|
COPY --from=core-gathered /go/src/github.com/zitadel/zitadel .
|
|
|
|
# #######################################
|
|
# unit test core
|
|
# #######################################
|
|
FROM test-core-base AS test-core-unit
|
|
RUN go test -race -v -coverprofile=profile.cov ./...
|
|
|
|
# #######################################
|
|
# coverage output
|
|
# #######################################
|
|
FROM scratch AS coverage-core-unit
|
|
COPY --from=test-core-unit /go/src/github.com/zitadel/zitadel/profile.cov /coverage/
|
|
|
|
# #######################################
|
|
# integration test core
|
|
# #######################################
|
|
FROM test-core-base AS test-core-integration
|
|
ENV ZITADEL_MASTERKEY=MasterkeyNeedsToHave32Characters
|
|
|
|
COPY build/core-integration-test.sh /usr/local/bin/run-tests.sh
|
|
RUN chmod +x /usr/local/bin/run-tests.sh
|
|
|
|
RUN run-tests.sh
|
|
|
|
# #######################################
|
|
# coverage output
|
|
# #######################################
|
|
FROM scratch AS coverage-core-integration
|
|
COPY --from=test-core-integration /go/src/github.com/zitadel/zitadel/profile.cov /coverage/
|
|
|
|
# ##############################################################################
|
|
# linting
|
|
# ##############################################################################
|
|
|
|
# #######################################
|
|
# api
|
|
# #######################################
|
|
FROM bufbuild/buf:latest AS lint-api
|
|
|
|
COPY proto proto
|
|
COPY buf.*.yaml .
|
|
|
|
RUN buf lint
|
|
|
|
# #######################################
|
|
# console
|
|
# #######################################
|
|
FROM console-gathered AS lint-console
|
|
|
|
COPY console/.eslintrc.js .
|
|
COPY console/.prettier* .
|
|
RUN pnpm lint
|
|
|
|
# #######################################
|
|
# core
|
|
# #######################################
|
|
FROM golangci/golangci-lint:latest AS lint-core
|
|
ARG LINT_EXIT_CODE=1
|
|
|
|
WORKDIR /go/src/github.com/zitadel/zitadel
|
|
|
|
COPY .golangci.yaml .
|
|
COPY .git/ .git/
|
|
COPY --from=core-deps /go/pkg/mod /go/pkg/mod
|
|
COPY --from=core-gathered /go/src/github.com/zitadel/zitadel .
|
|
|
|
RUN git fetch https://github.com/zitadel/zitadel main:main
|
|
|
|
RUN golangci-lint run \
|
|
--timeout 10m \
|
|
--config ./.golangci.yaml \
|
|
--out-format=github-actions:report,colored-line-number \
|
|
--issues-exit-code=${LINT_EXIT_CODE} \
|
|
--concurrency=$(getconf _NPROCESSORS_ONLN)
|
|
|
|
# #######################################
|
|
# report output
|
|
# #######################################
|
|
FROM scratch AS lint-core-report
|
|
COPY --from=lint-core /go/src/github.com/zitadel/zitadel/report . |