TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2025-11-03 12:14:20 +00:00
Code Issues Packages Projects Releases Wiki Activity
4,319 Commits 259 Branches 2,098 Tags
59b57dce55bb297ed84b856bbd73ab4fa6de2274
Commit Graph

4 Commits

Author SHA1 Message Date
Elio Bischof
37acd2a9d9 chore: rehaul DevX (#10571) 
# Which Problems Are Solved

Replaces Turbo by Nx and lays the foundation for the next CI
improvements. It enables using Nx Cloud to speed the up the pipelines
that affect any node package.
It streamlines the dev experience for frontend and backend developers by
providing the following commands:

| Task | Command | Notes |
|------|---------|--------|
| **Production** | `nx run PROJECT:prod` | Production server |
| **Develop** | `nx run PROJECT:dev` | Hot reloading development server
|
| **Test** | `nx run PROJECT:test` | Run all tests |
| **Lint** | `nx run PROJECT:lint` | Check code style |
| **Lint Fix** | `nx run PROJECT:lint-fix` | Auto-fix style issues |

The following values can be used for PROJECT:

- @zitadel/zitadel (root commands)
- @zitadel/api,
- @zitadel/login,
- @zitadel/console,
- @zitadel/docs,
- @zitadel/client
- @zitadel/proto

The project names and folders are streamlined:

| Old Folder | New Folder |
| --- | --- |
| ./e2e | ./tests/functional-ui |
| ./load-test | ./benchmark |
| ./build/zitadel | ./apps/api |
| ./console | ./apps/console (postponed so the PR is reviewable) |

Also, all references to the TypeScript repo are removed so we can
archive it.

# How the Problems Are Solved

- Ran `npx nx@latest init`
- Replaced all turbo.json by project.json and fixed the target configs
- Removed Turbo dependency
- All JavaScript related code affected by a PRs changes is
quality-checked using the `nx affected` command
- We move PR checks that are runnable using Nx into the `check`
workflow. For workflows where we don't use Nx, yet, we restore
previously built dependency artifacts from Nx.
- We only use a single and easy to understand dev container
- The CONTRIBUTING.md is streamlined
- The setup with a generated client pat is orchestrated with Nx
- Everything related to the TypeScript repo is updated or removed. A
**Deploy with Vercel** button is added to the docs and the
CONTRIBUTING.md.

# Additional Changes

- NPM package names have a consistent pattern.
- Docker bake is removed. The login container is built and released like
the core container.
- The integration tests build the login container before running, so
they don't rely on the login container action anymore. This fixes
consistently failing checks on PRs from forks.
- The docs build in GitHub actions is removed, as we already build on
Vercel.

# Additional Context

- Internal discussion:
https://zitadel.slack.com/archives/C087ADF8LRX/p1756277884928169
- Workflow dispatch test:
https://github.com/zitadel/zitadel/actions/runs/17760122959

---------

Co-authored-by: Florian Forster <florian@zitadel.com>
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
(cherry picked from commit f69a6ed4f3)

# Conflicts:
#	.github/workflows/build.yml
#	.github/workflows/console.yml
#	.github/workflows/core.yml
#	CONTRIBUTING.md
#	Makefile
#	backend/v3/storage/database/events_testing/events_test.go
#	backend/v3/storage/database/events_testing/id_provider_instance_test.go
#	backend/v3/storage/database/events_testing/instance_test.go
#	console/README.md
#	console/package.json
#	internal/api/grpc/group/v2/integration_test/query_test.go
#	pnpm-lock.yaml
 2025-10-09 16:53:19 +02:00
Elio Bischof
8724e5f500 chore: fix make console_dependencies (#10452) 
# Which Problems Are Solved

The PR ensures that the `console_dependencies` Make recipe installs all
dependencies needed to build the console. This makes for example `make
compile` work again.

# How the Problems Are Solved

- For the current pnpm version 10, dependency overrides must be moved
from the package.json to the pnpm-workspace.yaml.
- The syntax for selecting a pnpm package and its workspace dependencies
is fixed.

# Additional Context

- Closes https://github.com/zitadel/zitadel/issues/10435

---------

Co-authored-by: Marco A. <marco@zitadel.com>
(cherry picked from commit 0dffb27fcf)
 2025-08-15 14:46:26 +02:00
Elio Bischof
b10455b51f chore: reproducible pipeline with dev containers (#10305) 
# Which Problems Are Solved

- The previous monorepo in monorepo structure for the login app and its
related packages was fragmented, complicated and buggy.
- The process for building and testing the login container was
inconsistent between local development and CI.
- Lack of clear documentation as well as easy and reliable ways for
non-frontend developers to reproduce and fix failing PR checks locally.

# How the Problems Are Solved

- Consolidated the login app and its related npm packages by moving the
main package to `apps/login/apps/login` and merging
`apps/login/packages/integration` and `apps/login/packages/acceptance`
into the main `apps/login` package.
- Migrated from Docker Compose-based test setups to dev container-based
setups, adding support for multiple dev container configurations:
  - `.devcontainer/base`
  - `.devcontainer/turbo-lint-unit`
  - `.devcontainer/turbo-lint-unit-debug`
  - `.devcontainer/login-integration`
  - `.devcontainer/login-integration-debug`
- Added npm scripts to run the new dev container setups, enabling exact
reproduction of GitHub PR checks locally, and updated the pipeline to
use these containers.
- Cleaned up Dockerfiles and docker-bake.hcl files to only build the
production image for the login app.
- Cleaned up compose files to focus on dev environments in dev
containers.
- Updated `CONTRIBUTING.md` with guidance on running and debugging PR
checks locally using the new dev container approach.
- Introduced separate Dockerfiles for the login app to distinguish
between using published client packages and building clients from local
protos.
- Ensured the login container is always built in the pipeline for use in
integration and acceptance tests.
- Updated Makefile and GitHub Actions workflows to use
`--frozen-lockfile` for installing pnpm packages, ensuring reproducible
installs.
- Disabled GitHub release creation by the changeset action.
- Refactored the `/build` directory structure for clarity and
maintainability.
- Added a `clean` command to `docks/package.json`.
- Experimentally added `knip` to the `zitadel-client` package for
improved linting of dependencies and exports.

# Additional Changes

- Fixed Makefile commands for consistency and reliability.
- Improved the structure and clarity of the `/build` directory to
support seamless integration of the login build.
- Enhanced documentation and developer experience for running and
debugging CI checks locally.

# Additional Context

- See updated `CONTRIBUTING.md` for new local development and debugging
instructions.
- These changes are a prerequisite for further improvements to the CI
pipeline and local development workflow.
- Closes #10276
 2025-07-24 14:22:32 +02:00
Max Peintner
312b7b6010 chore: 🚀 Migrate monorepo from Yarn to pnpm + Turbo integration + Configuration cleanup (#10165) 
This PR modernizes the ZITADEL monorepo build system by migrating from
Yarn to pnpm, introducing Turbo for improved build orchestration, and
cleaning up configuration inconsistencies across all apps and packages.

### 🎯 Key Improvements

#### 📦 **Package Manager Migration (Yarn → pnpm)**
- **Performance**: Faster installs with pnpm's efficient symlink-based
node_modules structure
- **Disk space**: Significant reduction in disk usage through
content-addressable storage
- **Lockfile**: More reliable dependency resolution with pnpm-lock.yaml
- **Workspace support**: Better monorepo dependency management

#### ⚡ **Turbo Integration**
- **Build orchestration**: Dependency-aware task execution across the
monorepo
- **Intelligent caching**: Dramatically faster builds on CI/CD and local
development
- **Parallel execution**: Optimal task scheduling based on dependency
graphs
- **Vercel optimization**: Enhanced build performance and caching on
Vercel deployments

#### 🧹 **Configuration Cleanup & Unification**
- **Removed config packages**: Eliminated `@zitadel/*-config` packages
and inlined configurations
- **Simplified dependencies**: Reduced complexity in package.json files
across all apps
- **Consistent tooling**: Unified prettier, ESLint, and TypeScript
configurations
- **Standalone support**: Improved prepare-standalone.js script for
subtree deployments

### 📋 Detailed Changes

#### **🔧 Build System & Dependencies**
- ✅ Updated all package.json scripts to use `pnpm` instead of `yarn`
- ✅ Replaced `yarn.lock` with pnpm-lock.yaml and regenerated
dependencies
- ✅ Added Turbo configuration (turbo.json) to root and individual
packages
- ✅ Configured proper dependency chains: `@zitadel/proto#generate` →
`@zitadel/client#build` → `console#build`
- ✅ Added missing `@bufbuild/protobuf` dependency to console app for
TypeScript compilation

#### **🚀 CI/CD & Workflows**
- ✅ Updated all GitHub Actions workflows to use `pnpm/action-setup@v4`
- ✅ Migrated build processes to use Turbo with directory-based filters
(`--filter=./console`)
- ✅ **New**: Added `docs.yml` workflow for building documentation
locally (helpful for contributors without Vercel access)
- ✅ Fixed dependency resolution issues in lint workflows
- ✅ Ensured proto generation always runs before builds and linting

#### **📚 Documentation & Proto Generation**
- ✅ **Robust plugin management**: Enhanced plugin-download.sh with retry
logic and error handling
- ✅ **Vercel compatibility**: Fixed protoc-gen-connect-openapi plugin
availability in Vercel builds
- ✅ **API docs generation**: Resolved Docusaurus build errors with
OpenAPI plugin configuration
- ✅ **Type safety**: Improved TypeScript type extraction patterns in
Angular components

#### **🛠️ Developer Experience**
- ✅ Updated all README files to reference pnpm commands
- ✅ Improved Makefile targets to use Turbo for consistent builds
- ✅ Enhanced standalone build process for login app subtree deployments
- ✅ Added debug utilities for troubleshooting build issues

#### **🗂️ File Structure & Cleanup**
- ✅ Removed obsolete configuration packages and their references
- ✅ Cleaned up Docker files to remove non-existent package copies
- ✅ Updated workspace references and import paths
- ✅ Streamlined turbo.json configurations across all packages

### 🎉 Benefits

1. **⚡ Faster Builds**: Turbo's caching and parallel execution
significantly reduce build times
2. **🔄 Better Caching**: Improved cache hits on Vercel and CI/CD
environments
3. **🛠️ Simplified Maintenance**: Unified tooling and configuration
management
4. **📈 Developer Productivity**: Faster local development with optimized
dependency resolution
5. **🚀 Enhanced CI/CD**: More reliable and faster automated builds and
deployments
6. **📖 Better Documentation**: Comprehensive build documentation and
troubleshooting guides

### 🧪 Testing

- ✅ All apps build successfully with new pnpm + Turbo setup
- ✅ Proto generation works correctly across console, login, and docs
- ✅ GitHub Actions workflows pass with new configuration
- ✅ Vercel deployments work with enhanced plugin management
- ✅ Local development workflow verified and documented

This migration sets a solid foundation for future development while
maintaining backward compatibility and improving the overall developer
experience.

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
 2025-07-16 09:10:19 +02:00