TheArchive/zitadel
1
0
Fork 0
mirror of https://github.com/zitadel/zitadel.git synced 2025-08-11 21:37:32 +00:00
Code Issues Packages Projects Releases Wiki Activity

chore: 🚀 Migrate monorepo from Yarn to pnpm + Turbo integration + Configuration cleanup (#10165)

Browse Source 
This PR modernizes the ZITADEL monorepo build system by migrating from
Yarn to pnpm, introducing Turbo for improved build orchestration, and
cleaning up configuration inconsistencies across all apps and packages.

### 🎯 Key Improvements

#### 📦 **Package Manager Migration (Yarn → pnpm)**
- **Performance**: Faster installs with pnpm's efficient symlink-based
node_modules structure
- **Disk space**: Significant reduction in disk usage through
content-addressable storage
- **Lockfile**: More reliable dependency resolution with pnpm-lock.yaml
- **Workspace support**: Better monorepo dependency management

#### ⚡ **Turbo Integration**
- **Build orchestration**: Dependency-aware task execution across the
monorepo
- **Intelligent caching**: Dramatically faster builds on CI/CD and local
development
- **Parallel execution**: Optimal task scheduling based on dependency
graphs
- **Vercel optimization**: Enhanced build performance and caching on
Vercel deployments

#### 🧹 **Configuration Cleanup & Unification**
- **Removed config packages**: Eliminated `@zitadel/*-config` packages
and inlined configurations
- **Simplified dependencies**: Reduced complexity in package.json files
across all apps
- **Consistent tooling**: Unified prettier, ESLint, and TypeScript
configurations
- **Standalone support**: Improved prepare-standalone.js script for
subtree deployments

### 📋 Detailed Changes

#### **🔧 Build System & Dependencies**
- ✅ Updated all package.json scripts to use `pnpm` instead of `yarn`
- ✅ Replaced `yarn.lock` with pnpm-lock.yaml and regenerated
dependencies
- ✅ Added Turbo configuration (turbo.json) to root and individual
packages
- ✅ Configured proper dependency chains: `@zitadel/proto#generate` →
`@zitadel/client#build` → `console#build`
- ✅ Added missing `@bufbuild/protobuf` dependency to console app for
TypeScript compilation

#### **🚀 CI/CD & Workflows**
- ✅ Updated all GitHub Actions workflows to use `pnpm/action-setup@v4`
- ✅ Migrated build processes to use Turbo with directory-based filters
(`--filter=./console`)
- ✅ **New**: Added `docs.yml` workflow for building documentation
locally (helpful for contributors without Vercel access)
- ✅ Fixed dependency resolution issues in lint workflows
- ✅ Ensured proto generation always runs before builds and linting

#### **📚 Documentation & Proto Generation**
- ✅ **Robust plugin management**: Enhanced plugin-download.sh with retry
logic and error handling
- ✅ **Vercel compatibility**: Fixed protoc-gen-connect-openapi plugin
availability in Vercel builds
- ✅ **API docs generation**: Resolved Docusaurus build errors with
OpenAPI plugin configuration
- ✅ **Type safety**: Improved TypeScript type extraction patterns in
Angular components

#### **🛠️ Developer Experience**
- ✅ Updated all README files to reference pnpm commands
- ✅ Improved Makefile targets to use Turbo for consistent builds
- ✅ Enhanced standalone build process for login app subtree deployments
- ✅ Added debug utilities for troubleshooting build issues

#### **🗂️ File Structure & Cleanup**
- ✅ Removed obsolete configuration packages and their references
- ✅ Cleaned up Docker files to remove non-existent package copies
- ✅ Updated workspace references and import paths
- ✅ Streamlined turbo.json configurations across all packages

### 🎉 Benefits

1. **⚡ Faster Builds**: Turbo's caching and parallel execution
significantly reduce build times
2. **🔄 Better Caching**: Improved cache hits on Vercel and CI/CD
environments
3. **🛠️ Simplified Maintenance**: Unified tooling and configuration
management
4. **📈 Developer Productivity**: Faster local development with optimized
dependency resolution
5. **🚀 Enhanced CI/CD**: More reliable and faster automated builds and
deployments
6. **📖 Better Documentation**: Comprehensive build documentation and
troubleshooting guides

### 🧪 Testing

- ✅ All apps build successfully with new pnpm + Turbo setup
- ✅ Proto generation works correctly across console, login, and docs
- ✅ GitHub Actions workflows pass with new configuration
- ✅ Vercel deployments work with enhanced plugin management
- ✅ Local development workflow verified and documented

This migration sets a solid foundation for future development while
maintaining backward compatibility and improving the overall developer
experience.

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>
This commit is contained in:
Max Peintner
2025-07-16 09:10:19 +02:00
committed by GitHub
parent 6d11145c77
commit 312b7b6010
152 changed files with 34249 additions and 37195 deletions
Download Patch File Download Diff File Expand all files Collapse all files

138
console/README.md
View File

@@ -1,27 +1,137 @@
# Console
# Console Angular App
This project was generated with [Angular CLI](https://github.com/angular/angular-cli) version 8.3.20.
This is the ZITADEL Console Angular application.
## Development server
## Development
Run `ng serve` for a dev server. Navigate to `http://localhost:4200/`. The app will automatically reload if you change any of the source files.
### Prerequisites
## Code scaffolding
- Node.js 18 or later
- pnpm (latest)
Run `ng generate component component-name` to generate a new component. You can also use `ng generate directive|pipe|service|class|guard|interface|enum|module`.
### Installation
## Build
```bash
pnpm install
```
Run `ng build` to build the project. The build artifacts will be stored in the `dist/` directory. Use the `--prod` flag for a production build.
### Proto Generation
## Running unit tests
The Console app uses **dual proto generation** with Turbo dependency management:
Run `ng test` to execute the unit tests via [Karma](https://karma-runner.github.io).
1. **`@zitadel/proto` generation**: Modern ES modules with `@bufbuild/protobuf` for v2 APIs
2. **Local `buf.gen.yaml` generation**: Traditional protobuf JavaScript classes for v1 APIs
## Running end-to-end tests
The Console app's `turbo.json` ensures that `@zitadel/proto#generate` runs before the Console's own generation, providing both:
Please refer to the [contributing guide](../CONTRIBUTING.md#console)
- Modern schemas from `@zitadel/proto` (e.g., `UserSchema`, `DetailsSchema`)
- Legacy classes from `src/app/proto/generated` (e.g., `User`, `Project`)
## Further help
Generated files:
To get more help on the Angular CLI use `ng help` or go check out the [Angular CLI README](https://github.com/angular/angular-cli/blob/master/README.md).
- **`@zitadel/proto`**: Modern ES modules in `login/packages/zitadel-proto/`
- **Local generation**: Traditional protobuf files in `src/app/proto/generated/`
- TypeScript definition files (`.d.ts`)
- JavaScript files (`.js`)
- gRPC client files (`*ServiceClientPb.ts`)
- OpenAPI/Swagger JSON files (`.swagger.json`)
To generate proto files:
```bash
pnpm run generate
```
This automatically runs both generations in the correct order via Turbo dependencies.
### Development Server
To start the development server:
```bash
pnpm start
```
This will:
1. Fetch the environment configuration from the server
2. Serve the app on the default port
### Building
To build for production:
```bash
pnpm run build
```
This will:
1. Generate proto files (via `prebuild` script)
2. Build the Angular app with production optimizations
### Linting
To run linting and formatting checks:
```bash
pnpm run lint
```
To auto-fix formatting issues:
```bash
pnpm run lint:fix
```
## Project Structure
- `src/app/proto/generated/` - Generated proto files (Angular-specific format)
- `buf.gen.yaml` - Local proto generation configuration
- `turbo.json` - Turbo dependency configuration for proto generation
- `prebuild.development.js` - Development environment configuration script
## Proto Generation Details
The Console app uses **dual proto generation** managed by Turbo dependencies:
### Dependency Chain
The Console app has the following build dependencies managed by Turbo:
1. `@zitadel/proto#generate` - Generates modern protobuf files
2. `@zitadel/client#build` - Builds the TypeScript gRPC client library
3. `console#generate` - Generates Console-specific protobuf files
4. `console#build` - Builds the Angular application
This ensures that the Console always has access to the latest client library and protobuf definitions.
### Legacy v1 API (Traditional Protobuf)
- Uses local `buf.gen.yaml` configuration
- Generates traditional Google protobuf JavaScript classes extending `jspb.Message`
- Uses plugins: `protocolbuffers/js`, `grpc/web`, `grpc-ecosystem/openapiv2`
- Output: `src/app/proto/generated/`
- Used for: Most existing Console functionality
### Modern v2 API (ES Modules)
- Uses `@zitadel/proto` package generation
- Generates modern ES modules with `@bufbuild/protobuf`
- Uses plugin: `@bufbuild/es` with ES modules and JSON types
- Output: `login/packages/zitadel-proto/`
- Used for: New user v2 API and services
### Dependency Management
The Console's `turbo.json` ensures proper execution order:
1. `@zitadel/proto#generate` runs first (modern ES modules)
2. Console's local generation runs second (traditional protobuf)
3. Build/lint/start tasks depend on both generations being complete
This approach allows the Console app to use both v1 and v2 APIs while maintaining proper build dependencies.
## Legacy Information
This project was originally generated with Angular CLI version 8.3.20 and has been updated over time.