chore: 🚀 Migrate monorepo from Yarn to pnpm + Turbo integration + Configuration cleanup (#10165)

This PR modernizes the ZITADEL monorepo build system by migrating from
Yarn to pnpm, introducing Turbo for improved build orchestration, and
cleaning up configuration inconsistencies across all apps and packages.

### 🎯 Key Improvements

#### 📦 **Package Manager Migration (Yarn → pnpm)**
- **Performance**: Faster installs with pnpm's efficient symlink-based
node_modules structure
- **Disk space**: Significant reduction in disk usage through
content-addressable storage
- **Lockfile**: More reliable dependency resolution with pnpm-lock.yaml
- **Workspace support**: Better monorepo dependency management

#### ⚡ **Turbo Integration**
- **Build orchestration**: Dependency-aware task execution across the
monorepo
- **Intelligent caching**: Dramatically faster builds on CI/CD and local
development
- **Parallel execution**: Optimal task scheduling based on dependency
graphs
- **Vercel optimization**: Enhanced build performance and caching on
Vercel deployments

#### 🧹 **Configuration Cleanup & Unification**
- **Removed config packages**: Eliminated `@zitadel/*-config` packages
and inlined configurations
- **Simplified dependencies**: Reduced complexity in package.json files
across all apps
- **Consistent tooling**: Unified prettier, ESLint, and TypeScript
configurations
- **Standalone support**: Improved prepare-standalone.js script for
subtree deployments

### 📋 Detailed Changes

#### **🔧 Build System & Dependencies**
- ✅ Updated all package.json scripts to use `pnpm` instead of `yarn`
- ✅ Replaced `yarn.lock` with pnpm-lock.yaml and regenerated
dependencies
- ✅ Added Turbo configuration (turbo.json) to root and individual
packages
- ✅ Configured proper dependency chains: `@zitadel/proto#generate` →
`@zitadel/client#build` → `console#build`
- ✅ Added missing `@bufbuild/protobuf` dependency to console app for
TypeScript compilation

#### **🚀 CI/CD & Workflows**
- ✅ Updated all GitHub Actions workflows to use `pnpm/action-setup@v4`
- ✅ Migrated build processes to use Turbo with directory-based filters
(`--filter=./console`)
- ✅ **New**: Added `docs.yml` workflow for building documentation
locally (helpful for contributors without Vercel access)
- ✅ Fixed dependency resolution issues in lint workflows
- ✅ Ensured proto generation always runs before builds and linting

#### **📚 Documentation & Proto Generation**
- ✅ **Robust plugin management**: Enhanced plugin-download.sh with retry
logic and error handling
- ✅ **Vercel compatibility**: Fixed protoc-gen-connect-openapi plugin
availability in Vercel builds
- ✅ **API docs generation**: Resolved Docusaurus build errors with
OpenAPI plugin configuration
- ✅ **Type safety**: Improved TypeScript type extraction patterns in
Angular components

#### **🛠️ Developer Experience**
- ✅ Updated all README files to reference pnpm commands
- ✅ Improved Makefile targets to use Turbo for consistent builds
- ✅ Enhanced standalone build process for login app subtree deployments
- ✅ Added debug utilities for troubleshooting build issues

#### **🗂️ File Structure & Cleanup**
- ✅ Removed obsolete configuration packages and their references
- ✅ Cleaned up Docker files to remove non-existent package copies
- ✅ Updated workspace references and import paths
- ✅ Streamlined turbo.json configurations across all packages

### 🎉 Benefits

1. **⚡ Faster Builds**: Turbo's caching and parallel execution
significantly reduce build times
2. **🔄 Better Caching**: Improved cache hits on Vercel and CI/CD
environments
3. **🛠️ Simplified Maintenance**: Unified tooling and configuration
management
4. **📈 Developer Productivity**: Faster local development with optimized
dependency resolution
5. **🚀 Enhanced CI/CD**: More reliable and faster automated builds and
deployments
6. **📖 Better Documentation**: Comprehensive build documentation and
troubleshooting guides

### 🧪 Testing

- ✅ All apps build successfully with new pnpm + Turbo setup
- ✅ Proto generation works correctly across console, login, and docs
- ✅ GitHub Actions workflows pass with new configuration
- ✅ Vercel deployments work with enhanced plugin management
- ✅ Local development workflow verified and documented

This migration sets a solid foundation for future development while
maintaining backward compatibility and improving the overall developer
experience.

---------

Co-authored-by: Elio Bischof <elio@zitadel.com>

console/.gitignore

@@ -36,7 +36,7 @@ speed-measure-plugin*.json
 /coverage
 /libpeerconnection.log
 npm-debug.log
-yarn-error.log
+pnpm-debug.log
 testem.log
 /typings
 

console/README.md

@@ -1,27 +1,137 @@
-# Console
+# Console Angular App
 
-This project was generated with [Angular CLI](https://github.com/angular/angular-cli) version 8.3.20.
+This is the ZITADEL Console Angular application.
 
-## Development server
+## Development
 
-Run `ng serve` for a dev server. Navigate to `http://localhost:4200/`. The app will automatically reload if you change any of the source files.
+### Prerequisites
 
-## Code scaffolding
+- Node.js 18 or later
+- pnpm (latest)
 
-Run `ng generate component component-name` to generate a new component. You can also use `ng generate directive|pipe|service|class|guard|interface|enum|module`.
+### Installation
 
-## Build
+```bash
+pnpm install
+```
 
-Run `ng build` to build the project. The build artifacts will be stored in the `dist/` directory. Use the `--prod` flag for a production build.
+### Proto Generation
 
-## Running unit tests
+The Console app uses **dual proto generation** with Turbo dependency management:
 
-Run `ng test` to execute the unit tests via [Karma](https://karma-runner.github.io).
+1. **`@zitadel/proto` generation**: Modern ES modules with `@bufbuild/protobuf` for v2 APIs
+2. **Local `buf.gen.yaml` generation**: Traditional protobuf JavaScript classes for v1 APIs
 
-## Running end-to-end tests
+The Console app's `turbo.json` ensures that `@zitadel/proto#generate` runs before the Console's own generation, providing both:
 
-Please refer to the [contributing guide](../CONTRIBUTING.md#console)
+- Modern schemas from `@zitadel/proto` (e.g., `UserSchema`, `DetailsSchema`)
+- Legacy classes from `src/app/proto/generated` (e.g., `User`, `Project`)
 
-## Further help
+Generated files:
 
-To get more help on the Angular CLI use `ng help` or go check out the [Angular CLI README](https://github.com/angular/angular-cli/blob/master/README.md).
+- **`@zitadel/proto`**: Modern ES modules in `login/packages/zitadel-proto/`
+- **Local generation**: Traditional protobuf files in `src/app/proto/generated/`
+  - TypeScript definition files (`.d.ts`)
+  - JavaScript files (`.js`)
+  - gRPC client files (`*ServiceClientPb.ts`)
+  - OpenAPI/Swagger JSON files (`.swagger.json`)
+
+To generate proto files:
+
+```bash
+pnpm run generate
+```
+
+This automatically runs both generations in the correct order via Turbo dependencies.
+
+### Development Server
+
+To start the development server:
+
+```bash
+pnpm start
+```
+
+This will:
+
+1. Fetch the environment configuration from the server
+2. Serve the app on the default port
+
+### Building
+
+To build for production:
+
+```bash
+pnpm run build
+```
+
+This will:
+
+1. Generate proto files (via `prebuild` script)
+2. Build the Angular app with production optimizations
+
+### Linting
+
+To run linting and formatting checks:
+
+```bash
+pnpm run lint
+```
+
+To auto-fix formatting issues:
+
+```bash
+pnpm run lint:fix
+```
+
+## Project Structure
+
+- `src/app/proto/generated/` - Generated proto files (Angular-specific format)
+- `buf.gen.yaml` - Local proto generation configuration
+- `turbo.json` - Turbo dependency configuration for proto generation
+- `prebuild.development.js` - Development environment configuration script
+
+## Proto Generation Details
+
+The Console app uses **dual proto generation** managed by Turbo dependencies:
+
+### Dependency Chain
+
+The Console app has the following build dependencies managed by Turbo:
+
+1. `@zitadel/proto#generate` - Generates modern protobuf files
+2. `@zitadel/client#build` - Builds the TypeScript gRPC client library
+3. `console#generate` - Generates Console-specific protobuf files
+4. `console#build` - Builds the Angular application
+
+This ensures that the Console always has access to the latest client library and protobuf definitions.
+
+### Legacy v1 API (Traditional Protobuf)
+
+- Uses local `buf.gen.yaml` configuration
+- Generates traditional Google protobuf JavaScript classes extending `jspb.Message`
+- Uses plugins: `protocolbuffers/js`, `grpc/web`, `grpc-ecosystem/openapiv2`
+- Output: `src/app/proto/generated/`
+- Used for: Most existing Console functionality
+
+### Modern v2 API (ES Modules)
+
+- Uses `@zitadel/proto` package generation
+- Generates modern ES modules with `@bufbuild/protobuf`
+- Uses plugin: `@bufbuild/es` with ES modules and JSON types
+- Output: `login/packages/zitadel-proto/`
+- Used for: New user v2 API and services
+
+### Dependency Management
+
+The Console's `turbo.json` ensures proper execution order:
+
+1. `@zitadel/proto#generate` runs first (modern ES modules)
+2. Console's local generation runs second (traditional protobuf)
+3. Build/lint/start tasks depend on both generations being complete
+
+This approach allows the Console app to use both v1 and v2 APIs while maintaining proper build dependencies.
+
+## Legacy Information
+
+This project was originally generated with Angular CLI version 8.3.20 and has been updated over time.

console/package.json

@@ -3,12 +3,12 @@
   "version": "0.0.0",
   "scripts": {
     "ng": "ng",
+    "dev": "node prebuild.development.js && ng serve",
     "start": "node prebuild.development.js && ng serve",
     "build": "ng build --configuration production --base-href=/ui/console/",
-    "prelint": "npm run generate",
     "lint": "ng lint && prettier --check src",
     "lint:fix": "prettier --write src",
-    "generate": "buf generate ../proto --include-imports --include-wkt"
+    "generate": "pnpm exec buf generate ../proto --include-imports --include-wkt"
   },
   "private": true,
   "dependencies": {
@@ -24,6 +24,7 @@
     "@angular/platform-browser-dynamic": "^16.2.12",
     "@angular/router": "^16.2.12",
     "@angular/service-worker": "^16.2.12",
+    "@bufbuild/protobuf": "^2.2.2",
     "@connectrpc/connect": "^2.0.0",
     "@connectrpc/connect-web": "^2.0.0",
     "@ctrl/ngx-codemirror": "^6.1.0",
@@ -31,8 +32,8 @@
     "@fortawesome/fontawesome-svg-core": "^6.7.2",
     "@fortawesome/free-brands-svg-icons": "^6.7.2",
     "@ngx-translate/core": "^15.0.0",
-    "@zitadel/client": "1.2.0",
-    "@zitadel/proto": "1.2.0",
+    "@zitadel/client": "workspace:*",
+    "@zitadel/proto": "workspace:*",
     "angular-oauth2-oidc": "^15.0.1",
     "angularx-qrcode": "^16.0.2",
     "buffer": "^6.0.3",

console/src/app/components/features/features.component.ts

@@ -139,7 +139,7 @@ export class FeaturesComponent {
     }, {});
 
     // to save special flags they have to be handled here
-    req.loginV2 = {
+    req['loginV2'] = {
       required: toggleStates.loginV2.enabled,
       baseUri: toggleStates.loginV2.baseUri,
     };

console/src/app/modules/actions-two/actions-two-add-target/actions-two-add-target-dialog.component.ts

@@ -89,7 +89,7 @@ export class ActionTwoAddTargetDialogComponent {
       nanos: 0,
     };
 
-    const targetType: Extract<MessageInitShape<typeof CreateTargetRequestSchema>['targetType'], { case: TargetTypes }> =
+    const targetType: MessageInitShape<typeof CreateTargetRequestSchema>['targetType'] =
       type === 'restWebhook'
         ? { case: type, value: { interruptOnError } }
         : type === 'restCall'

console/src/app/modules/policies/oidc-webkeys/oidc-webkeys.component.ts

@@ -22,9 +22,8 @@ const CACHE_WARNING_MS = 5 * 60 * 1000; // 5 minutes
   templateUrl: './oidc-webkeys.component.html',
   changeDetection: ChangeDetectionStrategy.OnPush,
 })
-export class OidcWebKeysComponent implements OnInit {
+export class OidcWebKeysComponent {
   protected readonly refresh = new Subject<true>();
-  protected readonly webKeysEnabled$: Observable<boolean>;
   protected readonly webKeys$: Observable<WebKey[]>;
   protected readonly inactiveWebKeys$: Observable<WebKey[]>;
   protected readonly nextWebKeyCandidate$: Observable<WebKey | undefined>;
@@ -34,17 +33,12 @@ export class OidcWebKeysComponent implements OnInit {
 
   constructor(
     private readonly webKeysService: WebKeysService,
-    private readonly featureService: NewFeatureService,
     private readonly toast: ToastService,
     private readonly timestampToDatePipe: TimestampToDatePipe,
     private readonly dialog: MatDialog,
     private readonly destroyRef: DestroyRef,
-    private readonly router: Router,
-    private readonly route: ActivatedRoute,
   ) {
-    this.webKeysEnabled$ = this.getWebKeysEnabled().pipe(shareReplay({ refCount: true, bufferSize: 1 }));
-
-    const webKeys$ = this.getWebKeys(this.webKeysEnabled$).pipe(shareReplay({ refCount: true, bufferSize: 1 }));
+    const webKeys$ = this.getWebKeys().pipe(shareReplay({ refCount: true, bufferSize: 1 }));
 
     this.webKeys$ = webKeys$.pipe(map((webKeys) => webKeys.filter((webKey) => webKey.state !== State.INACTIVE)));
     this.inactiveWebKeys$ = webKeys$.pipe(map((webKeys) => webKeys.filter((webKey) => webKey.state === State.INACTIVE)));
@@ -52,34 +46,7 @@ export class OidcWebKeysComponent implements OnInit {
     this.nextWebKeyCandidate$ = this.getNextWebKeyCandidate(this.webKeys$);
   }
 
-  ngOnInit(): void {
-    // redirect away from this page if web keys are not enabled
-    // this also preloads the web keys enabled state
-    this.webKeysEnabled$.pipe(takeUntilDestroyed(this.destroyRef)).subscribe(async (webKeysEnabled) => {
-      if (webKeysEnabled) {
-        return;
-      }
-      await this.router.navigate([], {
-        relativeTo: this.route,
-        queryParamsHandling: 'merge',
-        queryParams: {
-          id: null,
-        },
-      });
-    });
-  }
-
-  private getWebKeysEnabled() {
-    return defer(() => this.featureService.getInstanceFeatures()).pipe(
-      map((features) => features.webKey?.enabled ?? false),
-      catchError((err) => {
-        this.toast.showError(err);
-        return of(false);
-      }),
-    );
-  }
-
-  private getWebKeys(webKeysEnabled$: Observable<boolean>) {
+  private getWebKeys() {
     return this.refresh.pipe(
       startWith(true),
       switchMap(() => {
@@ -87,12 +54,6 @@ export class OidcWebKeysComponent implements OnInit {
       }),
       map(({ webKeys }) => webKeys),
       catchError(async (err) => {
-        const webKeysEnabled = await firstValueFrom(webKeysEnabled$);
-        // suppress errors if web keys are not enabled
-        if (!webKeysEnabled) {
-          return [];
-        }
-
         this.toast.showError(err);
         return [];
       }),

console/src/app/pages/users/user-create/user-create-v2/user-create-v2.component.ts

@@ -204,7 +204,7 @@ export class UserCreateV2Component implements OnInit {
 
     if (authenticationFactor.factor === 'initialPassword') {
       const { password } = authenticationFactor.form.getRawValue();
-      humanReq.passwordType = {
+      humanReq['passwordType'] = {
         case: 'password',
         value: {
           password,

console/src/app/pages/users/user-detail/detail-form/detail-form.component.ts

@@ -8,12 +8,14 @@ import { Gender, HumanProfile, HumanProfileSchema } from '@zitadel/proto/zitadel
 import { filter, startWith } from 'rxjs/operators';
 import { takeUntilDestroyed } from '@angular/core/rxjs-interop';
 import { Profile } from '@zitadel/proto/zitadel/user_pb';
-import { create } from '@bufbuild/protobuf';
+//@ts-ignore
+import { create } from '@zitadel/client';
 
 function toHumanProfile(profile: HumanProfile | Profile): HumanProfile {
   if (profile.$typeName === 'zitadel.user.v2.HumanProfile') {
     return profile;
   }
+
   return create(HumanProfileSchema, {
     givenName: profile.firstName,
     familyName: profile.lastName,

console/src/app/pages/users/user-list/user-table/user-table.component.ts

@@ -36,10 +36,10 @@ import { AuthenticationService } from 'src/app/services/authentication.service';
 import { GrpcAuthService } from 'src/app/services/grpc-auth.service';
 import { UserState as UserStateV1 } from 'src/app/proto/generated/zitadel/user_pb';
 
-type Query = Exclude<
-  Exclude<MessageInitShape<typeof ListUsersRequestSchema>['queries'], undefined>[number]['query'],
-  undefined
->;
+type ListUsersRequest = MessageInitShape<typeof ListUsersRequestSchema>;
+type QueriesArray = NonNullable<ListUsersRequest['queries']>;
+type QueryWrapper = QueriesArray extends readonly (infer T)[] ? T : never;
+type Query = NonNullable<QueryWrapper extends { query?: infer Q } ? Q : never>;
 
 @Component({
   selector: 'cnsl-user-table',

console/src/app/services/user.service.ts

@@ -229,94 +229,3 @@ export class UserService {
     return this.grpcService.userNew.setPassword(create(SetPasswordRequestSchema, req));
   }
 }
-
-function userToV2(user: User): UserV2 {
-  const details = user.getDetails();
-  return create(UserSchema, {
-    userId: user.getId(),
-    details: details && detailsToV2(details),
-    state: user.getState() as number as UserState,
-    username: user.getUserName(),
-    loginNames: user.getLoginNamesList(),
-    preferredLoginName: user.getPreferredLoginName(),
-    type: typeToV2(user),
-  });
-}
-
-function detailsToV2(details: ObjectDetails): Details {
-  const changeDate = details.getChangeDate();
-  return create(DetailsSchema, {
-    sequence: BigInt(details.getSequence()),
-    changeDate: changeDate && timestampToV2(changeDate),
-    resourceOwner: details.getResourceOwner(),
-  });
-}
-
-function timestampToV2(timestamp: Timestamp): TimestampV2 {
-  return create(TimestampSchema, {
-    seconds: BigInt(timestamp.getSeconds()),
-    nanos: timestamp.getNanos(),
-  });
-}
-
-function typeToV2(user: User): UserV2['type'] {
-  const human = user.getHuman();
-  if (human) {
-    return { case: 'human', value: humanToV2(user, human) };
-  }
-
-  const machine = user.getMachine();
-  if (machine) {
-    return { case: 'machine', value: machineToV2(machine) };
-  }
-
-  return { case: undefined };
-}
-
-function humanToV2(user: User, human: Human): HumanUser {
-  const profile = human.getProfile();
-  const email = human.getEmail()?.getEmail();
-  const phone = human.getPhone();
-  const passwordChanged = human.getPasswordChanged();
-
-  return create(HumanUserSchema, {
-    userId: user.getId(),
-    state: user.getState() as number as UserState,
-    username: user.getUserName(),
-    loginNames: user.getLoginNamesList(),
-    preferredLoginName: user.getPreferredLoginName(),
-    profile: profile && humanProfileToV2(profile),
-    email: { email },
-    phone: phone && humanPhoneToV2(phone),
-    passwordChangeRequired: false,
-    passwordChanged: passwordChanged && timestampToV2(passwordChanged),
-  });
-}
-
-function humanProfileToV2(profile: Profile): HumanProfile {
-  return create(HumanProfileSchema, {
-    givenName: profile.getFirstName(),
-    familyName: profile.getLastName(),
-    nickName: profile.getNickName(),
-    displayName: profile.getDisplayName(),
-    preferredLanguage: profile.getPreferredLanguage(),
-    gender: profile.getGender() as number as Gender,
-    avatarUrl: profile.getAvatarUrl(),
-  });
-}
-
-function humanPhoneToV2(phone: Phone): HumanPhone {
-  return create(HumanPhoneSchema, {
-    phone: phone.getPhone(),
-    isVerified: phone.getIsPhoneVerified(),
-  });
-}
-
-function machineToV2(machine: Machine): MachineUser {
-  return create(MachineUserSchema, {
-    name: machine.getName(),
-    description: machine.getDescription(),
-    hasSecret: machine.getHasSecret(),
-    accessTokenType: machine.getAccessTokenType() as number as AccessTokenType,
-  });
-}

console/turbo.json

@@ -0,0 +1,28 @@
+{
+  "$schema": "https://turbo.build/schema.json",
+  "extends": ["//"],
+  "tasks": {
+    "generate": {
+      "dependsOn": ["@zitadel/proto#generate"],
+      "outputs": ["src/app/proto/generated/**"]
+    },
+    "build": {
+      "dependsOn": ["generate", "@zitadel/client#build"],
+      "outputs": ["dist/**"]
+    },
+    "dev": {
+      "dependsOn": ["generate", "@zitadel/client#build"],
+      "cache": false,
+      "persistent": true
+    },
+    "start": {
+      "dependsOn": ["generate", "@zitadel/client#build"],
+      "cache": false,
+      "persistent": true
+    },
+    "lint": {
+      "dependsOn": ["generate"],
+      "outputs": []
+    }
+  }
+}