Max Peintner
312b7b6010
chore: 🚀 Migrate monorepo from Yarn to pnpm + Turbo integration + Configuration cleanup ( #10165 )
...
This PR modernizes the ZITADEL monorepo build system by migrating from
Yarn to pnpm, introducing Turbo for improved build orchestration, and
cleaning up configuration inconsistencies across all apps and packages.
### 🎯 Key Improvements
#### 📦 **Package Manager Migration (Yarn → pnpm)**
- **Performance**: Faster installs with pnpm's efficient symlink-based
node_modules structure
- **Disk space**: Significant reduction in disk usage through
content-addressable storage
- **Lockfile**: More reliable dependency resolution with pnpm-lock.yaml
- **Workspace support**: Better monorepo dependency management
#### ⚡ **Turbo Integration**
- **Build orchestration**: Dependency-aware task execution across the
monorepo
- **Intelligent caching**: Dramatically faster builds on CI/CD and local
development
- **Parallel execution**: Optimal task scheduling based on dependency
graphs
- **Vercel optimization**: Enhanced build performance and caching on
Vercel deployments
#### 🧹 **Configuration Cleanup & Unification**
- **Removed config packages**: Eliminated `@zitadel/*-config` packages
and inlined configurations
- **Simplified dependencies**: Reduced complexity in package.json files
across all apps
- **Consistent tooling**: Unified prettier, ESLint, and TypeScript
configurations
- **Standalone support**: Improved prepare-standalone.js script for
subtree deployments
### 📋 Detailed Changes
#### **🔧 Build System & Dependencies**
- ✅ Updated all package.json scripts to use `pnpm` instead of `yarn`
- ✅ Replaced `yarn.lock` with pnpm-lock.yaml and regenerated
dependencies
- ✅ Added Turbo configuration (turbo.json) to root and individual
packages
- ✅ Configured proper dependency chains: `@zitadel/proto#generate` →
`@zitadel/client#build` → `console#build`
- ✅ Added missing `@bufbuild/protobuf` dependency to console app for
TypeScript compilation
#### **🚀 CI/CD & Workflows**
- ✅ Updated all GitHub Actions workflows to use `pnpm/action-setup@v4`
- ✅ Migrated build processes to use Turbo with directory-based filters
(`--filter=./console`)
- ✅ **New**: Added `docs.yml` workflow for building documentation
locally (helpful for contributors without Vercel access)
- ✅ Fixed dependency resolution issues in lint workflows
- ✅ Ensured proto generation always runs before builds and linting
#### **📚 Documentation & Proto Generation**
- ✅ **Robust plugin management**: Enhanced plugin-download.sh with retry
logic and error handling
- ✅ **Vercel compatibility**: Fixed protoc-gen-connect-openapi plugin
availability in Vercel builds
- ✅ **API docs generation**: Resolved Docusaurus build errors with
OpenAPI plugin configuration
- ✅ **Type safety**: Improved TypeScript type extraction patterns in
Angular components
#### **🛠️ Developer Experience**
- ✅ Updated all README files to reference pnpm commands
- ✅ Improved Makefile targets to use Turbo for consistent builds
- ✅ Enhanced standalone build process for login app subtree deployments
- ✅ Added debug utilities for troubleshooting build issues
#### **🗂️ File Structure & Cleanup**
- ✅ Removed obsolete configuration packages and their references
- ✅ Cleaned up Docker files to remove non-existent package copies
- ✅ Updated workspace references and import paths
- ✅ Streamlined turbo.json configurations across all packages
### 🎉 Benefits
1. **⚡ Faster Builds**: Turbo's caching and parallel execution
significantly reduce build times
2. **🔄 Better Caching**: Improved cache hits on Vercel and CI/CD
environments
3. **🛠️ Simplified Maintenance**: Unified tooling and configuration
management
4. **📈 Developer Productivity**: Faster local development with optimized
dependency resolution
5. **🚀 Enhanced CI/CD**: More reliable and faster automated builds and
deployments
6. **📖 Better Documentation**: Comprehensive build documentation and
troubleshooting guides
### 🧪 Testing
- ✅ All apps build successfully with new pnpm + Turbo setup
- ✅ Proto generation works correctly across console, login, and docs
- ✅ GitHub Actions workflows pass with new configuration
- ✅ Vercel deployments work with enhanced plugin management
- ✅ Local development workflow verified and documented
This migration sets a solid foundation for future development while
maintaining backward compatibility and improving the overall developer
experience.
---------
Co-authored-by: Elio Bischof <elio@zitadel.com >
2025-07-16 09:10:19 +02:00
Livio Spring
618143931b
chore(ci): fix container build ( #9765 )
...
<!--
Please inform yourself about the contribution guidelines on submitting a
PR here:
https://github.com/zitadel/zitadel/blob/main/CONTRIBUTING.md#submit-a-pull-request-pr .
Take note of how PR/commit titles should be written and replace the
template texts in the sections below. Don't remove any of the sections.
It is important that the commit history clearly shows what is changed
and why.
Important: By submitting a contribution you agree to the terms from our
Licensing Policy as described here:
https://github.com/zitadel/zitadel/blob/main/LICENSING.md#community-contributions .
-->
# Which Problems Are Solved
While creating a new release, the [pipeline
failed](https://github.com/zitadel/zitadel/actions/runs/14509737111/job/40705906723 )
as GH sunset the old actions cache service:
https://github.blog/changelog/2025-03-20-notification-of-upcoming-breaking-changes-in-github-actions/#decommissioned-cache-service-brownouts
# How the Problems Are Solved
The `driver-opts` parameter is removed from the buildx actions to use
the latest stable image. ([new cache service is used by BuildKit >=
v0.20.0](https://docs.docker.com/build/ci/github-actions/cache/#cache-backend-api ))
# Additional Changes
Updated docker/build-push-action to v6 in a first attempt to solve the
issue, but kept it as it gave some more insights (incl. build summary)
# Additional Context
Since the containers are only built on workflow triggers, here's the
corresponding pipeline run:
https://github.com/zitadel/zitadel/actions/runs/14513926232
2025-04-22 06:22:54 +00:00
Livio Spring
3424204291
chore(workflow): run jobs on public runners ( #9643 )
...
# Which Problems Are Solved
Build and test workflows are currently running on specific GitHub hosted
runners. These is not needed for most worklfows and just costs more.
# How the Problems Are Solved
Moved all the steps apart from integration-tests to public runners.
# Additional Changes
None
# Additional Context
None
2025-03-26 10:06:46 +00:00
Elio Bischof
fdf0434133
fix(console): remove navigation flakiness ( #8439 )
...
# Which Problems Are Solved
The navigation in the console default settings is flaky. Sometimes it
arbitrarily jumps to the organizations page.
# How the Problems Are Solved
The lifecycle hooks were extended to react differently to changes that
come from 'outside' and from the component itself.
# Additional Changes
The e2e tests are supposed to run against Firefox and Chrome. However
they are run twice against Electon. Fixing this revealed the console
navigation flakiness that was less visible on Electron.
The following issues are also fixed with this PR to reduce flakiness in
e2e tests.
- The custom command in the pipeline is removed from the e2e action
step, so the browser argument is respected.
- The npm packages of the e2e tests are updated to their latest version.
- Notification tests run against a clean state now so they don't depend
on each other anymore. This resolved some flakiness and improved
debuggability of the tests.
- E2E page load timeout is increased, reducing flakiness.
- E2E tests wait on some elements to be enabled before they interact
with them, reducing flakiness.
# Additional Context
- Closes #8404
- Follow-up: https://github.com/zitadel/zitadel/issues/8471
The e2e tests ran three times in a row successfully in the pipeline
against both browsers.
---------
Co-authored-by: Max Peintner <max@caos.ch >
Co-authored-by: Livio Spring <livio.a@gmail.com >
Co-authored-by: Tim Möhlmann <tim+github@zitadel.com >
2024-08-22 09:48:36 +02:00
Silvan
0bfcf2c317
refactor(query): use new packages for org by id query ( #7826 )
...
If the feature is enabled the new packages are used to query org by id
Part of: https://github.com/zitadel/zitadel/issues/7639
### Definition of Ready
- [x] I am happy with the code
- [x] Short description of the feature/issue is added in the pr
description
- [x] PR is linked to the corresponding user story
- [ ] Acceptance criteria are met
- [ ] All open todos and follow ups are defined in a new ticket and
justified
- [ ] Deviations from the acceptance criteria and design are agreed with
the PO and documented.
- [x] No debug or dead code
- [x] My code has no repetitions
- [ ] Critical parts are tested automatically
- [ ] Where possible E2E tests are implemented
- [ ] Documentation/examples are up-to-date
- [ ] All non-functional requirements are met
- [x] Functionality of the acceptance criteria is checked manually on
the dev system.
2024-05-24 13:32:57 +02:00
Livio Spring
f862e43ede
chore(workflow): run e2e on non standard runners ( #7698 )
...
Co-authored-by: Elio Bischof <elio@zitadel.com >
2024-04-03 22:15:00 +00:00
Livio Spring
4c945f8bdc
chore(workflow): stop previous GH action pipeline on new changes and move back to public runners ( #7659 )
...
* chore(workflow): stop previous GH action pipeline on new changes
* skip previous code ql actions
* try running e2e on gh runner again
2024-03-28 07:15:03 +00:00
Livio Spring
5d2cfc06d5
chore: update github actions ( #7564 )
2024-03-14 09:26:31 +00:00
Livio Spring
98bd355ce2
chore(workflow): update github actions ( #7508 )
2024-03-07 15:38:38 +00:00
dependabot[bot]
c03c054aea
chore(deps): bump docker/setup-qemu-action from 2 to 3 ( #7474 )
...
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action ) from 2 to 3.
- [Release notes](https://github.com/docker/setup-qemu-action/releases )
- [Commits](https://github.com/docker/setup-qemu-action/compare/v2...v3 )
---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-06 08:29:51 +00:00
dependabot[bot]
a8dbb052de
chore(deps): bump actions/upload-artifact from 3 to 4 ( #7471 )
...
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact ) from 3 to 4.
- [Release notes](https://github.com/actions/upload-artifact/releases )
- [Commits](https://github.com/actions/upload-artifact/compare/v3...v4 )
---
updated-dependencies:
- dependency-name: actions/upload-artifact
dependency-type: direct:production
update-type: version-update:semver-major
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-06 07:26:03 +00:00
Silvan
0948a0b9ae
ci: set runner group on parallel jobs ( #6916 )
2023-11-14 10:48:41 +00:00
Stefan Benz
77e561af72
chore: add assets to releases ( #6346 )
...
* chore: change pipeline to add assets to the release
2023-08-11 14:49:28 +02:00
Stefan Benz
3b9dabcf36
chore: build image for digest and release on workflow_dispatch ( #6287 )
2023-08-10 12:21:01 +00:00
Silvan
1c354ca977
ci: improve performance ( #5953 )
...
* pipeline runs on ubuntu instead of docker
* added Makefile to build zitadel core (backend) and console (frontend)
* pipeline runs in parallel where possible
* pipeline is split into multiple jobs
* removed goreleaser
* added command to check if zitadel instance is running
2023-07-17 10:08:20 +02:00
Elio Bischof
abacb6c5aa
chore: improve development for non-Linux contributors ( #5288 )
...
* test: fix e2e against console dev server
* chore: get rid of network_mode host
* explain e2e commands
* chore: fix pipelines
* fix e2e paths
* fix dockerized e2e
* chore: map cypress run service ports
* simplify localhost
* access db via compose service
* access db via compose service
* fix npm run open:angular and e2e:angular
* docs: add empty line
* chore: remove unused file
* docs: update contrib
---------
Co-authored-by: adlerhurst <silvan.reusser@gmail.com >
2023-02-28 17:49:19 +00:00
Livio Spring
fa1e6c0bb2
chore(workflow): update actions (node 16) ( #5177 )
2023-02-17 12:31:55 +01:00
Elio Bischof
c0f65088a1
ci(e2e): fix typo in workflow ( #4527 )
2022-10-10 10:46:35 +02:00
Christian Jakob
0b7ca8a143
ci(e2e): get correct last semver tag ( #4497 )
...
fix: get last senver tag
2022-10-07 09:18:47 +00:00
Elio Bischof
c98170c19b
ci(e2e): save interesting container logs ( #4442 )
...
* ci(e2e): save prepare logs
* workflow run and dispatch logs
* ensure artifact dir exists
* fix steps order
2022-09-23 13:37:39 +02:00
Christian Jakob
9bf656deec
chore: changed trigger to follow release ( #4163 )
...
* fix: changed trigger to follow release
* fix: changed variable trigger
2022-08-11 09:28:01 +00:00
Christian Jakob
33e4c5c151
chore: added multiple browser support ( #4161 )
...
* chore: added multiple browser support
* chore: use matrix for parallel jobs
* fix: browsername
2022-08-10 17:24:19 +02:00
Elio Bischof
52f098e6ad
fix e2e workflow ( #4133 )
...
* fix e2e workflow
* configure registry once
2022-08-05 20:23:10 +02:00
Max Peintner
fc99ec87c5
ci(e2e): Run Tests in Pipelines ( #3903 )
...
* cy10 changes
* test: setup local e2e env
* test(e2e): migrate e2e setup
* add more config
* make e2e setup work
* align variables
* fix config
* skip mfa
* set user register to false
* read ids from database if not provided
* don't read ids withing env file
* fix escaping in id queries
* fix project root
* export projectRoot path
* export projectRoot
* add e2e-setup.sh
* specify GOOS and GOARCH for dockerfile compatible binary
* add org default redirect uri
* correctly initialize org policy
* await ids
* fix awaiting ids
* fix cypress configuration
* fix some tests
* initial compose setup
* fix working directory
* fix references
* make tests less flaky
* run go tests
* compose works until e2e-setup incl
* pass created e2e sa key
* make cypress run
* derive e2e orgs domain from baseurl
* use host from baseurl for setup ctx
* move defaults.yaml back to cmd pkg
* just create org owner
* Don't render element if no roles are passed
* use map instead of switchMap
* fix e2e tests
* added testdata for e3e
* zipped dump
* removed dumpDir
* cypress workflow with compose
* quote name
* cleanup vars
* eliminate need for e2e setup
* compose has no builds anymore
* use compose run and zitadel nw
* test e2e on pr (#4114 )
* test e2e on pr
* install goreleaser
* install npm dev dependencies
* run cypress wf
* dynamic release version
* skip flaky user tests
* skip flaky permissions test
* cache docker layers in pipeline
* Update .github/workflows/cypress.yml
Co-authored-by: Florian Forster <florian@caos.ch >
* align goreleaser version
* get rid of install.sh
* remove cypress-terminal-report
* Revert "remove cypress-terminal-report"
This reverts commit 254b5a1f87florian@caos.ch >
Co-authored-by: Christian Jakob <christian@caos.ch >
* cleanup
Co-authored-by: Elio Bischof <eliobischof@gmail.com >
Co-authored-by: Christian Jakob <christian@caos.ch >
Co-authored-by: Florian Forster <florian@caos.ch >
2022-08-05 20:00:46 +02:00
