zitadel/.github/workflows/zitadel.yml
Livio Amstutz bacfc3b099
fix: operator picks (#1463)
* feat(crd): add crd mode for operators (#1329)

* feat(operator): add base for zitadel operator

* fix(operator): changed pipeline to release operator

* fix(operator): fmt with only one parameter

* fix(operator): corrected workflow job name

* fix(zitadelctl): added restore and backuplist command

* fix(zitadelctl): scale for restore

* chore(container): use scratch for deploy container

* fix(zitadelctl): limit image to scratch

* fix(migration): added migration scripts for newer version

* fix(operator): changed handling of kubeconfig in operator logic

* fix(operator): changed handling of secrets in operator logic

* fix(operator): use new version of zitadel

* fix(operator): added path for migrations

* fix(operator): delete doublets of migration scripts

* fix(operator): delete subpaths and integrate logic into init container

* fix(operator): corrected path in dockerfile for local migrations

* fix(operator): added migrations for cockroachdb-secure

* fix(operator): delete logic for ambassador module

* fix(operator): added read and write secret commands

* fix(operator): correct and align operator pipeline with zitadel pipeline

* fix(operator): correct yaml error in operator pipeline

* fix(operator): correct action name in operator pipeline

* fix(operator): correct case-sensitive filename in operator pipeline

* fix(operator): upload artifacts from buildx output

* fix(operator): corrected attribute spelling error

* fix(operator): combined jobs for operator binary and image

* fix(operator): added missing comma in operator pipeline

* fix(operator): added codecov for operator image

* fix(operator): added codecov for operator image

* fix(testing): code changes for testing and several unit-tests (#1009)

* fix(operator): usage of interface of kubernetes client for testing and several unit-tests

* fix(operator): several unit-tests

* fix(operator): several unit-tests

* fix(operator): changed order for the operator logic

* fix(operator): added version of zitadelctl from semantic release

* fix(operator): corrected function call with version of zitadelctl

* fix(operator): corrected function call with version of zitadelctl

* fix(operator): add check output to operator release pipeline

* fix(operator): set --short length everywhere to 12

* fix(operator): zitadel setup in job instead of exec with several unit tests

* fix(operator): fixes to combine newest zitadel and testing branch

* fix(operator): corrected path in Dockerfile

* fix(operator): fixed unit-test that was ignored during changes

* fix(operator): fixed unit-test that was ignored during changes

* fix(operator): corrected Dockerfile to correctly use env variable

* fix(operator): quickfix takeoff deployment

* fix(operator): corrected the clusterrolename in the applied artifacts

* fix: update secure migrations

* fix(operator): migrations (#1057)

* fix(operator): copied migrations from orbos repository

* fix(operator): newest migrations

* chore: use cockroach-secure

* fix: rename migration

* fix: remove insecure cockroach migrations

Co-authored-by: Stefan Benz <stefan@caos.ch>

* fix: finalize labels

* fix(operator): cli logging concurrent and fixe deployment of operator during restore

* fix: finalize labels and cli commands

* fix: restore

* chore: cockroachdb is always secure

* chore: use orbos consistent-labels latest commit

* test: make tests compatible with new labels

* fix: default to sa token for start command

* fix: use cockroachdb v12.02

* fix: don't delete flyway user

* test: fix migration test

* fix: use correct table qualifiers

* fix: don't alter sequence ownership

* fix: upgrade flyway

* fix: change ownership of all dbs and tables to admin user

* fix: change defaultdb user

* fix: treat clientid status codes >= 400 as errors

* fix: reconcile specified ZITADEL version, not binary version

* fix: add ca-certs

* fix: use latest orbos code

* fix: use orbos with fixed race condition

* fix: use latest ORBOS code

* fix: use latest ORBOS code

* fix: make migration and scaling around restoring work

* fix(operator): move zitadel operator

* chore(migrations): include owner change migration

* feat(db): add code base for database operator

* fix(db): change used image registry for database operator

* fix(db): generated mock

* fix(db): add accidentally ignored file

* fix(db): add cockroachdb backup image to pipeline

* fix(db): correct pipeline and image versions

* fix(db): correct version of used orbos

* fix(db): correct database import

* fix(db): go mod tidy

* fix(db): use new version for orbos

* fix(migrations): include migrations into zitadelctl binary (#1211)

* fix(db): use statik to integrate migrations into binary

* fix(migrations): corrections unit tests and pipeline for integrated migrations into zitadelctl binary

* fix(migrations): correction in dockerfile for pipeline build

* fix(migrations): correction in dockerfile for pipeline build

* fix(migrations):  dockerfile changes for cache optimization

* fix(database): correct used part-of label in database operator

* fix(database): correct used selectable label in zitadel operator

* fix(operator): correct lables for user secrets in zitadel operator

* fix(operator): correct lables for service test in zitadel operator

* fix: don't enable database features for user operations (#1227)

* fix: don't enable database features for user operations

* fix: omit database feature for connection info adapter

* fix: use latest orbos version

* fix(crd): corrected logic to get database connection and other info

* fix(crd): corrected yaml tags and start for zitadel operator

* fix(crd): move some dependencies and use consistent structure

* fix(crd): corrected unit-tests

* fix(crd): corrected main files for debug starts

* chore(pipeline): use correct version for zitadelctl build

* fix(crd): correct calculating of current db state for zitadel operator

* fix(crd): use binary version for deployment of crd mode operators

* fix(crd): add gitops attribute for reconciling

* fix(crd): corrected crd with newest version

* fix(migration): collect cleanup functions and only use them if all jobs are successful

* fix(zitadelctl): import gcp auth to connect to gke cluster

* feat: Add read and writesecret options for crd mode (#1435)

* fix: don't require orbconfig for crd mode

* test: pass

* fix(zitadelctl): import gcp auth to connect to gke cluster

* feat: add read and writesecret option for crd mode

* test: fix

* fix: make all crd secrets writable

* fix: use in-cluster configs for in-cluster operators

* chore: remove unnecessary debug files

Co-authored-by: Stefan Benz <stefan@caos.ch>

* fix: Crdoperatormerge review (#1385)

* fix: don't require orbconfig for crd mode

* test: pass

* fix(zitadelctl): import gcp auth to connect to gke cluster

* fix: ensure caos-system namespace

* fix: apply orbconfig at takeoff

* docs: improve help for creating an orbconfig

* docs: describe orbconfig properties

* docs: add --gitops to help message example

* fix(pipeline): correct upload of artifacts in dev releases

* test: pass

Co-authored-by: Stefan Benz <stefan@caos.ch>

* fix(test): corrected falsely merged tests

* chore: update orbos library

* fix: only handle exactly named and namespaced crd resource

* fix: print errors, check correct crd namespace

* fix: validate bucket secret

* chore: compile

* fix(operator): corrected secret handling when unused secrets are not defined

* fix(operator): corrected handling of jobs

* fix: dont print logs when readsecret path is provided

* fix(operator): corrected handling of jobs and sort for mounted volumes

* fix(operator): sort for volumes

* fix(operator): change orboos import to newest release

Co-authored-by: Florian Forster <florian@caos.ch>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>

(cherry picked from commit fa9bd5a8e7)

* fix(operator): Standard timeout handling (#1458)

* fix: always use standard time.Duration

* fix: give backup and restore more time

* fix: give backup and restore jobs more time

(cherry picked from commit 7468b7d1e8)

* fix go mod

Co-authored-by: Stefan Benz <46600784+stebenz@users.noreply.github.com>
Co-authored-by: Elio Bischof <eliobischof@gmail.com>
2021-03-24 10:31:19 +01:00

282 lines
12 KiB
YAML

name: Zitadel Release
on:
push:
branches:
- '**'
tags-ignore:
- '**'
env:
GITHUB_TOKEN: ${{ secrets.CR_PAT }}
REGISTRY: ghcr.io
NODE_VERSION: '12'
GO_VERSION: '1.15'
OPERATOR_IMAGE_NAME: ${{ github.repository }}-operator
CRDB_IMAGE_NAME: ${{ github.repository }}-crbackup
ARTIFACTS_FOLDER: './artifacts'
jobs:
refs:
runs-on: ubuntu-18.04
outputs:
sha_short: ${{ steps.refs.outputs.sha_short }}
short_ref: ${{ steps.refs.outputs.short_ref }}
version: ${{ steps.refs.outputs.version }}
steps:
- name: Source checkout
uses: actions/checkout@v2
- name: Semantic Release
id: semantic
uses: cycjimmy/semantic-release-action@v2
with:
dry_run: true
semantic_version: 17.0.4
- name: Set outputs
id: refs
run: |
export BRANCH=${GITHUB_REF#refs/*/}
echo "branch: ${BRANCH}"
export BRANCHTRANSLATED=$(echo ${BRANCH} | tr '/' '_')
echo "short_ref: ${BRANCHTRANSLATED}"
export VERSION=""
if ${{ steps.semantic.outputs.new_release_published == 'true' }}; then
export VERSION=${{ steps.semantic.outputs.new_release_version }}
else
export VERSION=${BRANCHTRANSLATED}
fi
echo "New semantic release: ${{ steps.semantic.outputs.new_release_published }}"
echo "version: ${VERSION}"
echo "::set-output name=short_ref::${BRANCHTRANSLATED}"
echo "::set-output name=sha_short::SHA-$(git rev-parse --short=12 HEAD)"
echo "::set-output name=version::${VERSION}"
zitadel-image:
needs: refs
runs-on: ubuntu-18.04
steps:
- name: Source checkout
uses: actions/checkout@v2
- name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ github.actor }}
password: ${{ secrets.CR_PAT }}
registry: ${{ env.REGISTRY }}
- uses: docker/build-push-action@v2
with:
context: .
file: ./build/dockerfile
platforms: linux/amd64
tags: ${{ env.REGISTRY }}/${{ github.repository }}:${{ needs.refs.outputs.sha_short }},${{ env.REGISTRY }}/${{ github.repository }}:${{ needs.refs.outputs.short_ref }}
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache
operator-image:
needs: refs
runs-on: ubuntu-18.04
name: Build ${{ matrix.goos }}-${{ matrix.goarch }}
strategy:
matrix:
goos: [ 'linux', 'darwin', 'windows' ]
goarch: [ 'amd64' ]
steps:
- name: Source checkout
uses: actions/checkout@v2
- name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-op-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-op-
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ github.actor }}
password: ${{ secrets.CR_PAT }}
registry: ${{ env.REGISTRY }}
- uses: docker/build-push-action@v2
name: onlybuild
with:
context: .
file: ./build/operator/Dockerfile
platforms: linux/amd64
tags: ${{ env.REGISTRY }}/${{ env.OPERATOR_IMAGE_NAME }}:${{ needs.refs.outputs.sha_short }},${{ env.REGISTRY }}/${{ env.OPERATOR_IMAGE_NAME }}:${{ needs.refs.outputs.short_ref }}
push: false
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache
outputs: type=local,dest=/tmp/operator
build-args: |
OS=${{ matrix.goos }}
ARCH=${{ matrix.goarch }}
VERSION=${{ needs.refs.outputs.version }}
- shell: bash
run: |
mv /tmp/operator/zitadelctl /tmp/operator/zitadelctl-${{ matrix.goos }}-${{ matrix.goarch }}
- uses: actions/upload-artifact@v2
with:
name: zitadelctl-${{ matrix.goos }}-${{ matrix.goarch }}
path: /tmp/operator/zitadelctl-${{ matrix.goos }}-${{ matrix.goarch }}
- uses: docker/build-push-action@v2
if: ${{ matrix.goos == 'linux' && matrix.goarch == 'amd64' }}
name: buildandpush
with:
context: .
file: ./build/operator/Dockerfile
platforms: linux/amd64
tags: ${{ env.REGISTRY }}/${{ env.OPERATOR_IMAGE_NAME }}:${{ needs.refs.outputs.sha_short }},${{ env.REGISTRY }}/${{ env.OPERATOR_IMAGE_NAME }}:${{ needs.refs.outputs.short_ref }}
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache
build-args: |
OS=${{ matrix.goos }}
ARCH=${{ matrix.goarch }}
VERSION=${{ needs.refs.outputs.version }}
crdb-image:
needs: refs
runs-on: ubuntu-18.04
steps:
- name: Source checkout
uses: actions/checkout@v2
- name: Cache Docker layers
uses: actions/cache@v2
with:
path: /tmp/.buildx-cache
key: ${{ runner.os }}-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-buildx-
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Login to DockerHub
uses: docker/login-action@v1
with:
username: ${{ github.actor }}
password: ${{ secrets.CR_PAT }}
registry: ${{ env.REGISTRY }}
- uses: docker/build-push-action@v2
name: buildandpush
with:
context: .
file: ./build/cr-backup/Dockerfile
platforms: linux/amd64
tags: ${{ env.REGISTRY }}/${{ env.CRDB_IMAGE_NAME }}:${{ needs.refs.outputs.sha_short }},${{ env.REGISTRY }}/${{ env.CRDB_IMAGE_NAME }}:${{ needs.refs.outputs.short_ref }}
push: true
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache
release:
runs-on: ubuntu-18.04
needs: [ refs, zitadel-image, operator-image, crdb-image ]
env:
DOCKER_USERNAME: ${{ github.actor }}
DOCKER_PASSWORD: ${{ secrets.CR_PAT }}
steps:
- name: Source checkout
uses: actions/checkout@v2
- name: Docker Login
run: docker login $REGISTRY -u $GITHUB_ACTOR -p $GITHUB_TOKEN
- name: Docker Pull short-sha
run: docker pull $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.sha_short }}
- name: Docker Pull short-sha
run: docker pull $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.sha_short }}
- name: Docker Pull short-sha
run: docker pull $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.sha_short }}
- name: Download all zitadelctl artifact
uses: actions/download-artifact@v2
with:
path: ${{ env.ARTIFACTS_FOLDER }}/
- name: Workaround for exe
run: |
mv ${ARTIFACTS_FOLDER}/zitadelctl-windows-amd64/zitadelctl-windows-amd64 ${ARTIFACTS_FOLDER}/zitadelctl-windows-amd64/zitadelctl-windows-amd64.exe
find ${ARTIFACTS_FOLDER}
- name: Semantic Release
id: semantic
uses: cycjimmy/semantic-release-action@v2
with:
dry_run: false
semantic_version: 17.0.4
- name: Do something when a new release published
if: steps.semantic.outputs.new_release_published == 'true'
run: |
echo ${{ steps.semantic.outputs.new_release_version }}
echo ${{ steps.semantic.outputs.new_release_major_version }}
echo ${{ steps.semantic.outputs.new_release_minor_version }}
echo ${{ steps.semantic.outputs.new_release_patch_version }}
- name: Docker Tag Version
run: |
docker tag $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.sha_short }} $REGISTRY/$GITHUB_REPOSITORY:${{ steps.semantic.outputs.new_release_version }}
docker tag $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$OPERATOR_IMAGE_NAME:${{ steps.semantic.outputs.new_release_version }}
docker tag $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$CRDB_IMAGE_NAME:${{ steps.semantic.outputs.new_release_version }}
if: steps.semantic.outputs.new_release_published == 'true'
- name: Docker Tag Latest
run: |
docker tag $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.sha_short }} $REGISTRY/$GITHUB_REPOSITORY:latest
docker tag $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$OPERATOR_IMAGE_NAME:latest
docker tag $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$CRDB_IMAGE_NAME:latest
if: steps.semantic.outputs.new_release_published == 'true'
- name: Docker Push Version
run: |
docker push $REGISTRY/$GITHUB_REPOSITORY:${{ steps.semantic.outputs.new_release_version }}
docker push $REGISTRY/$OPERATOR_IMAGE_NAME:${{ steps.semantic.outputs.new_release_version }}
docker push $REGISTRY/$CRDB_IMAGE_NAME:${{ steps.semantic.outputs.new_release_version }}
if: steps.semantic.outputs.new_release_published == 'true'
- name: Docker Push Latest
run: |
docker push $REGISTRY/$GITHUB_REPOSITORY:latest
docker push $REGISTRY/$OPERATOR_IMAGE_NAME:latest
docker push $REGISTRY/$CRDB_IMAGE_NAME:latest
if: steps.semantic.outputs.new_release_published == 'true'
- name: Docker Tag Version
run: |
docker tag $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.sha_short }} $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.short_ref }}
docker tag $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.short_ref }}
docker tag $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.sha_short }} $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.short_ref }}
if: steps.semantic.outputs.new_release_published != 'true' && needs.refs.outputs.short_ref != 'master' && needs.refs.outputs.short_ref != ''
- name: Docker Push Version
run: |
docker push $REGISTRY/$GITHUB_REPOSITORY:${{ needs.refs.outputs.short_ref }}
docker push $REGISTRY/$OPERATOR_IMAGE_NAME:${{ needs.refs.outputs.short_ref }}
docker push $REGISTRY/$CRDB_IMAGE_NAME:${{ needs.refs.outputs.short_ref }}
if: steps.semantic.outputs.new_release_published != 'true' && needs.refs.outputs.short_ref != 'master' && needs.refs.outputs.short_ref != ''
- name: Dev-Release
id: create_release
uses: ncipollo/release-action@v1.8.1
if: steps.semantic.outputs.new_release_published != 'true' && needs.refs.outputs.short_ref != 'master' && needs.refs.outputs.short_ref != ''
with:
artifacts: "${{ env.ARTIFACTS_FOLDER }}/zitadelctl-darwin-amd64/zitadelctl-darwin-amd64,${{ env.ARTIFACTS_FOLDER }}/zitadelctl-linux-amd64/zitadelctl-linux-amd64,${{ env.ARTIFACTS_FOLDER }}/zitadelctl-windows-amd64/zitadelctl-windows-amd64.exe"
body: |
This is a release from a development branch.
Do not use these artifacts in production.
tag: ${{ needs.refs.outputs.short_ref }}-dev
commit: ${{ needs.refs.outputs.short_ref }}
name: Branch ${{ needs.refs.outputs.short_ref }}
token: ${{ env.GITHUB_TOKEN }}
replacesArtifacts: true
prerelease: true
draft: false
allowUpdates: true