verbose and test

Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>

.github/ISSUE_TEMPLATE/bug_report.md

@@ -6,19 +6,24 @@ labels: ["bug"]
 assignees: ""
 ---
 
-<!-- Headscale is a multinational community across the globe. Our common language is English. Please consider raising the bug report in this language. -->
+<!--
+Before posting a bug report, discuss the behaviour you are expecting with the Discord community
+to make sure that it is truly a bug.
+The issue tracker is not the place to ask for support or how to set up Headscale.
 
-**Bug description**
+Bug reports without the sufficient information will be closed.
+
+Headscale is a multinational community across the globe. Our language is English.
+All bug reports needs to be in English.
+-->
+
+## Bug description
 
 <!-- A clear and concise description of what the bug is. Describe the expected bahavior
   and how it is currently different. If you are unsure if it is a bug, consider discussing
   it on our Discord server first. -->
 
-**To Reproduce**
-
-<!-- Steps to reproduce the behavior. -->
-
-**Context info**
+## Environment
 
 <!-- Please add relevant information about your system. For example:
 - Version of headscale used
@@ -28,3 +33,20 @@ assignees: ""
 - The relevant config parameters you used
 - Log output
 -->
+
+- OS:
+- Headscale version:
+- Tailscale version:
+
+<!--
+We do not support running Headscale in a container nor behind a (reverse) proxy.
+If either of these are true for your environment, ask the community in Discord
+instead of filing a bug report.
+-->
+
+- [ ] Headscale is behind a (reverse) proxy
+- [ ] Headscale runs in a container
+
+## To Reproduce
+
+<!-- Steps to reproduce the behavior. -->

.github/ISSUE_TEMPLATE/feature_request.md

@@ -6,12 +6,21 @@ labels: ["enhancement"]
 assignees: ""
 ---
 
-<!-- Headscale is a multinational community across the globe. Our common language is English. Please consider raising the feature request in this language. -->
+<!--
+We typically have a clear roadmap for what we want to improve and reserve the right
+to close feature requests that does not fit in the roadmap, or fit with the scope
+of the project, or we actually want to implement ourselves.
 
-**Feature request**
+Headscale is a multinational community across the globe. Our language is English.
+All bug reports needs to be in English.
+-->
 
-<!-- A clear and precise description of what new or changed feature you want. -->
+## Why
 
-<!-- Please include the reason, why you would need the feature. E.g. what problem
+<!-- Include the reason, why you would need the feature. E.g. what problem
   does it solve? Or which workflow is currently frustrating and will be improved by
   this? -->
+
+## Description
+
+<!-- A clear and precise description of what new or changed feature you want. -->

.github/ISSUE_TEMPLATE/other_issue.md

@@ -1,30 +0,0 @@
----
-name: "Other issue"
-about: "Report a different issue"
-title: ""
-labels: ["bug"]
-assignees: ""
----
-
-<!-- Headscale is a multinational community across the globe. Our common language is English. Please consider raising the issue in this language. -->
-
-<!-- If you have a question, please consider using our Discord for asking questions -->
-
-**Issue description**
-
-<!-- Please add your issue description. -->
-
-**To Reproduce**
-
-<!-- Steps to reproduce the behavior. -->
-
-**Context info**
-
-<!-- Please add relevant information about your system. For example:
-- Version of headscale used
-- Version of tailscale client
-- OS (e.g. Linux, Mac, Cygwin, WSL, etc.) and version
-- Kernel version
-- The relevant config parameters you used
-- Log output
--->

.github/pull_request_template.md

@@ -1,3 +1,15 @@
+<!--
+Headscale is "Open Source, acknowledged contribution", this means that any
+contribution will have to be discussed with the Maintainers before being submitted.
+
+This model has been chosen to reduce the risk of burnout by limiting the
+maintenance overhead of reviewing and validating third-party code.
+
+Headscale is open to code contributions for bug fixes without discussion.
+
+If you find mistakes in the documentation, please submit a fix to the documentation.
+-->
+
 <!-- Please tick if the following things apply. You… -->
 
 - [ ] read the [CONTRIBUTING guidelines](README.md#contributing)

.github/renovate.json

@@ -6,31 +6,27 @@
   "onboarding": false,
   "extends": ["config:base", ":rebaseStalePrs"],
   "ignorePresets": [":prHourlyLimit2"],
-  "enabledManagers": ["dockerfile", "gomod", "github-actions","regex" ],
+  "enabledManagers": ["dockerfile", "gomod", "github-actions", "regex"],
   "includeForks": true,
   "repositories": ["juanfont/headscale"],
   "platform": "github",
   "packageRules": [
     {
-        "matchDatasources": ["go"],
-        "groupName": "Go modules",
-        "groupSlug": "gomod",
-        "separateMajorMinor": false
+      "matchDatasources": ["go"],
+      "groupName": "Go modules",
+      "groupSlug": "gomod",
+      "separateMajorMinor": false
     },
     {
-        "matchDatasources": ["docker"],
-        "groupName": "Dockerfiles",
-        "groupSlug": "dockerfiles"
-    } 
+      "matchDatasources": ["docker"],
+      "groupName": "Dockerfiles",
+      "groupSlug": "dockerfiles"
+    }
   ],
   "regexManagers": [
     {
-      "fileMatch": [
-          ".github/workflows/.*.yml$"
-      ],
-      "matchStrings": [
-        "\\s*go-version:\\s*\"?(?<currentValue>.*?)\"?\\n"
-      ],
+      "fileMatch": [".github/workflows/.*.yml$"],
+      "matchStrings": ["\\s*go-version:\\s*\"?(?<currentValue>.*?)\"?\\n"],
       "datasourceTemplate": "golang-version",
       "depNameTemplate": "actions/go-version"
     }

.github/workflows/build.yml

@@ -33,7 +33,9 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v16
+      - uses: DeterminateSystems/nix-installer-action@main
+        if: steps.changed-files.outputs.any_changed == 'true'
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.any_changed == 'true'
 
       - name: Run build

.github/workflows/docs.yml

@@ -0,0 +1,45 @@
+name: Build documentation
+on:
+  push:
+    branches:
+      - main
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Install python
+        uses: actions/setup-python@v4
+        with:
+          python-version: 3.x
+      - name: Setup cache
+        uses: actions/cache@v2
+        with:
+          key: ${{ github.ref }}
+          path: .cache
+      - name: Setup dependencies
+        run: pip install -r docs/requirements.txt
+      - name: Build docs
+        run: mkdocs build --strict
+      - name: Upload artifact
+        uses: actions/upload-pages-artifact@v1
+        with:
+          path: ./site
+  deploy:
+    environment:
+      name: github-pages
+      url: ${{ steps.deployment.outputs.page_url }}
+    runs-on: ubuntu-latest
+    needs: build
+    steps:
+      - name: Deploy to GitHub Pages
+        id: deployment
+        uses: actions/deploy-pages@v1

.github/workflows/release-docker.yml

@@ -0,0 +1,138 @@
+---
+name: Release Docker
+
+on:
+  push:
+    tags:
+      - "*" # triggers only if push new tag version
+  workflow_dispatch:
+
+jobs:
+  docker-release:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+      - name: Set up QEMU for multiple platforms
+        uses: docker/setup-qemu-action@master
+        with:
+          platforms: arm64,amd64
+      - name: Cache Docker layers
+        uses: actions/cache@v2
+        with:
+          path: /tmp/.buildx-cache
+          key: ${{ runner.os }}-buildx-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-buildx-
+      - name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v3
+        with:
+          # list of Docker images to use as base name for tags
+          images: |
+            ${{ secrets.DOCKERHUB_USERNAME }}/headscale
+            ghcr.io/${{ github.repository_owner }}/headscale
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=semver,pattern={{major}}
+            type=sha
+            type=raw,value=develop
+      - name: Login to DockerHub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - name: Login to GHCR
+        uses: docker/login-action@v1
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Build and push
+        id: docker_build
+        uses: docker/build-push-action@v2
+        with:
+          push: true
+          context: .
+          tags: ${{ steps.meta.outputs.tags }}
+          labels: ${{ steps.meta.outputs.labels }}
+          platforms: linux/amd64,linux/arm64
+          cache-from: type=local,src=/tmp/.buildx-cache
+          cache-to: type=local,dest=/tmp/.buildx-cache-new
+          build-args: |
+            VERSION=${{ steps.meta.outputs.version }}
+      - name: Prepare cache for next build
+        run: |
+          rm -rf /tmp/.buildx-cache
+          mv /tmp/.buildx-cache-new /tmp/.buildx-cache
+
+  docker-debug-release:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+      - name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v1
+      - name: Set up QEMU for multiple platforms
+        uses: docker/setup-qemu-action@master
+        with:
+          platforms: arm64,amd64
+      - name: Cache Docker layers
+        uses: actions/cache@v2
+        with:
+          path: /tmp/.buildx-cache-debug
+          key: ${{ runner.os }}-buildx-debug-${{ github.sha }}
+          restore-keys: |
+            ${{ runner.os }}-buildx-debug-
+      - name: Docker meta
+        id: meta-debug
+        uses: docker/metadata-action@v3
+        with:
+          # list of Docker images to use as base name for tags
+          images: |
+            ${{ secrets.DOCKERHUB_USERNAME }}/headscale
+            ghcr.io/${{ github.repository_owner }}/headscale
+          flavor: |
+            suffix=-debug,onlatest=true
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+            type=semver,pattern={{major}}
+            type=sha
+            type=raw,value=develop
+      - name: Login to DockerHub
+        uses: docker/login-action@v1
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      - name: Login to GHCR
+        uses: docker/login-action@v1
+        with:
+          registry: ghcr.io
+          username: ${{ github.repository_owner }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Build and push
+        id: docker_build
+        uses: docker/build-push-action@v2
+        with:
+          push: true
+          context: .
+          file: Dockerfile.debug
+          tags: ${{ steps.meta-debug.outputs.tags }}
+          labels: ${{ steps.meta-debug.outputs.labels }}
+          platforms: linux/amd64,linux/arm64
+          cache-from: type=local,src=/tmp/.buildx-cache-debug
+          cache-to: type=local,dest=/tmp/.buildx-cache-debug-new
+          build-args: |
+            VERSION=${{ steps.meta-debug.outputs.version }}
+      - name: Prepare cache for next build
+        run: |
+          rm -rf /tmp/.buildx-cache-debug
+          mv /tmp/.buildx-cache-debug-new /tmp/.buildx-cache-debug

.github/workflows/release.yml

@@ -16,138 +16,10 @@ jobs:
         with:
           fetch-depth: 0
 
-      - uses: cachix/install-nix-action@v16
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
 
       - name: Run goreleaser
-        run: nix develop --command -- goreleaser release --rm-dist
+        run: nix develop --command -- goreleaser release --clean --verbose
         env:
           GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
-
-  docker-release:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
-      - name: Set up QEMU for multiple platforms
-        uses: docker/setup-qemu-action@master
-        with:
-          platforms: arm64,amd64
-      - name: Cache Docker layers
-        uses: actions/cache@v2
-        with:
-          path: /tmp/.buildx-cache
-          key: ${{ runner.os }}-buildx-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-buildx-
-      - name: Docker meta
-        id: meta
-        uses: docker/metadata-action@v3
-        with:
-          # list of Docker images to use as base name for tags
-          images: |
-            ${{ secrets.DOCKERHUB_USERNAME }}/headscale
-            ghcr.io/${{ github.repository_owner }}/headscale
-          tags: |
-            type=semver,pattern={{version}}
-            type=semver,pattern={{major}}.{{minor}}
-            type=semver,pattern={{major}}
-            type=sha
-            type=raw,value=develop
-      - name: Login to DockerHub
-        uses: docker/login-action@v1
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Login to GHCR
-        uses: docker/login-action@v1
-        with:
-          registry: ghcr.io
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-      - name: Build and push
-        id: docker_build
-        uses: docker/build-push-action@v2
-        with:
-          push: true
-          context: .
-          tags: ${{ steps.meta.outputs.tags }}
-          labels: ${{ steps.meta.outputs.labels }}
-          platforms: linux/amd64,linux/arm64
-          cache-from: type=local,src=/tmp/.buildx-cache
-          cache-to: type=local,dest=/tmp/.buildx-cache-new
-          build-args: |
-            VERSION=${{ steps.meta.outputs.version }}
-      - name: Prepare cache for next build
-        run: |
-          rm -rf /tmp/.buildx-cache
-          mv /tmp/.buildx-cache-new /tmp/.buildx-cache
-
-  docker-debug-release:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout
-        uses: actions/checkout@v3
-        with:
-          fetch-depth: 0
-      - name: Set up Docker Buildx
-        uses: docker/setup-buildx-action@v1
-      - name: Set up QEMU for multiple platforms
-        uses: docker/setup-qemu-action@master
-        with:
-          platforms: arm64,amd64
-      - name: Cache Docker layers
-        uses: actions/cache@v2
-        with:
-          path: /tmp/.buildx-cache-debug
-          key: ${{ runner.os }}-buildx-debug-${{ github.sha }}
-          restore-keys: |
-            ${{ runner.os }}-buildx-debug-
-      - name: Docker meta
-        id: meta-debug
-        uses: docker/metadata-action@v3
-        with:
-          # list of Docker images to use as base name for tags
-          images: |
-            ${{ secrets.DOCKERHUB_USERNAME }}/headscale
-            ghcr.io/${{ github.repository_owner }}/headscale
-          flavor: |
-            suffix=-debug,onlatest=true
-          tags: |
-            type=semver,pattern={{version}}
-            type=semver,pattern={{major}}.{{minor}}
-            type=semver,pattern={{major}}
-            type=sha
-            type=raw,value=develop
-      - name: Login to DockerHub
-        uses: docker/login-action@v1
-        with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
-          password: ${{ secrets.DOCKERHUB_TOKEN }}
-      - name: Login to GHCR
-        uses: docker/login-action@v1
-        with:
-          registry: ghcr.io
-          username: ${{ github.repository_owner }}
-          password: ${{ secrets.GITHUB_TOKEN }}
-      - name: Build and push
-        id: docker_build
-        uses: docker/build-push-action@v2
-        with:
-          push: true
-          context: .
-          file: Dockerfile.debug
-          tags: ${{ steps.meta-debug.outputs.tags }}
-          labels: ${{ steps.meta-debug.outputs.labels }}
-          platforms: linux/amd64,linux/arm64
-          cache-from: type=local,src=/tmp/.buildx-cache-debug
-          cache-to: type=local,dest=/tmp/.buildx-cache-debug-new
-          build-args: |
-            VERSION=${{ steps.meta-debug.outputs.version }}
-      - name: Prepare cache for next build
-        run: |
-          rm -rf /tmp/.buildx-cache-debug
-          mv /tmp/.buildx-cache-debug-new /tmp/.buildx-cache-debug

.github/workflows/stale.yml

@@ -0,0 +1,22 @@
+name: Close inactive issues
+on:
+  schedule:
+    - cron: "30 1 * * *"
+
+jobs:
+  close-issues:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/stale@v5
+        with:
+          days-before-issue-stale: 90
+          days-before-issue-close: 7
+          stale-issue-label: "stale"
+          stale-issue-message: "This issue is stale because it has been open for 90 days with no activity."
+          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
+          days-before-pr-stale: -1
+          days-before-pr-close: -1
+          repo-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/test-integration-cli.yml

@@ -1,35 +0,0 @@
-name: Integration Test CLI
-
-on: [pull_request]
-
-jobs:
-  integration-test-cli:
-    runs-on: ubuntu-latest
-
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          fetch-depth: 2
-
-      - name: Set Swap Space
-        uses: pierotofy/set-swap-space@master
-        with:
-          swap-size-gb: 10
-
-      - name: Get changed files
-        id: changed-files
-        uses: tj-actions/changed-files@v34
-        with:
-          files: |
-            *.nix
-            go.*
-            **/*.go
-            integration_test/
-            config-example.yaml
-
-      - uses: cachix/install-nix-action@v16
-        if: steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run CLI integration tests
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: nix develop --command -- make test_integration_cli

.github/workflows/test-integration-derp.yml

@@ -1,35 +0,0 @@
-name: Integration Test DERP
-
-on: [pull_request]
-
-jobs:
-  integration-test-derp:
-    runs-on: ubuntu-latest
-
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          fetch-depth: 2
-
-      - name: Set Swap Space
-        uses: pierotofy/set-swap-space@master
-        with:
-          swap-size-gb: 10
-
-      - name: Get changed files
-        id: changed-files
-        uses: tj-actions/changed-files@v34
-        with:
-          files: |
-            *.nix
-            go.*
-            **/*.go
-            integration_test/
-            config-example.yaml
-
-      - uses: cachix/install-nix-action@v16
-        if: steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run Embedded DERP server integration tests
-        if: steps.changed-files.outputs.any_changed == 'true'
-        run: nix develop --command -- make test_integration_derp

.github/workflows/test-integration-v2-TestACLAllowStarDst.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestACLAllowStarDst:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestACLAllowStarDst
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLAllowUser80Dst.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestACLAllowUser80Dst:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestACLAllowUser80Dst
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLAllowUserDst.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestACLAllowUserDst:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestACLAllowUserDst
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLDenyAllPort80.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestACLDenyAllPort80:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestACLDenyAllPort80
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLDevice1CanAccessDevice2.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestACLDevice1CanAccessDevice2
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestACLDevice1CanAccessDevice2:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestACLDevice1CanAccessDevice2
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestACLDevice1CanAccessDevice2$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLHostsInNetMapTable.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestACLHostsInNetMapTable:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestACLHostsInNetMapTable
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLNamedHostsCanReach.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestACLNamedHostsCanReach
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestACLNamedHostsCanReach:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestACLNamedHostsCanReach
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestACLNamedHostsCanReach$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestACLNamedHostsCanReachBySubnet.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestACLNamedHostsCanReachBySubnet
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestACLNamedHostsCanReachBySubnet:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestACLNamedHostsCanReachBySubnet
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestACLNamedHostsCanReachBySubnet$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestApiKeyCommand.yaml

@@ -1,7 +1,7 @@
 # DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
 # To regenerate, run "go generate" in cmd/gh-action-integration-generator/
 
-name: Integration Test v2 - TestSSHOneUserAllToAll
+name: Integration Test v2 - TestApiKeyCommand
 
 on: [pull_request]
 
@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestApiKeyCommand:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestApiKeyCommand
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,15 +48,20 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
-                  -run "^TestSSHOneUserAllToAll$"
+                  -run "^TestApiKeyCommand$"
 
       - uses: actions/upload-artifact@v3
         if: always() && steps.changed-files.outputs.any_changed == 'true'
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestAuthKeyLogoutAndRelogin.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestAuthKeyLogoutAndRelogin:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestAuthKeyLogoutAndRelogin
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestAuthWebFlowAuthenticationPingAll.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestAuthWebFlowAuthenticationPingAll:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestAuthWebFlowAuthenticationPingAll
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestAuthWebFlowLogoutAndRelogin.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestAuthWebFlowLogoutAndRelogin:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestAuthWebFlowLogoutAndRelogin
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestCreateTailscale.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestCreateTailscale:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestCreateTailscale
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestDERPServerScenario.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestDERPServerScenario
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestDERPServerScenario:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestDERPServerScenario
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestDERPServerScenario$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestEnableDisableAutoApprovedRoute.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestEnableDisableAutoApprovedRoute
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestEnableDisableAutoApprovedRoute:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestEnableDisableAutoApprovedRoute
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestEnableDisableAutoApprovedRoute$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestEnablingRoutes.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestEnablingRoutes:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestEnablingRoutes
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestEphemeral.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestEphemeral:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestEphemeral
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestExpireNode.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestExpireNode:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestExpireNode
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestHASubnetRouterFailover.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestHASubnetRouterFailover
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestHASubnetRouterFailover:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestHASubnetRouterFailover
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestHASubnetRouterFailover$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestHeadscale.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestHeadscale:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestHeadscale
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeAdvertiseTagNoACLCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeAdvertiseTagNoACLCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeAdvertiseTagNoACLCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeAdvertiseTagNoACLCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeAdvertiseTagNoACLCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeAdvertiseTagWithACLCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeAdvertiseTagWithACLCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeAdvertiseTagWithACLCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeAdvertiseTagWithACLCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeAdvertiseTagWithACLCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeCommand.yaml

@@ -1,7 +1,7 @@
 # DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
 # To regenerate, run "go generate" in cmd/gh-action-integration-generator/
 
-name: Integration Test v2 - TestSSUserOnlyIsolation
+name: Integration Test v2 - TestNodeCommand
 
 on: [pull_request]
 
@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestNodeCommand:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestNodeCommand
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,15 +48,20 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
-                  -run "^TestSSUserOnlyIsolation$"
+                  -run "^TestNodeCommand$"
 
       - uses: actions/upload-artifact@v3
         if: always() && steps.changed-files.outputs.any_changed == 'true'
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeExpireCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeExpireCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeExpireCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeExpireCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeExpireCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeMoveCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeMoveCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeMoveCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeMoveCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeMoveCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeOnlineLastSeenStatus.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeOnlineLastSeenStatus
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeOnlineLastSeenStatus:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeOnlineLastSeenStatus
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeOnlineLastSeenStatus$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeRenameCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeRenameCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeRenameCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeRenameCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeRenameCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestNodeTagCommand.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestNodeTagCommand
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestNodeTagCommand:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestNodeTagCommand
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestNodeTagCommand$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestOIDCAuthenticationPingAll.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestOIDCAuthenticationPingAll:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestOIDCAuthenticationPingAll
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestOIDCExpireNodesBasedOnTokenExpiry.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestOIDCExpireNodesBasedOnTokenExpiry:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestOIDCExpireNodesBasedOnTokenExpiry
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestPingAllByHostname.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestPingAllByHostname:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestPingAllByHostname
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestPingAllByIP.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestPingAllByIP:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestPingAllByIP
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestPreAuthKeyCommand.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestPreAuthKeyCommand:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestPreAuthKeyCommand
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestPreAuthKeyCommandReusableEphemeral.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestPreAuthKeyCommandReusableEphemeral:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestPreAuthKeyCommandReusableEphemeral
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestPreAuthKeyCommandWithoutExpiry.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestPreAuthKeyCommandWithoutExpiry:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestPreAuthKeyCommandWithoutExpiry
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestResolveMagicDNS.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestResolveMagicDNS:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestResolveMagicDNS
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSSHIsBlockedInACL.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestSSHIsBlockedInACL:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestSSHIsBlockedInACL
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSSHMultipleUsersAllToAll.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestSSHMultipleUsersAllToAll:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestSSHMultipleUsersAllToAll
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSSHNoSSHConfigured.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestSSHNoSSHConfigured:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestSSHNoSSHConfigured
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSSHOneUserToAll.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestSSHOneUserToAll
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestSSHOneUserToAll:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestSSHOneUserToAll
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestSSHOneUserToAll$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSSHUserOnlyIsolation.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestSSHUserOnlyIsolation
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestSSHUserOnlyIsolation:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestSSHUserOnlyIsolation
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestSSHUserOnlyIsolation$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestSubnetRouteACL.yaml

@@ -0,0 +1,67 @@
+# DO NOT EDIT, generated with cmd/gh-action-integration-generator/main.go
+# To regenerate, run "go generate" in cmd/gh-action-integration-generator/
+
+name: Integration Test v2 - TestSubnetRouteACL
+
+on: [pull_request]
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  TestSubnetRouteACL:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v3
+        with:
+          fetch-depth: 2
+
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
+      - name: Get changed files
+        id: changed-files
+        uses: tj-actions/changed-files@v34
+        with:
+          files: |
+            *.nix
+            go.*
+            **/*.go
+            integration_test/
+            config-example.yaml
+
+      - name: Run TestSubnetRouteACL
+        uses: Wandalen/wretry.action@master
+        if: steps.changed-files.outputs.any_changed == 'true'
+        with:
+          attempt_limit: 5
+          command: |
+            nix develop --command -- docker run \
+              --tty --rm \
+              --volume ~/.cache/hs-integration-go:/go \
+              --name headscale-test-suite \
+              --volume $PWD:$PWD -w $PWD/integration \
+              --volume /var/run/docker.sock:/var/run/docker.sock \
+              --volume $PWD/control_logs:/tmp/control \
+              golang:1 \
+                go run gotest.tools/gotestsum@latest -- ./... \
+                  -failfast \
+                  -timeout 120m \
+                  -parallel 1 \
+                  -run "^TestSubnetRouteACL$"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: logs
+          path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestTaildrop.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestTaildrop:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestTaildrop
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestTailscaleNodesJoiningHeadcale.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestTailscaleNodesJoiningHeadcale:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestTailscaleNodesJoiningHeadcale
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test-integration-v2-TestUserCommand.yaml

@@ -10,7 +10,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  TestUserCommand:
     runs-on: ubuntu-latest
 
     steps:
@@ -18,6 +18,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -29,12 +34,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: ${{ env.ACT }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run TestUserCommand
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -43,8 +48,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -55,3 +59,9 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"

.github/workflows/test.yml

@@ -26,7 +26,9 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v16
+      - uses: DeterminateSystems/nix-installer-action@main
+        if: steps.changed-files.outputs.any_changed == 'true'
+      - uses: DeterminateSystems/magic-nix-cache-action@main
         if: steps.changed-files.outputs.any_changed == 'true'
 
       - name: Run tests

.github/workflows/update-flake.yml

@@ -0,0 +1,18 @@
+name: update-flake-lock
+on:
+  workflow_dispatch: # allows manual triggering
+  schedule:
+    - cron: "0 0 * * 0" # runs weekly on Sunday at 00:00
+
+jobs:
+  lockfile:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@v3
+      - name: Install Nix
+        uses: DeterminateSystems/nix-installer-action@main
+      - name: Update flake.lock
+        uses: DeterminateSystems/update-flake-lock@main
+        with:
+          pr-title: "Update flake.lock"

.gitignore

@@ -1,3 +1,7 @@
+ignored/
+tailscale/
+.vscode/
+
 # Binaries for programs and plugins
 *.exe
 *.exe~
@@ -12,8 +16,9 @@
 *.out
 
 # Dependency directories (remove the comment below to include it)
-# vendor/
+vendor/
 
+dist/
 /headscale
 config.json
 config.yaml
@@ -34,3 +39,9 @@ result
 .direnv/
 
 integration_test/etc/config.dump.yaml
+
+# MkDocs
+.cache
+/site
+
+__debug_bin

.golangci.yaml

@@ -10,6 +10,8 @@ issues:
 linters:
   enable-all: true
   disable:
+    - depguard
+
     - exhaustivestruct
     - revive
     - lll
@@ -30,6 +32,7 @@ linters:
     - exhaustruct
     - nolintlint
     - musttag # causes issues with imported libs
+    - depguard
 
     # deprecated
     - structcheck # replaced by unused

.goreleaser.yml

@@ -1,21 +1,28 @@
 ---
 before:
   hooks:
-    - go mod tidy -compat=1.19
+    - go mod tidy -compat=1.20
+    - go mod vendor
 
 release:
   prerelease: auto
 
 builds:
-  - id: darwin-amd64
+  - id: headscale
     main: ./cmd/headscale/headscale.go
     mod_timestamp: "{{ .CommitTimestamp }}"
     env:
       - CGO_ENABLED=0
-    goos:
-      - darwin
-    goarch:
-      - amd64
+    targets:
+      # - darwin_amd64
+      # - darwin_arm64
+      # - freebsd_amd64
+      # - linux_386
+      - linux_amd64
+      # - linux_arm64
+      # - linux_arm_5
+      # - linux_arm_6
+      # - linux_arm_7
     flags:
       - -mod=readonly
     ldflags:
@@ -23,60 +30,87 @@ builds:
     tags:
       - ts2019
 
-  - id: darwin-arm64
-    main: ./cmd/headscale/headscale.go
-    mod_timestamp: "{{ .CommitTimestamp }}"
-    env:
-      - CGO_ENABLED=0
-    goos:
-      - darwin
-    goarch:
-      - arm64
-    flags:
-      - -mod=readonly
-    ldflags:
-      - -s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=v{{.Version}}
-    tags:
-      - ts2019
-
-  - id: linux-amd64
-    mod_timestamp: "{{ .CommitTimestamp }}"
-    env:
-      - CGO_ENABLED=0
-    goos:
-      - linux
-    goarch:
-      - amd64
-    main: ./cmd/headscale/headscale.go
-    ldflags:
-      - -s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=v{{.Version}}
-    tags:
-      - ts2019
-
-  - id: linux-arm64
-    mod_timestamp: "{{ .CommitTimestamp }}"
-    env:
-      - CGO_ENABLED=0
-    goos:
-      - linux
-    goarch:
-      - arm64
-    main: ./cmd/headscale/headscale.go
-    ldflags:
-      - -s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=v{{.Version}}
-    tags:
-      - ts2019
-
 archives:
   - id: golang-cross
-    builds:
-      - darwin-amd64
-      - darwin-arm64
-      - linux-amd64
-      - linux-arm64
-    name_template: "{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}"
+    name_template: '{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ with .Arm }}v{{ . }}{{ end }}{{ with .Mips }}_{{ . }}{{ end }}{{ if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}'
     format: binary
 
+source:
+  enabled: true
+  name_template: "{{ .ProjectName }}_{{ .Version }}"
+  format: tar.gz
+  files:
+    - "vendor/"
+
+nfpms:
+  # Configure nFPM for .deb and .rpm releases
+  #
+  # See https://nfpm.goreleaser.com/configuration/
+  # and https://goreleaser.com/customization/nfpm/
+  #
+  # Useful tools for debugging .debs:
+  # List file contents: dpkg -c dist/headscale...deb
+  # Package metadata: dpkg --info dist/headscale....deb
+  #
+  - builds:
+      - headscale
+    package_name: headscale
+    priority: optional
+    vendor: headscale
+    maintainer: Kristoffer Dalby <kristoffer@dalby.cc>
+    homepage: https://github.com/juanfont/headscale
+    license: BSD
+    bindir: /usr/bin
+    formats:
+      - deb
+    contents:
+      - src: ./config-example.yaml
+        dst: /etc/headscale/config.yaml
+        type: config|noreplace
+        file_info:
+          mode: 0644
+      - src: ./docs/packaging/headscale.systemd.service
+        dst: /usr/lib/systemd/system/headscale.service
+      - dst: /var/lib/headscale
+        type: dir
+      - dst: /var/run/headscale
+        type: dir
+    scripts:
+      postinstall: ./docs/packaging/postinstall.sh
+      postremove: ./docs/packaging/postremove.sh
+
+kos:
+  - id: ghcr
+    build: headscale
+    base_image: gcr.io/distroless/base-debian11
+    repository: ghcr.io/juanfont/headscale
+    platforms:
+      - linux/amd64
+      # - linux/386
+      # - linux/arm64
+      # - linux/arm/v7
+      # - linux/arm/v6
+      # - linux/arm/v5
+    tags:
+      - latest
+      - '{{.Tag}}'
+      - '{{ .Major }}.{{ .Minor }}'
+      - '{{ .Major }}'
+  # - id: dockerhub
+  #   build: headscale
+  #   base_image: gcr.io/distroless/base-debian11
+  #   repository: headscale/headscale
+  #   platforms:
+  #     - linux/amd64
+  #     - linux/386
+  #     - linux/arm64
+  #     - linux/arm/v7
+  #     - linux/arm/v6
+  #     - linux/arm/v5
+  #   tags:
+  #     - latest
+  #    - '{{.Tag}}'
+
 checksum:
   name_template: "checksums.txt"
 snapshot:

CHANGELOG.md

@@ -1,10 +1,84 @@
 # CHANGELOG
 
-## 0.22.0 (2023-XX-XX)
+## 0.23.0 (2023-XX-XX)
+
+This release is mainly a code reorganisation and refactoring, significantly improving the maintainability of the codebase. This should allow us to improve further and make it easier for the maintainers to keep on top of the project.
+
+**Please remember to always back up your database between versions**
+
+#### Here is a short summary of the broad topics of changes:
+
+Code has been organised into modules, reducing use of global variables/objects, isolating concerns and “putting the right things in the logical place”.
+
+The new [policy](https://github.com/juanfont/headscale/tree/main/hscontrol/policy) and [mapper](https://github.com/juanfont/headscale/tree/main/hscontrol/mapper) package, containing the ACL/Policy logic and the logic for creating the data served to clients (the network “map”) has been rewritten and improved. This change has allowed us to finish SSH support and add additional tests throughout the code to ensure correctness.
+
+The [“poller”, or streaming logic](https://github.com/juanfont/headscale/blob/main/hscontrol/poll.go) has been rewritten and instead of keeping track of the latest updates, checking at a fixed interval, it now uses go channels, implemented in our new [notifier](https://github.com/juanfont/headscale/tree/main/hscontrol/notifier) package and it allows us to send updates to connected clients immediately. This should both improve performance and potential latency before a client picks up an update.
+
+Headscale now supports sending “delta” updates, thanks to the new mapper and poller logic, allowing us to only inform nodes about new nodes, changed nodes and removed nodes. Previously we sent the entire state of the network every time an update was due.
+
+While we have a pretty good [test harness](https://github.com/search?q=repo%3Ajuanfont%2Fheadscale+path%3A_test.go&type=code) for validating our changes, we have rewritten over [10000 lines of code](https://github.com/juanfont/headscale/compare/b01f1f1867136d9b2d7b1392776eb363b482c525...main) and bugs are expected. We need help testing this release. In addition, while we think the performance should in general be better, there might be regressions in parts of the platform, particularly where we prioritised correctness over speed.
+
+There are also several bugfixes that has been encountered and fixed as part of implementing these changes, particularly
+after improving the test harness as part of adopting [#1460](https://github.com/juanfont/headscale/pull/1460).
+
+### BREAKING
+
+- Code reorganisation, a lot of code has moved, please review the following PRs accordingly [#1473](https://github.com/juanfont/headscale/pull/1473)
+- API: Machine is now Node [#1553](https://github.com/juanfont/headscale/pull/1553)
+- Remove support for older Tailscale clients [#1611](https://github.com/juanfont/headscale/pull/1611)
+  - The latest supported client is 1.36
+- Headscale checks that _at least_ one DERP is defined at start [#1564](https://github.com/juanfont/headscale/pull/1564)
+  - If no DERP is configured, the server will fail to start, this can be because it cannot load the DERPMap from file or url.
+- Embedded DERP server requires a private key [#1611](https://github.com/juanfont/headscale/pull/1611)
+  - Add a filepath entry to [`derp.server.private_key_path`](https://github.com/juanfont/headscale/blob/b35993981297e18393706b2c963d6db882bba6aa/config-example.yaml#L95)
 
 ### Changes
 
+Use versioned migrations [#1644](https://github.com/juanfont/headscale/pull/1644)
+Make the OIDC callback page better [#1484](https://github.com/juanfont/headscale/pull/1484)
+SSH support [#1487](https://github.com/juanfont/headscale/pull/1487)
+State management has been improved [#1492](https://github.com/juanfont/headscale/pull/1492)
+Use error group handling to ensure tests actually pass [#1535](https://github.com/juanfont/headscale/pull/1535) based on [#1460](https://github.com/juanfont/headscale/pull/1460)
+Fix hang on SIGTERM [#1492](https://github.com/juanfont/headscale/pull/1492) taken from [#1480](https://github.com/juanfont/headscale/pull/1480)
+Send logs to stderr by default [#1524](https://github.com/juanfont/headscale/pull/1524)
+Fix [TS-2023-006](https://tailscale.com/security-bulletins/#ts-2023-006) security UPnP issue [#1563](https://github.com/juanfont/headscale/pull/1563)
+Turn off gRPC logging [#1640](https://github.com/juanfont/headscale/pull/1640) fixes [#1259](https://github.com/juanfont/headscale/issues/1259)
+Added the possibility to manually create a DERP-map entry which can be customized, instead of automatically creating it. [#1565](https://github.com/juanfont/headscale/pull/1565)
+
+## 0.22.3 (2023-05-12)
+
+### Changes
+
+- Added missing ca-certificates in Docker image [#1463](https://github.com/juanfont/headscale/pull/1463)
+
+## 0.22.2 (2023-05-10)
+
+### Changes
+
+- Add environment flags to enable pprof (profiling) [#1382](https://github.com/juanfont/headscale/pull/1382)
+  - Profiles are continously generated in our integration tests.
+- Fix systemd service file location in `.deb` packages [#1391](https://github.com/juanfont/headscale/pull/1391)
+- Improvements on Noise implementation [#1379](https://github.com/juanfont/headscale/pull/1379)
+- Replace node filter logic, ensuring nodes with access can see eachother [#1381](https://github.com/juanfont/headscale/pull/1381)
+- Disable (or delete) both exit routes at the same time [#1428](https://github.com/juanfont/headscale/pull/1428)
+- Ditch distroless for Docker image, create default socket dir in `/var/run/headscale` [#1450](https://github.com/juanfont/headscale/pull/1450)
+
+## 0.22.1 (2023-04-20)
+
+### Changes
+
+- Fix issue where systemd could not bind to port 80 [#1365](https://github.com/juanfont/headscale/pull/1365)
+
+## 0.22.0 (2023-04-20)
+
+### Changes
+
+- Add `.deb` packages to release process [#1297](https://github.com/juanfont/headscale/pull/1297)
+- Update and simplify the documentation to use new `.deb` packages [#1349](https://github.com/juanfont/headscale/pull/1349)
+- Add 32-bit Arm platforms to release process [#1297](https://github.com/juanfont/headscale/pull/1297)
 - Fix longstanding bug that would prevent "\*" from working properly in ACLs (issue [#699](https://github.com/juanfont/headscale/issues/699)) [#1279](https://github.com/juanfont/headscale/pull/1279)
+- Fix issue where IPv6 could not be used in, or while using ACLs (part of [#809](https://github.com/juanfont/headscale/issues/809)) [#1339](https://github.com/juanfont/headscale/pull/1339)
+- Target Go 1.20 and Tailscale 1.38 for Headscale [#1323](https://github.com/juanfont/headscale/pull/1323)
 
 ## 0.21.0 (2023-03-20)
 

Dockerfile

@@ -1,5 +1,8 @@
-# Builder image
-FROM docker.io/golang:1.19-bullseye AS build
+# This Dockerfile and the images produced are for testing headscale,
+# and are in no way endorsed by Headscale's maintainers as an
+# official nor supported release or distribution.
+
+FROM docker.io/golang:1.21-bookworm AS build
 ARG VERSION=dev
 ENV GOPATH /go
 WORKDIR /go/src/headscale
@@ -9,15 +12,22 @@ RUN go mod download
 
 COPY . .
 
-RUN CGO_ENABLED=0 GOOS=linux go install -tags ts2019 -ldflags="-s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=$VERSION" -a ./cmd/headscale
+RUN CGO_ENABLED=0 GOOS=linux go install -ldflags="-s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=$VERSION" -a ./cmd/headscale
 RUN strip /go/bin/headscale
 RUN test -e /go/bin/headscale
 
 # Production image
-FROM gcr.io/distroless/base-debian11
+FROM docker.io/debian:bookworm-slim
+
+RUN apt-get update \
+  && apt-get install -y ca-certificates \
+  && rm -rf /var/lib/apt/lists/* \
+  && apt-get clean
 
 COPY --from=build /go/bin/headscale /bin/headscale
 ENV TZ UTC
 
+RUN mkdir -p /var/run/headscale
+
 EXPOSE 8080/tcp
 CMD ["headscale"]

Dockerfile.debug

@@ -1,5 +1,8 @@
-# Builder image
-FROM docker.io/golang:1.19-bullseye AS build
+# This Dockerfile and the images produced are for testing headscale,
+# and are in no way endorsed by Headscale's maintainers as an
+# official nor supported release or distribution.
+
+FROM docker.io/golang:1.21-bookworm AS build
 ARG VERSION=dev
 ENV GOPATH /go
 WORKDIR /go/src/headscale
@@ -9,15 +12,21 @@ RUN go mod download
 
 COPY . .
 
-RUN CGO_ENABLED=0 GOOS=linux go install -tags ts2019 -ldflags="-s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=$VERSION" -a ./cmd/headscale
+RUN CGO_ENABLED=0 GOOS=linux go install -ldflags="-s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=$VERSION" -a ./cmd/headscale
 RUN test -e /go/bin/headscale
 
 # Debug image
-FROM docker.io/golang:1.19.0-bullseye
+FROM docker.io/golang:1.21-bookworm
 
 COPY --from=build /go/bin/headscale /bin/headscale
 ENV TZ UTC
 
+RUN apt-get update \
+  && apt-get install --no-install-recommends --yes less jq \
+  && rm -rf /var/lib/apt/lists/* \
+  && apt-get clean
+RUN mkdir -p /var/run/headscale
+
 # Need to reset the entrypoint or everything will run as a busybox script
 ENTRYPOINT []
 EXPOSE 8080/tcp

Dockerfile.tailscale

@@ -1,19 +0,0 @@
-FROM ubuntu:latest
-
-ARG TAILSCALE_VERSION=*
-ARG TAILSCALE_CHANNEL=stable
-
-RUN apt-get update \
-    && apt-get install -y gnupg curl ssh \
-    && curl -fsSL https://pkgs.tailscale.com/${TAILSCALE_CHANNEL}/ubuntu/focal.gpg | apt-key add - \
-    && curl -fsSL https://pkgs.tailscale.com/${TAILSCALE_CHANNEL}/ubuntu/focal.list | tee /etc/apt/sources.list.d/tailscale.list \
-    && apt-get update \
-    && apt-get install -y ca-certificates tailscale=${TAILSCALE_VERSION} dnsutils \
-    && rm -rf /var/lib/apt/lists/*
-
-RUN adduser --shell=/bin/bash ssh-it-user
-
-ADD integration_test/etc_embedded_derp/tls/server.crt /usr/local/share/ca-certificates/
-RUN chmod 644 /usr/local/share/ca-certificates/server.crt
-
-RUN update-ca-certificates

Dockerfile.tailscale-HEAD

@@ -1,7 +1,11 @@
+# This Dockerfile and the images produced are for testing headscale,
+# and are in no way endorsed by Headscale's maintainers as an
+# official nor supported release or distribution.
+
 FROM golang:latest
 
 RUN apt-get update \
-    && apt-get install -y ca-certificates dnsutils git iptables ssh \
+    && apt-get install -y dnsutils git iptables ssh ca-certificates \
     && rm -rf /var/lib/apt/lists/*
 
 RUN useradd --shell=/bin/bash --create-home ssh-it-user
@@ -10,15 +14,8 @@ RUN git clone https://github.com/tailscale/tailscale.git
 
 WORKDIR /go/tailscale
 
-RUN git checkout main
-
-RUN sh build_dist.sh tailscale.com/cmd/tailscale
-RUN sh build_dist.sh tailscale.com/cmd/tailscaled
-
-RUN cp tailscale /usr/local/bin/
-RUN cp tailscaled /usr/local/bin/
-
-ADD integration_test/etc_embedded_derp/tls/server.crt /usr/local/share/ca-certificates/
-RUN chmod 644 /usr/local/share/ca-certificates/server.crt
-
-RUN update-ca-certificates
+RUN git checkout main \
+    && sh build_dist.sh tailscale.com/cmd/tailscale \
+    && sh build_dist.sh tailscale.com/cmd/tailscaled \
+    && cp tailscale /usr/local/bin/ \
+    && cp tailscaled /usr/local/bin/

Makefile

@@ -10,8 +10,6 @@ ifeq ($(filter $(GOOS), openbsd netbsd soloaris plan9), )
 else
 endif
 
-TAGS = -tags ts2019
-
 # GO_SOURCES = $(wildcard *.go)
 # PROTO_SOURCES = $(wildcard **/*.proto)
 GO_SOURCES = $(call rwildcard,,*.go)
@@ -24,31 +22,9 @@ build:
 dev: lint test build
 
 test:
-	@go test $(TAGS) -short -coverprofile=coverage.out ./...
+	gotestsum -- -short -coverprofile=coverage.out ./...
 
-test_integration: test_integration_cli test_integration_derp test_integration_v2_general
-
-test_integration_cli:
-	docker network rm $$(docker network ls --filter name=headscale --quiet) || true
-	docker network create headscale-test || true
-	docker run -t --rm \
-		--network headscale-test \
-		-v ~/.cache/hs-integration-go:/go \
-		-v $$PWD:$$PWD -w $$PWD \
-		-v /var/run/docker.sock:/var/run/docker.sock golang:1 \
-		go test $(TAGS) -failfast -timeout 30m -count=1 -run IntegrationCLI ./...
-
-test_integration_derp:
-	docker network rm $$(docker network ls --filter name=headscale --quiet) || true
-	docker network create headscale-test || true
-	docker run -t --rm \
-		--network headscale-test \
-		-v ~/.cache/hs-integration-go:/go \
-		-v $$PWD:$$PWD -w $$PWD \
-		-v /var/run/docker.sock:/var/run/docker.sock golang:1 \
-		go test $(TAGS) -failfast -timeout 30m -count=1 -run IntegrationDERP ./...
-
-test_integration_v2_general:
+test_integration:
 	docker run \
 		-t --rm \
 		-v ~/.cache/hs-integration-go:/go \
@@ -56,13 +32,7 @@ test_integration_v2_general:
 		-v $$PWD:$$PWD -w $$PWD/integration \
 		-v /var/run/docker.sock:/var/run/docker.sock \
 		golang:1 \
-		go test $(TAGS) -failfast ./... -timeout 120m -parallel 8
-
-coverprofile_func:
-	go tool cover -func=coverage.out
-
-coverprofile_html:
-	go tool cover -html=coverage.out
+		go run gotest.tools/gotestsum@latest -- -failfast ./... -timeout 120m -parallel 8
 
 lint:
 	golangci-lint run --fix --timeout 10m
@@ -80,11 +50,4 @@ compress: build
 
 generate:
 	rm -rf gen
-	go run github.com/bufbuild/buf/cmd/buf generate proto
-
-install-protobuf-plugins:
-	go install \
-		github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway \
-		github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2 \
-		google.golang.org/protobuf/cmd/protoc-gen-go \
-		google.golang.org/grpc/cmd/protoc-gen-go-grpc
+	buf generate proto

README.md

@@ -32,22 +32,18 @@ organisation.
 
 ## Design goal
 
-`headscale` aims to implement a self-hosted, open source alternative to the Tailscale
-control server. `headscale` has a narrower scope and an instance of `headscale`
-implements a _single_ Tailnet, which is typically what a single organisation, or
-home/personal setup would use.
+Headscale aims to implement a self-hosted, open source alternative to the Tailscale
+control server.
+Headscale's goal is to provide self-hosters and hobbyists with an open-source
+server they can use for their projects and labs.
+It implements a narrow scope, a single Tailnet, suitable for a personal use, or a small
+open-source organisation.
 
-`headscale` uses terms that maps to Tailscale's control server, consult the
-[glossary](./docs/glossary.md) for explainations.
-
-## Support
+## Supporting Headscale
 
 If you like `headscale` and find it useful, there is a sponsorship and donation
 buttons available in the repo.
 
-If you would like to sponsor features, bugs or prioritisation, reach out to
-one of the maintainers.
-
 ## Features
 
 - Full "base" support of Tailscale's features
@@ -79,16 +75,10 @@ one of the maintainers.
 
 ## Running headscale
 
-Please have a look at the documentation under [`docs/`](docs/).
+**Please note that we do not support nor encourage the use of reverse proxies
+and container to run Headscale.**
 
-## Graphical Control Panels
-
-Headscale provides an API for complete management of your Tailnet.
-These are community projects not directly affiliated with the Headscale project.
-
-| Name            | Repository Link                                      | Description                                            | Status |
-| --------------- | ---------------------------------------------------- | ------------------------------------------------------ | ------ |
-| headscale-webui | [Github](https://github.com/ifargle/headscale-webui) | A simple Headscale web UI for small-scale deployments. | Alpha  |
+Please have a look at the [`documentation`](https://headscale.net/).
 
 ## Talks
 
@@ -97,11 +87,23 @@ These are community projects not directly affiliated with the Headscale project.
 
 ## Disclaimer
 
-1. We have nothing to do with Tailscale, or Tailscale Inc.
+1. This project is not associated with Tailscale Inc.
 2. The purpose of Headscale is maintaining a working, self-hosted Tailscale control panel.
 
 ## Contributing
 
+Headscale is "Open Source, acknowledged contribution", this means that any
+contribution will have to be discussed with the Maintainers before being submitted.
+
+This model has been chosen to reduce the risk of burnout by limiting the
+maintenance overhead of reviewing and validating third-party code.
+
+Headscale is open to code contributions for bug fixes without discussion.
+
+If you find mistakes in the documentation, please submit a fix to the documentation.
+
+### Requirements
+
 To contribute to headscale you would need the lastest version of [Go](https://golang.org)
 and [Buf](https://buf.build)(Protobuf generator).
 
@@ -109,8 +111,6 @@ We recommend using [Nix](https://nixos.org/) to setup a development environment.
 be done with `nix develop`, which will install the tools and give you a shell.
 This guarantees that you will have the same dev env as `headscale` maintainers.
 
-PRs and suggestions are welcome.
-
 ### Code style
 
 To ensure we have some consistency with a growing number of contributions,
@@ -240,7 +240,7 @@ make build
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/arch4ngel>
+        <a href=https://github.com/ImpostorKeanu>
             <img src=https://avatars.githubusercontent.com/u/11574161?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Justin Angel/>
             <br />
             <sub style="font-size:14px"><b>Justin Angel</b></sub>
@@ -253,15 +253,6 @@ make build
             <sub style="font-size:14px"><b>Alessandro (Ale) Segala</b></sub>
         </a>
     </td>
-    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/unreality>
-            <img src=https://avatars.githubusercontent.com/u/352522?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=unreality/>
-            <br />
-            <sub style="font-size:14px"><b>unreality</b></sub>
-        </a>
-    </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/ohdearaugustin>
             <img src=https://avatars.githubusercontent.com/u/14001491?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=ohdearaugustin/>
@@ -269,6 +260,8 @@ make build
             <sub style="font-size:14px"><b>ohdearaugustin</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/mpldr>
             <img src=https://avatars.githubusercontent.com/u/33086936?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Moritz Poldrack/>
@@ -276,6 +269,13 @@ make build
             <sub style="font-size:14px"><b>Moritz Poldrack</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/Orhideous>
+            <img src=https://avatars.githubusercontent.com/u/2265184?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Andriy Kushnir/>
+            <br />
+            <sub style="font-size:14px"><b>Andriy Kushnir</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/GrigoriyMikhalkin>
             <img src=https://avatars.githubusercontent.com/u/3637857?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=GrigoriyMikhalkin/>
@@ -350,6 +350,13 @@ make build
     </td>
 </tr>
 <tr>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/majst01>
+            <img src=https://avatars.githubusercontent.com/u/410110?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Stefan Majer/>
+            <br />
+            <sub style="font-size:14px"><b>Stefan Majer</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/fdelucchijr>
             <img src=https://avatars.githubusercontent.com/u/69133647?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Fernando De Lucchi/>
@@ -364,13 +371,6 @@ make build
             <sub style="font-size:14px"><b>Orville Q. Song</b></sub>
         </a>
     </td>
-    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/majst01>
-            <img src=https://avatars.githubusercontent.com/u/410110?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Stefan Majer/>
-            <br />
-            <sub style="font-size:14px"><b>Stefan Majer</b></sub>
-        </a>
-    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/hdhoang>
             <img src=https://avatars.githubusercontent.com/u/12537?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=hdhoang/>
@@ -401,6 +401,13 @@ make build
             <sub style="font-size:14px"><b>Jamie Greeff</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/jonathanspw>
+            <img src=https://avatars.githubusercontent.com/u/8390543?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Jonathan Wright/>
+            <br />
+            <sub style="font-size:14px"><b>Jonathan Wright</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/ChibangLW>
             <img src=https://avatars.githubusercontent.com/u/22293464?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=ChibangLW/>
@@ -408,6 +415,13 @@ make build
             <sub style="font-size:14px"><b>ChibangLW</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/majabojarska>
+            <img src=https://avatars.githubusercontent.com/u/33836570?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Maja Bojarska/>
+            <br />
+            <sub style="font-size:14px"><b>Maja Bojarska</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/mevansam>
             <img src=https://avatars.githubusercontent.com/u/403630?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Mevan Samaratunga/>
@@ -422,6 +436,8 @@ make build
             <sub style="font-size:14px"><b>Michael G.</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/ptman>
             <img src=https://avatars.githubusercontent.com/u/24669?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Paul Tötterman/>
@@ -436,8 +452,27 @@ make build
             <sub style="font-size:14px"><b>Samuel Lock</b></sub>
         </a>
     </td>
-</tr>
-<tr>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/loprima-l>
+            <img src=https://avatars.githubusercontent.com/u/69201633?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=loprima-l/>
+            <br />
+            <sub style="font-size:14px"><b>loprima-l</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/unreality>
+            <img src=https://avatars.githubusercontent.com/u/352522?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=unreality/>
+            <br />
+            <sub style="font-size:14px"><b>unreality</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/vsychov>
+            <img src=https://avatars.githubusercontent.com/u/2186303?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=MichaelKo/>
+            <br />
+            <sub style="font-size:14px"><b>MichaelKo</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/kevin1sMe>
             <img src=https://avatars.githubusercontent.com/u/6886076?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=kevinlin/>
@@ -445,6 +480,8 @@ make build
             <sub style="font-size:14px"><b>kevinlin</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/QZAiXH>
             <img src=https://avatars.githubusercontent.com/u/23068780?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Snack/>
@@ -480,15 +517,15 @@ make build
             <sub style="font-size:14px"><b>Josh Taylor</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/CNLHC>
-            <img src=https://avatars.githubusercontent.com/u/21005146?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=LiuHanCheng/>
+            <img src=https://avatars.githubusercontent.com/u/21005146?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=LIU HANCHENG/>
             <br />
-            <sub style="font-size:14px"><b>LiuHanCheng</b></sub>
+            <sub style="font-size:14px"><b>LIU HANCHENG</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/motiejus>
             <img src=https://avatars.githubusercontent.com/u/107720?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Motiejus Jakštys/>
@@ -524,15 +561,15 @@ make build
             <sub style="font-size:14px"><b>Victor Freire</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/lachy2849>
-            <img src=https://avatars.githubusercontent.com/u/98844035?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=lachy2849/>
+        <a href=https://github.com/qzydustin>
+            <img src=https://avatars.githubusercontent.com/u/44362429?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Zhenyu Qi/>
             <br />
-            <sub style="font-size:14px"><b>lachy2849</b></sub>
+            <sub style="font-size:14px"><b>Zhenyu Qi</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/t56k>
             <img src=https://avatars.githubusercontent.com/u/12165422?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=thomas/>
@@ -540,6 +577,20 @@ make build
             <sub style="font-size:14px"><b>thomas</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/puzpuzpuz>
+            <img src=https://avatars.githubusercontent.com/u/37772591?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Andrei Pechkurov/>
+            <br />
+            <sub style="font-size:14px"><b>Andrei Pechkurov</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/linsomniac>
+            <img src=https://avatars.githubusercontent.com/u/466380?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Sean Reifschneider/>
+            <br />
+            <sub style="font-size:14px"><b>Sean Reifschneider</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/aberoham>
             <img src=https://avatars.githubusercontent.com/u/586805?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Abraham Ingersoll/>
@@ -554,13 +605,6 @@ make build
             <sub style="font-size:14px"><b>Albert Copeland</b></sub>
         </a>
     </td>
-    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/puzpuzpuz>
-            <img src=https://avatars.githubusercontent.com/u/37772591?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Andrei Pechkurov/>
-            <br />
-            <sub style="font-size:14px"><b>Andrei Pechkurov</b></sub>
-        </a>
-    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/theryecatcher>
             <img src=https://avatars.githubusercontent.com/u/16442416?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Anoop Sundaresh/>
@@ -577,6 +621,13 @@ make build
             <sub style="font-size:14px"><b>Antoine POPINEAU</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/tony1661>
+            <img src=https://avatars.githubusercontent.com/u/5287266?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Antonio Fernandez/>
+            <br />
+            <sub style="font-size:14px"><b>Antonio Fernandez</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/aofei>
             <img src=https://avatars.githubusercontent.com/u/5037285?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Aofei Sheng/>
@@ -605,6 +656,15 @@ make build
             <sub style="font-size:14px"><b>Avirut Mehta</b></sub>
         </a>
     </td>
+</tr>
+<tr>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/winterheart>
+            <img src=https://avatars.githubusercontent.com/u/81112?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Azamat H. Hackimov/>
+            <br />
+            <sub style="font-size:14px"><b>Azamat H. Hackimov</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/stensonb>
             <img src=https://avatars.githubusercontent.com/u/933389?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Bryan Stenson/>
@@ -612,8 +672,6 @@ make build
             <sub style="font-size:14px"><b>Bryan Stenson</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/yangchuansheng>
             <img src=https://avatars.githubusercontent.com/u/15308462?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt= Carson Yang/>
@@ -623,9 +681,9 @@ make build
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/kundel>
-            <img src=https://avatars.githubusercontent.com/u/10158899?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=kundel/>
+            <img src=https://avatars.githubusercontent.com/u/10158899?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Darrell Kundel/>
             <br />
-            <sub style="font-size:14px"><b>kundel</b></sub>
+            <sub style="font-size:14px"><b>Darrell Kundel</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -642,6 +700,8 @@ make build
             <sub style="font-size:14px"><b>Felix Kronlage-Dammers</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/felixonmars>
             <img src=https://avatars.githubusercontent.com/u/1006477?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Felix Yan/>
@@ -649,6 +709,13 @@ make build
             <sub style="font-size:14px"><b>Felix Yan</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/gabe565>
+            <img src=https://avatars.githubusercontent.com/u/7717888?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Gabe Cook/>
+            <br />
+            <sub style="font-size:14px"><b>Gabe Cook</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/JJGadgets>
             <img src=https://avatars.githubusercontent.com/u/5709019?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=JJGadgets/>
@@ -656,8 +723,6 @@ make build
             <sub style="font-size:14px"><b>JJGadgets</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/hrtkpf>
             <img src=https://avatars.githubusercontent.com/u/42646788?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=hrtkpf/>
@@ -665,6 +730,13 @@ make build
             <sub style="font-size:14px"><b>hrtkpf</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/jessebot>
+            <img src=https://avatars.githubusercontent.com/u/2389292?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=JesseBot/>
+            <br />
+            <sub style="font-size:14px"><b>JesseBot</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/jimt>
             <img src=https://avatars.githubusercontent.com/u/180326?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Jim Tittsler/>
@@ -672,6 +744,8 @@ make build
             <sub style="font-size:14px"><b>Jim Tittsler</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/jsiebens>
             <img src=https://avatars.githubusercontent.com/u/499769?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Johan Siebens/>
@@ -693,6 +767,29 @@ make build
             <sub style="font-size:14px"><b>Jonathan de Jong</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/JulienFloris>
+            <img src=https://avatars.githubusercontent.com/u/20380255?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Julien Zweverink/>
+            <br />
+            <sub style="font-size:14px"><b>Julien Zweverink</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/win-t>
+            <img src=https://avatars.githubusercontent.com/u/1589120?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Kurnia D Win/>
+            <br />
+            <sub style="font-size:14px"><b>Kurnia D Win</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/Lucalux>
+            <img src=https://avatars.githubusercontent.com/u/70356955?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Lucalux/>
+            <br />
+            <sub style="font-size:14px"><b>Lucalux</b></sub>
+        </a>
+    </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/foxtrot>
             <img src=https://avatars.githubusercontent.com/u/4153572?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Marc/>
@@ -700,13 +797,11 @@ make build
             <sub style="font-size:14px"><b>Marc</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/magf>
-            <img src=https://avatars.githubusercontent.com/u/11992737?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Maxim Gajdaj/>
+        <a href=https://github.com/mhameed>
+            <img src=https://avatars.githubusercontent.com/u/447017?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Mesar Hameed/>
             <br />
-            <sub style="font-size:14px"><b>Maxim Gajdaj</b></sub>
+            <sub style="font-size:14px"><b>Mesar Hameed</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -716,6 +811,13 @@ make build
             <sub style="font-size:14px"><b>Michael Savage</b></sub>
         </a>
     </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/pkrivanec>
+            <img src=https://avatars.githubusercontent.com/u/25530641?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Philipp Krivanec/>
+            <br />
+            <sub style="font-size:14px"><b>Philipp Krivanec</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/piec>
             <img src=https://avatars.githubusercontent.com/u/781471?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Pierre Carru/>
@@ -724,12 +826,14 @@ make build
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/Donran>
+        <a href=https://github.com/donran>
             <img src=https://avatars.githubusercontent.com/u/4838348?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Pontus N/>
             <br />
             <sub style="font-size:14px"><b>Pontus N</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/nnsee>
             <img src=https://avatars.githubusercontent.com/u/36747857?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Rasmus Moorats/>
@@ -744,8 +848,6 @@ make build
             <sub style="font-size:14px"><b>rcursaru</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/renovate-bot>
             <img src=https://avatars.githubusercontent.com/u/25180681?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Mend Renovate/>
@@ -761,10 +863,10 @@ make build
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/linsomniac>
-            <img src=https://avatars.githubusercontent.com/u/466380?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Sean Reifschneider/>
+        <a href=https://github.com/muzy>
+            <img src=https://avatars.githubusercontent.com/u/321723?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Sebastian/>
             <br />
-            <sub style="font-size:14px"><b>Sean Reifschneider</b></sub>
+            <sub style="font-size:14px"><b>Sebastian</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -774,6 +876,15 @@ make build
             <sub style="font-size:14px"><b>Shaanan Cohney</b></sub>
         </a>
     </td>
+</tr>
+<tr>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/6ixfalls>
+            <img src=https://avatars.githubusercontent.com/u/23470032?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Six/>
+            <br />
+            <sub style="font-size:14px"><b>Six</b></sub>
+        </a>
+    </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/stefanvanburen>
             <img src=https://avatars.githubusercontent.com/u/622527?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Stefan VanBuren/>
@@ -788,8 +899,6 @@ make build
             <sub style="font-size:14px"><b>sophware</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/m-tanner-dev0>
             <img src=https://avatars.githubusercontent.com/u/97977342?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Tanner/>
@@ -811,6 +920,8 @@ make build
             <sub style="font-size:14px"><b>The Gitter Badger</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/tianon>
             <img src=https://avatars.githubusercontent.com/u/161631?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Tianon Gravi/>
@@ -832,17 +943,15 @@ make build
             <sub style="font-size:14px"><b>Tjerk Woudsma</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/y0ngb1n>
-            <img src=https://avatars.githubusercontent.com/u/25719408?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Yang Bin/>
+            <img src=https://avatars.githubusercontent.com/u/25719408?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=杨斌 Aben/>
             <br />
-            <sub style="font-size:14px"><b>Yang Bin</b></sub>
+            <sub style="font-size:14px"><b>杨斌 Aben</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/gozssky>
+        <a href=https://github.com/sleepymole>
             <img src=https://avatars.githubusercontent.com/u/17199941?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Yujie Xia/>
             <br />
             <sub style="font-size:14px"><b>Yujie Xia</b></sub>
@@ -855,6 +964,8 @@ make build
             <sub style="font-size:14px"><b>Zachary Newell</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/zekker6>
             <img src=https://avatars.githubusercontent.com/u/1367798?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Zakhar Bessarab/>
@@ -876,8 +987,6 @@ make build
             <sub style="font-size:14px"><b>Ziyuan Han</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/caelansar>
             <img src=https://avatars.githubusercontent.com/u/31852257?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=caelansar/>
@@ -899,6 +1008,8 @@ make build
             <sub style="font-size:14px"><b>dnaq</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/nning>
             <img src=https://avatars.githubusercontent.com/u/557430?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=henning mueller/>
@@ -920,8 +1031,6 @@ make build
             <sub style="font-size:14px"><b>jimyag</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/magichuihui>
             <img src=https://avatars.githubusercontent.com/u/10866198?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=suhelen/>
@@ -943,6 +1052,8 @@ make build
             <sub style="font-size:14px"><b>ma6174</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/manju-rn>
             <img src=https://avatars.githubusercontent.com/u/26291847?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=manju-rn/>
@@ -951,10 +1062,17 @@ make build
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/pernila>
-            <img src=https://avatars.githubusercontent.com/u/12460060?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=pernila/>
+        <a href=https://github.com/nicholas-yap>
+            <img src=https://avatars.githubusercontent.com/u/38109533?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=nicholas-yap/>
             <br />
-            <sub style="font-size:14px"><b>pernila</b></sub>
+            <sub style="font-size:14px"><b>nicholas-yap</b></sub>
+        </a>
+    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/pernila>
+            <img src=https://avatars.githubusercontent.com/u/12460060?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Tommi Pernila/>
+            <br />
+            <sub style="font-size:14px"><b>Tommi Pernila</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -964,13 +1082,11 @@ make build
             <sub style="font-size:14px"><b>phpmalik</b></sub>
         </a>
     </td>
-</tr>
-<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/Wakeful-Cloud>
-            <img src=https://avatars.githubusercontent.com/u/38930607?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Wakeful-Cloud/>
+            <img src=https://avatars.githubusercontent.com/u/38930607?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Wakeful Cloud/>
             <br />
-            <sub style="font-size:14px"><b>Wakeful-Cloud</b></sub>
+            <sub style="font-size:14px"><b>Wakeful Cloud</b></sub>
         </a>
     </td>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -980,6 +1096,8 @@ make build
             <sub style="font-size:14px"><b>zy</b></sub>
         </a>
     </td>
+</tr>
+<tr>
     <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
         <a href=https://github.com/atorregrosa-smd>
             <img src=https://avatars.githubusercontent.com/u/78434679?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Àlex Torregrosa/>

acls.go

@@ -1,780 +0,0 @@
-package headscale
-
-import (
-	"encoding/json"
-	"errors"
-	"fmt"
-	"io"
-	"net/netip"
-	"os"
-	"path/filepath"
-	"strconv"
-	"strings"
-	"time"
-
-	"github.com/rs/zerolog/log"
-	"github.com/tailscale/hujson"
-	"go4.org/netipx"
-	"gopkg.in/yaml.v3"
-	"tailscale.com/envknob"
-	"tailscale.com/tailcfg"
-)
-
-const (
-	errEmptyPolicy       = Error("empty policy")
-	errInvalidAction     = Error("invalid action")
-	errInvalidGroup      = Error("invalid group")
-	errInvalidTag        = Error("invalid tag")
-	errInvalidPortFormat = Error("invalid port format")
-	errWildcardIsNeeded  = Error("wildcard as port is required for the protocol")
-)
-
-const (
-	Base8              = 8
-	Base10             = 10
-	BitSize16          = 16
-	BitSize32          = 32
-	BitSize64          = 64
-	portRangeBegin     = 0
-	portRangeEnd       = 65535
-	expectedTokenItems = 2
-)
-
-// For some reason golang.org/x/net/internal/iana is an internal package.
-const (
-	protocolICMP     = 1   // Internet Control Message
-	protocolIGMP     = 2   // Internet Group Management
-	protocolIPv4     = 4   // IPv4 encapsulation
-	protocolTCP      = 6   // Transmission Control
-	protocolEGP      = 8   // Exterior Gateway Protocol
-	protocolIGP      = 9   // any private interior gateway (used by Cisco for their IGRP)
-	protocolUDP      = 17  // User Datagram
-	protocolGRE      = 47  // Generic Routing Encapsulation
-	protocolESP      = 50  // Encap Security Payload
-	protocolAH       = 51  // Authentication Header
-	protocolIPv6ICMP = 58  // ICMP for IPv6
-	protocolSCTP     = 132 // Stream Control Transmission Protocol
-	ProtocolFC       = 133 // Fibre Channel
-)
-
-var featureEnableSSH = envknob.RegisterBool("HEADSCALE_EXPERIMENTAL_FEATURE_SSH")
-
-// LoadACLPolicy loads the ACL policy from the specify path, and generates the ACL rules.
-func (h *Headscale) LoadACLPolicy(path string) error {
-	log.Debug().
-		Str("func", "LoadACLPolicy").
-		Str("path", path).
-		Msg("Loading ACL policy from path")
-
-	policyFile, err := os.Open(path)
-	if err != nil {
-		return err
-	}
-	defer policyFile.Close()
-
-	var policy ACLPolicy
-	policyBytes, err := io.ReadAll(policyFile)
-	if err != nil {
-		return err
-	}
-
-	switch filepath.Ext(path) {
-	case ".yml", ".yaml":
-		log.Debug().
-			Str("path", path).
-			Bytes("file", policyBytes).
-			Msg("Loading ACLs from YAML")
-
-		err := yaml.Unmarshal(policyBytes, &policy)
-		if err != nil {
-			return err
-		}
-
-		log.Trace().
-			Interface("policy", policy).
-			Msg("Loaded policy from YAML")
-
-	default:
-		ast, err := hujson.Parse(policyBytes)
-		if err != nil {
-			return err
-		}
-
-		ast.Standardize()
-		policyBytes = ast.Pack()
-		err = json.Unmarshal(policyBytes, &policy)
-		if err != nil {
-			return err
-		}
-	}
-
-	if policy.IsZero() {
-		return errEmptyPolicy
-	}
-
-	h.aclPolicy = &policy
-
-	return h.UpdateACLRules()
-}
-
-func (h *Headscale) UpdateACLRules() error {
-	machines, err := h.ListMachines()
-	if err != nil {
-		return err
-	}
-
-	if h.aclPolicy == nil {
-		return errEmptyPolicy
-	}
-
-	rules, err := generateACLRules(machines, *h.aclPolicy, h.cfg.OIDC.StripEmaildomain)
-	if err != nil {
-		return err
-	}
-	log.Trace().Interface("ACL", rules).Msg("ACL rules generated")
-	h.aclRules = rules
-
-	// Precompute a map of which sources can reach each destination, this is
-	// to provide quicker lookup when we calculate the peerlist for the map
-	// response to nodes.
-	aclPeerCacheMap := generateACLPeerCacheMap(rules)
-	h.aclPeerCacheMapRW.Lock()
-	h.aclPeerCacheMap = aclPeerCacheMap
-	h.aclPeerCacheMapRW.Unlock()
-
-	if featureEnableSSH() {
-		sshRules, err := h.generateSSHRules()
-		if err != nil {
-			return err
-		}
-		log.Trace().Interface("SSH", sshRules).Msg("SSH rules generated")
-		if h.sshPolicy == nil {
-			h.sshPolicy = &tailcfg.SSHPolicy{}
-		}
-		h.sshPolicy.Rules = sshRules
-	} else if h.aclPolicy != nil && len(h.aclPolicy.SSHs) > 0 {
-		log.Info().Msg("SSH ACLs has been defined, but HEADSCALE_EXPERIMENTAL_FEATURE_SSH is not enabled, this is a unstable feature, check docs before activating")
-	}
-
-	return nil
-}
-
-// generateACLPeerCacheMap takes a list of Tailscale filter rules and generates a map
-// of which Sources ("*" and IPs) can access destinations. This is to speed up the
-// process of generating MapResponses when deciding which Peers to inform nodes about.
-func generateACLPeerCacheMap(rules []tailcfg.FilterRule) map[string]map[string]struct{} {
-	aclCachePeerMap := make(map[string]map[string]struct{})
-	for _, rule := range rules {
-		for _, srcIP := range rule.SrcIPs {
-			for _, ip := range expandACLPeerAddr(srcIP) {
-				if data, ok := aclCachePeerMap[ip]; ok {
-					for _, dstPort := range rule.DstPorts {
-						for _, dstIP := range expandACLPeerAddr(dstPort.IP) {
-							data[dstIP] = struct{}{}
-						}
-					}
-				} else {
-					dstPortsMap := make(map[string]struct{}, len(rule.DstPorts))
-					for _, dstPort := range rule.DstPorts {
-						for _, dstIP := range expandACLPeerAddr(dstPort.IP) {
-							dstPortsMap[dstIP] = struct{}{}
-						}
-					}
-					aclCachePeerMap[ip] = dstPortsMap
-				}
-			}
-		}
-	}
-
-	log.Trace().Interface("ACL Cache Map", aclCachePeerMap).Msg("ACL Peer Cache Map generated")
-
-	return aclCachePeerMap
-}
-
-// expandACLPeerAddr takes a "tailcfg.FilterRule" "IP" and expands it into
-// something our cache logic can look up, which is "*" or single IP addresses.
-// This is probably quite inefficient, but it is a result of
-// "make it work, then make it fast", and a lot of the ACL stuff does not
-// work, but people have tried to make it fast.
-func expandACLPeerAddr(srcIP string) []string {
-	if ip, err := netip.ParseAddr(srcIP); err == nil {
-		return []string{ip.String()}
-	}
-
-	if cidr, err := netip.ParsePrefix(srcIP); err == nil {
-		addrs := []string{}
-
-		ipRange := netipx.RangeOfPrefix(cidr)
-
-		from := ipRange.From()
-		too := ipRange.To()
-
-		if from == too {
-			return []string{from.String()}
-		}
-
-		for from != too && from.Less(too) {
-			addrs = append(addrs, from.String())
-			from = from.Next()
-		}
-		addrs = append(addrs, too.String()) // Add the last IP address in the range
-
-		return addrs
-	}
-
-	// probably "*" or other string based "IP"
-	return []string{srcIP}
-}
-
-func generateACLRules(
-	machines []Machine,
-	aclPolicy ACLPolicy,
-	stripEmaildomain bool,
-) ([]tailcfg.FilterRule, error) {
-	rules := []tailcfg.FilterRule{}
-
-	for index, acl := range aclPolicy.ACLs {
-		if acl.Action != "accept" {
-			return nil, errInvalidAction
-		}
-
-		srcIPs := []string{}
-		for innerIndex, src := range acl.Sources {
-			srcs, err := generateACLPolicySrc(machines, aclPolicy, src, stripEmaildomain)
-			if err != nil {
-				log.Error().
-					Msgf("Error parsing ACL %d, Source %d", index, innerIndex)
-
-				return nil, err
-			}
-			srcIPs = append(srcIPs, srcs...)
-		}
-
-		protocols, needsWildcard, err := parseProtocol(acl.Protocol)
-		if err != nil {
-			log.Error().
-				Msgf("Error parsing ACL %d. protocol unknown %s", index, acl.Protocol)
-
-			return nil, err
-		}
-
-		destPorts := []tailcfg.NetPortRange{}
-		for innerIndex, dest := range acl.Destinations {
-			dests, err := generateACLPolicyDest(
-				machines,
-				aclPolicy,
-				dest,
-				needsWildcard,
-				stripEmaildomain,
-			)
-			if err != nil {
-				log.Error().
-					Msgf("Error parsing ACL %d, Destination %d", index, innerIndex)
-
-				return nil, err
-			}
-			destPorts = append(destPorts, dests...)
-		}
-
-		rules = append(rules, tailcfg.FilterRule{
-			SrcIPs:   srcIPs,
-			DstPorts: destPorts,
-			IPProto:  protocols,
-		})
-	}
-
-	return rules, nil
-}
-
-func (h *Headscale) generateSSHRules() ([]*tailcfg.SSHRule, error) {
-	rules := []*tailcfg.SSHRule{}
-
-	if h.aclPolicy == nil {
-		return nil, errEmptyPolicy
-	}
-
-	machines, err := h.ListMachines()
-	if err != nil {
-		return nil, err
-	}
-
-	acceptAction := tailcfg.SSHAction{
-		Message:                  "",
-		Reject:                   false,
-		Accept:                   true,
-		SessionDuration:          0,
-		AllowAgentForwarding:     false,
-		HoldAndDelegate:          "",
-		AllowLocalPortForwarding: true,
-	}
-
-	rejectAction := tailcfg.SSHAction{
-		Message:                  "",
-		Reject:                   true,
-		Accept:                   false,
-		SessionDuration:          0,
-		AllowAgentForwarding:     false,
-		HoldAndDelegate:          "",
-		AllowLocalPortForwarding: false,
-	}
-
-	for index, sshACL := range h.aclPolicy.SSHs {
-		action := rejectAction
-		switch sshACL.Action {
-		case "accept":
-			action = acceptAction
-		case "check":
-			checkAction, err := sshCheckAction(sshACL.CheckPeriod)
-			if err != nil {
-				log.Error().
-					Msgf("Error parsing SSH %d, check action with unparsable duration '%s'", index, sshACL.CheckPeriod)
-			} else {
-				action = *checkAction
-			}
-		default:
-			log.Error().
-				Msgf("Error parsing SSH %d, unknown action '%s'", index, sshACL.Action)
-
-			return nil, err
-		}
-
-		principals := make([]*tailcfg.SSHPrincipal, 0, len(sshACL.Sources))
-		for innerIndex, rawSrc := range sshACL.Sources {
-			expandedSrcs, err := expandAlias(
-				machines,
-				*h.aclPolicy,
-				rawSrc,
-				h.cfg.OIDC.StripEmaildomain,
-			)
-			if err != nil {
-				log.Error().
-					Msgf("Error parsing SSH %d, Source %d", index, innerIndex)
-
-				return nil, err
-			}
-			for _, expandedSrc := range expandedSrcs {
-				principals = append(principals, &tailcfg.SSHPrincipal{
-					NodeIP: expandedSrc,
-				})
-			}
-		}
-
-		userMap := make(map[string]string, len(sshACL.Users))
-		for _, user := range sshACL.Users {
-			userMap[user] = "="
-		}
-		rules = append(rules, &tailcfg.SSHRule{
-			RuleExpires: nil,
-			Principals:  principals,
-			SSHUsers:    userMap,
-			Action:      &action,
-		})
-	}
-
-	return rules, nil
-}
-
-func sshCheckAction(duration string) (*tailcfg.SSHAction, error) {
-	sessionLength, err := time.ParseDuration(duration)
-	if err != nil {
-		return nil, err
-	}
-
-	return &tailcfg.SSHAction{
-		Message:                  "",
-		Reject:                   false,
-		Accept:                   true,
-		SessionDuration:          sessionLength,
-		AllowAgentForwarding:     false,
-		HoldAndDelegate:          "",
-		AllowLocalPortForwarding: true,
-	}, nil
-}
-
-func generateACLPolicySrc(
-	machines []Machine,
-	aclPolicy ACLPolicy,
-	src string,
-	stripEmaildomain bool,
-) ([]string, error) {
-	return expandAlias(machines, aclPolicy, src, stripEmaildomain)
-}
-
-func generateACLPolicyDest(
-	machines []Machine,
-	aclPolicy ACLPolicy,
-	dest string,
-	needsWildcard bool,
-	stripEmaildomain bool,
-) ([]tailcfg.NetPortRange, error) {
-	tokens := strings.Split(dest, ":")
-	if len(tokens) < expectedTokenItems || len(tokens) > 3 {
-		return nil, errInvalidPortFormat
-	}
-
-	var alias string
-	// We can have here stuff like:
-	// git-server:*
-	// 192.168.1.0/24:22
-	// tag:montreal-webserver:80,443
-	// tag:api-server:443
-	// example-host-1:*
-	if len(tokens) == expectedTokenItems {
-		alias = tokens[0]
-	} else {
-		alias = fmt.Sprintf("%s:%s", tokens[0], tokens[1])
-	}
-
-	expanded, err := expandAlias(
-		machines,
-		aclPolicy,
-		alias,
-		stripEmaildomain,
-	)
-	if err != nil {
-		return nil, err
-	}
-	ports, err := expandPorts(tokens[len(tokens)-1], needsWildcard)
-	if err != nil {
-		return nil, err
-	}
-
-	dests := []tailcfg.NetPortRange{}
-	for _, d := range expanded {
-		for _, p := range *ports {
-			pr := tailcfg.NetPortRange{
-				IP:    d,
-				Ports: p,
-			}
-			dests = append(dests, pr)
-		}
-	}
-
-	return dests, nil
-}
-
-// parseProtocol reads the proto field of the ACL and generates a list of
-// protocols that will be allowed, following the IANA IP protocol number
-// https://www.iana.org/assignments/protocol-numbers/protocol-numbers.xhtml
-//
-// If the ACL proto field is empty, it allows ICMPv4, ICMPv6, TCP, and UDP,
-// as per Tailscale behaviour (see tailcfg.FilterRule).
-//
-// Also returns a boolean indicating if the protocol
-// requires all the destinations to use wildcard as port number (only TCP,
-// UDP and SCTP support specifying ports).
-func parseProtocol(protocol string) ([]int, bool, error) {
-	switch protocol {
-	case "":
-		return nil, false, nil
-	case "igmp":
-		return []int{protocolIGMP}, true, nil
-	case "ipv4", "ip-in-ip":
-		return []int{protocolIPv4}, true, nil
-	case "tcp":
-		return []int{protocolTCP}, false, nil
-	case "egp":
-		return []int{protocolEGP}, true, nil
-	case "igp":
-		return []int{protocolIGP}, true, nil
-	case "udp":
-		return []int{protocolUDP}, false, nil
-	case "gre":
-		return []int{protocolGRE}, true, nil
-	case "esp":
-		return []int{protocolESP}, true, nil
-	case "ah":
-		return []int{protocolAH}, true, nil
-	case "sctp":
-		return []int{protocolSCTP}, false, nil
-	case "icmp":
-		return []int{protocolICMP, protocolIPv6ICMP}, true, nil
-
-	default:
-		protocolNumber, err := strconv.Atoi(protocol)
-		if err != nil {
-			return nil, false, err
-		}
-		needsWildcard := protocolNumber != protocolTCP &&
-			protocolNumber != protocolUDP &&
-			protocolNumber != protocolSCTP
-
-		return []int{protocolNumber}, needsWildcard, nil
-	}
-}
-
-// expandalias has an input of either
-// - a user
-// - a group
-// - a tag
-// - a host
-// and transform these in IPAddresses.
-func expandAlias(
-	machines []Machine,
-	aclPolicy ACLPolicy,
-	alias string,
-	stripEmailDomain bool,
-) ([]string, error) {
-	ips := []string{}
-	if alias == "*" {
-		return []string{"*"}, nil
-	}
-
-	log.Debug().
-		Str("alias", alias).
-		Msg("Expanding")
-
-	if strings.HasPrefix(alias, "group:") {
-		users, err := expandGroup(aclPolicy, alias, stripEmailDomain)
-		if err != nil {
-			return ips, err
-		}
-		for _, n := range users {
-			nodes := filterMachinesByUser(machines, n)
-			for _, node := range nodes {
-				ips = append(ips, node.IPAddresses.ToStringSlice()...)
-			}
-		}
-
-		return ips, nil
-	}
-
-	if strings.HasPrefix(alias, "tag:") {
-		// check for forced tags
-		for _, machine := range machines {
-			if contains(machine.ForcedTags, alias) {
-				ips = append(ips, machine.IPAddresses.ToStringSlice()...)
-			}
-		}
-
-		// find tag owners
-		owners, err := expandTagOwners(aclPolicy, alias, stripEmailDomain)
-		if err != nil {
-			if errors.Is(err, errInvalidTag) {
-				if len(ips) == 0 {
-					return ips, fmt.Errorf(
-						"%w. %v isn't owned by a TagOwner and no forced tags are defined",
-						errInvalidTag,
-						alias,
-					)
-				}
-
-				return ips, nil
-			} else {
-				return ips, err
-			}
-		}
-
-		// filter out machines per tag owner
-		for _, user := range owners {
-			machines := filterMachinesByUser(machines, user)
-			for _, machine := range machines {
-				hi := machine.GetHostInfo()
-				if contains(hi.RequestTags, alias) {
-					ips = append(ips, machine.IPAddresses.ToStringSlice()...)
-				}
-			}
-		}
-
-		return ips, nil
-	}
-
-	// if alias is a user
-	nodes := filterMachinesByUser(machines, alias)
-	nodes = excludeCorrectlyTaggedNodes(aclPolicy, nodes, alias, stripEmailDomain)
-
-	for _, n := range nodes {
-		ips = append(ips, n.IPAddresses.ToStringSlice()...)
-	}
-	if len(ips) > 0 {
-		return ips, nil
-	}
-
-	// if alias is an host
-	if h, ok := aclPolicy.Hosts[alias]; ok {
-		return []string{h.String()}, nil
-	}
-
-	// if alias is an IP
-	ip, err := netip.ParseAddr(alias)
-	if err == nil {
-		return []string{ip.String()}, nil
-	}
-
-	// if alias is an CIDR
-	cidr, err := netip.ParsePrefix(alias)
-	if err == nil {
-		return []string{cidr.String()}, nil
-	}
-
-	log.Warn().Msgf("No IPs found with the alias %v", alias)
-
-	return ips, nil
-}
-
-// excludeCorrectlyTaggedNodes will remove from the list of input nodes the ones
-// that are correctly tagged since they should not be listed as being in the user
-// we assume in this function that we only have nodes from 1 user.
-func excludeCorrectlyTaggedNodes(
-	aclPolicy ACLPolicy,
-	nodes []Machine,
-	user string,
-	stripEmailDomain bool,
-) []Machine {
-	out := []Machine{}
-	tags := []string{}
-	for tag := range aclPolicy.TagOwners {
-		owners, _ := expandTagOwners(aclPolicy, user, stripEmailDomain)
-		ns := append(owners, user)
-		if contains(ns, user) {
-			tags = append(tags, tag)
-		}
-	}
-	// for each machine if tag is in tags list, don't append it.
-	for _, machine := range nodes {
-		hi := machine.GetHostInfo()
-
-		found := false
-		for _, t := range hi.RequestTags {
-			if contains(tags, t) {
-				found = true
-
-				break
-			}
-		}
-		if len(machine.ForcedTags) > 0 {
-			found = true
-		}
-		if !found {
-			out = append(out, machine)
-		}
-	}
-
-	return out
-}
-
-func expandPorts(portsStr string, needsWildcard bool) (*[]tailcfg.PortRange, error) {
-	if portsStr == "*" {
-		return &[]tailcfg.PortRange{
-			{First: portRangeBegin, Last: portRangeEnd},
-		}, nil
-	}
-
-	if needsWildcard {
-		return nil, errWildcardIsNeeded
-	}
-
-	ports := []tailcfg.PortRange{}
-	for _, portStr := range strings.Split(portsStr, ",") {
-		rang := strings.Split(portStr, "-")
-		switch len(rang) {
-		case 1:
-			port, err := strconv.ParseUint(rang[0], Base10, BitSize16)
-			if err != nil {
-				return nil, err
-			}
-			ports = append(ports, tailcfg.PortRange{
-				First: uint16(port),
-				Last:  uint16(port),
-			})
-
-		case expectedTokenItems:
-			start, err := strconv.ParseUint(rang[0], Base10, BitSize16)
-			if err != nil {
-				return nil, err
-			}
-			last, err := strconv.ParseUint(rang[1], Base10, BitSize16)
-			if err != nil {
-				return nil, err
-			}
-			ports = append(ports, tailcfg.PortRange{
-				First: uint16(start),
-				Last:  uint16(last),
-			})
-
-		default:
-			return nil, errInvalidPortFormat
-		}
-	}
-
-	return &ports, nil
-}
-
-func filterMachinesByUser(machines []Machine, user string) []Machine {
-	out := []Machine{}
-	for _, machine := range machines {
-		if machine.User.Name == user {
-			out = append(out, machine)
-		}
-	}
-
-	return out
-}
-
-// expandTagOwners will return a list of user. An owner can be either a user or a group
-// a group cannot be composed of groups.
-func expandTagOwners(
-	aclPolicy ACLPolicy,
-	tag string,
-	stripEmailDomain bool,
-) ([]string, error) {
-	var owners []string
-	ows, ok := aclPolicy.TagOwners[tag]
-	if !ok {
-		return []string{}, fmt.Errorf(
-			"%w. %v isn't owned by a TagOwner. Please add one first. https://tailscale.com/kb/1018/acls/#tag-owners",
-			errInvalidTag,
-			tag,
-		)
-	}
-	for _, owner := range ows {
-		if strings.HasPrefix(owner, "group:") {
-			gs, err := expandGroup(aclPolicy, owner, stripEmailDomain)
-			if err != nil {
-				return []string{}, err
-			}
-			owners = append(owners, gs...)
-		} else {
-			owners = append(owners, owner)
-		}
-	}
-
-	return owners, nil
-}
-
-// expandGroup will return the list of user inside the group
-// after some validation.
-func expandGroup(
-	aclPolicy ACLPolicy,
-	group string,
-	stripEmailDomain bool,
-) ([]string, error) {
-	outGroups := []string{}
-	aclGroups, ok := aclPolicy.Groups[group]
-	if !ok {
-		return []string{}, fmt.Errorf(
-			"group %v isn't registered. %w",
-			group,
-			errInvalidGroup,
-		)
-	}
-	for _, group := range aclGroups {
-		if strings.HasPrefix(group, "group:") {
-			return []string{}, fmt.Errorf(
-				"%w. A group cannot be composed of groups. https://tailscale.com/kb/1018/acls/#groups",
-				errInvalidGroup,
-			)
-		}
-		grp, err := NormalizeToFQDNRules(group, stripEmailDomain)
-		if err != nil {
-			return []string{}, fmt.Errorf(
-				"failed to normalize group %q, err: %w",
-				group,
-				errInvalidGroup,
-			)
-		}
-		outGroups = append(outGroups, grp)
-	}
-
-	return outGroups, nil
-}

api_common.go

@@ -1,113 +0,0 @@
-package headscale
-
-import (
-	"time"
-
-	"github.com/rs/zerolog/log"
-	"tailscale.com/tailcfg"
-)
-
-func (h *Headscale) generateMapResponse(
-	mapRequest tailcfg.MapRequest,
-	machine *Machine,
-) (*tailcfg.MapResponse, error) {
-	log.Trace().
-		Str("func", "generateMapResponse").
-		Str("machine", mapRequest.Hostinfo.Hostname).
-		Msg("Creating Map response")
-	node, err := h.toNode(*machine, h.cfg.BaseDomain, h.cfg.DNSConfig)
-	if err != nil {
-		log.Error().
-			Caller().
-			Str("func", "generateMapResponse").
-			Err(err).
-			Msg("Cannot convert to node")
-
-		return nil, err
-	}
-
-	peers, err := h.getValidPeers(machine)
-	if err != nil {
-		log.Error().
-			Caller().
-			Str("func", "generateMapResponse").
-			Err(err).
-			Msg("Cannot fetch peers")
-
-		return nil, err
-	}
-
-	profiles := h.getMapResponseUserProfiles(*machine, peers)
-
-	nodePeers, err := h.toNodes(peers, h.cfg.BaseDomain, h.cfg.DNSConfig)
-	if err != nil {
-		log.Error().
-			Caller().
-			Str("func", "generateMapResponse").
-			Err(err).
-			Msg("Failed to convert peers to Tailscale nodes")
-
-		return nil, err
-	}
-
-	dnsConfig := getMapResponseDNSConfig(
-		h.cfg.DNSConfig,
-		h.cfg.BaseDomain,
-		*machine,
-		peers,
-	)
-
-	now := time.Now()
-
-	resp := tailcfg.MapResponse{
-		KeepAlive: false,
-		Node:      node,
-
-		// TODO: Only send if updated
-		DERPMap: h.DERPMap,
-
-		// TODO: Only send if updated
-		Peers: nodePeers,
-
-		// TODO(kradalby): Implement:
-		// https://github.com/tailscale/tailscale/blob/main/tailcfg/tailcfg.go#L1351-L1374
-		// PeersChanged
-		// PeersRemoved
-		// PeersChangedPatch
-		// PeerSeenChange
-		// OnlineChange
-
-		// TODO: Only send if updated
-		DNSConfig: dnsConfig,
-
-		// TODO: Only send if updated
-		Domain: h.cfg.BaseDomain,
-
-		// Do not instruct clients to collect services, we do not
-		// support or do anything with them
-		CollectServices: "false",
-
-		// TODO: Only send if updated
-		PacketFilter: h.aclRules,
-
-		UserProfiles: profiles,
-
-		// TODO: Only send if updated
-		SSHPolicy: h.sshPolicy,
-
-		ControlTime: &now,
-
-		Debug: &tailcfg.Debug{
-			DisableLogTail:      !h.cfg.LogTail.Enabled,
-			RandomizeClientPort: h.cfg.RandomizeClientPort,
-		},
-	}
-
-	log.Trace().
-		Str("func", "generateMapResponse").
-		Str("machine", mapRequest.Hostinfo.Hostname).
-		// Interface("payload", resp).
-		Msgf("Generated map response: %s", tailMapResponseToString(resp))
-
-	return &resp, nil
-}

api_key.go

@@ -1,157 +0,0 @@
-package headscale
-
-import (
-	"fmt"
-	"strings"
-	"time"
-
-	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"golang.org/x/crypto/bcrypt"
-	"google.golang.org/protobuf/types/known/timestamppb"
-)
-
-const (
-	apiPrefixLength = 7
-	apiKeyLength    = 32
-
-	ErrAPIKeyFailedToParse = Error("Failed to parse ApiKey")
-)
-
-// APIKey describes the datamodel for API keys used to remotely authenticate with
-// headscale.
-type APIKey struct {
-	ID     uint64 `gorm:"primary_key"`
-	Prefix string `gorm:"uniqueIndex"`
-	Hash   []byte
-
-	CreatedAt  *time.Time
-	Expiration *time.Time
-	LastSeen   *time.Time
-}
-
-// CreateAPIKey creates a new ApiKey in a user, and returns it.
-func (h *Headscale) CreateAPIKey(
-	expiration *time.Time,
-) (string, *APIKey, error) {
-	prefix, err := GenerateRandomStringURLSafe(apiPrefixLength)
-	if err != nil {
-		return "", nil, err
-	}
-
-	toBeHashed, err := GenerateRandomStringURLSafe(apiKeyLength)
-	if err != nil {
-		return "", nil, err
-	}
-
-	// Key to return to user, this will only be visible _once_
-	keyStr := prefix + "." + toBeHashed
-
-	hash, err := bcrypt.GenerateFromPassword([]byte(toBeHashed), bcrypt.DefaultCost)
-	if err != nil {
-		return "", nil, err
-	}
-
-	key := APIKey{
-		Prefix:     prefix,
-		Hash:       hash,
-		Expiration: expiration,
-	}
-
-	if err := h.db.Save(&key).Error; err != nil {
-		return "", nil, fmt.Errorf("failed to save API key to database: %w", err)
-	}
-
-	return keyStr, &key, nil
-}
-
-// ListAPIKeys returns the list of ApiKeys for a user.
-func (h *Headscale) ListAPIKeys() ([]APIKey, error) {
-	keys := []APIKey{}
-	if err := h.db.Find(&keys).Error; err != nil {
-		return nil, err
-	}
-
-	return keys, nil
-}
-
-// GetAPIKey returns a ApiKey for a given key.
-func (h *Headscale) GetAPIKey(prefix string) (*APIKey, error) {
-	key := APIKey{}
-	if result := h.db.First(&key, "prefix = ?", prefix); result.Error != nil {
-		return nil, result.Error
-	}
-
-	return &key, nil
-}
-
-// GetAPIKeyByID returns a ApiKey for a given id.
-func (h *Headscale) GetAPIKeyByID(id uint64) (*APIKey, error) {
-	key := APIKey{}
-	if result := h.db.Find(&APIKey{ID: id}).First(&key); result.Error != nil {
-		return nil, result.Error
-	}
-
-	return &key, nil
-}
-
-// DestroyAPIKey destroys a ApiKey. Returns error if the ApiKey
-// does not exist.
-func (h *Headscale) DestroyAPIKey(key APIKey) error {
-	if result := h.db.Unscoped().Delete(key); result.Error != nil {
-		return result.Error
-	}
-
-	return nil
-}
-
-// ExpireAPIKey marks a ApiKey as expired.
-func (h *Headscale) ExpireAPIKey(key *APIKey) error {
-	if err := h.db.Model(&key).Update("Expiration", time.Now()).Error; err != nil {
-		return err
-	}
-
-	return nil
-}
-
-func (h *Headscale) ValidateAPIKey(keyStr string) (bool, error) {
-	prefix, hash, found := strings.Cut(keyStr, ".")
-	if !found {
-		return false, ErrAPIKeyFailedToParse
-	}
-
-	key, err := h.GetAPIKey(prefix)
-	if err != nil {
-		return false, fmt.Errorf("failed to validate api key: %w", err)
-	}
-
-	if key.Expiration.Before(time.Now()) {
-		return false, nil
-	}
-
-	if err := bcrypt.CompareHashAndPassword(key.Hash, []byte(hash)); err != nil {
-		return false, err
-	}
-
-	return true, nil
-}
-
-func (key *APIKey) toProto() *v1.ApiKey {
-	protoKey := v1.ApiKey{
-		Id:     key.ID,
-		Prefix: key.Prefix,
-	}
-
-	if key.Expiration != nil {
-		protoKey.Expiration = timestamppb.New(*key.Expiration)
-	}
-
-	if key.CreatedAt != nil {
-		protoKey.CreatedAt = timestamppb.New(*key.CreatedAt)
-	}
-
-	if key.LastSeen != nil {
-		protoKey.LastSeen = timestamppb.New(*key.LastSeen)
-	}
-
-	return &protoKey
-}

cmd/gh-action-integration-generator/main.go

@@ -31,7 +31,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test:
+  {{.Name}}:
     runs-on: ubuntu-latest
 
     steps:
@@ -39,6 +39,11 @@ jobs:
         with:
           fetch-depth: 2
 
+      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: satackey/action-docker-layer-caching@main
+        continue-on-error: true
+
       - name: Get changed files
         id: changed-files
         uses: tj-actions/changed-files@v34
@@ -50,12 +55,12 @@ jobs:
             integration_test/
             config-example.yaml
 
-      - uses: cachix/install-nix-action@v18
-        if: {{ "${{ env.ACT }}" }} || steps.changed-files.outputs.any_changed == 'true'
-
-      - name: Run general integration tests
+      - name: Run {{.Name}}
+        uses: Wandalen/wretry.action@master
         if: steps.changed-files.outputs.any_changed == 'true'
-        run: |
+        with:
+          attempt_limit: 5
+          command: |
             nix develop --command -- docker run \
               --tty --rm \
               --volume ~/.cache/hs-integration-go:/go \
@@ -64,8 +69,7 @@ jobs:
               --volume /var/run/docker.sock:/var/run/docker.sock \
               --volume $PWD/control_logs:/tmp/control \
               golang:1 \
-                go test ./... \
-                  -tags ts2019 \
+                go run gotest.tools/gotestsum@latest -- ./... \
                   -failfast \
                   -timeout 120m \
                   -parallel 1 \
@@ -76,6 +80,12 @@ jobs:
         with:
           name: logs
           path: "control_logs/*.log"
+
+      - uses: actions/upload-artifact@v3
+        if: always() && steps.changed-files.outputs.any_changed == 'true'
+        with:
+          name: pprof
+          path: "control_logs/*.pprof.tar"
 `),
 	)
 )

cmd/headscale/cli/api_key.go

@@ -5,8 +5,8 @@ import (
 	"strconv"
 	"time"
 
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/prometheus/common/model"
 	"github.com/pterm/pterm"
 	"github.com/rs/zerolog/log"
@@ -67,7 +67,7 @@ var listAPIKeys = &cobra.Command{
 		}
 
 		if output != "" {
-			SuccessOutput(response.ApiKeys, "", output)
+			SuccessOutput(response.GetApiKeys(), "", output)
 
 			return
 		}
@@ -75,15 +75,15 @@ var listAPIKeys = &cobra.Command{
 		tableData := pterm.TableData{
 			{"ID", "Prefix", "Expiration", "Created"},
 		}
-		for _, key := range response.ApiKeys {
+		for _, key := range response.GetApiKeys() {
 			expiration := "-"
 
 			if key.GetExpiration() != nil {
-				expiration = ColourTime(key.Expiration.AsTime())
+				expiration = ColourTime(key.GetExpiration().AsTime())
 			}
 
 			tableData = append(tableData, []string{
-				strconv.FormatUint(key.GetId(), headscale.Base10),
+				strconv.FormatUint(key.GetId(), util.Base10),
 				key.GetPrefix(),
 				expiration,
 				key.GetCreatedAt().AsTime().Format(HeadscaleDateTimeFormat),
@@ -155,7 +155,7 @@ If you loose a key, create a new one and revoke (expire) the old one.`,
 			return
 		}
 
-		SuccessOutput(response.ApiKey, response.ApiKey, output)
+		SuccessOutput(response.GetApiKey(), response.GetApiKey(), output)
 	},
 }
 

cmd/headscale/cli/debug.go

@@ -3,11 +3,11 @@ package cli
 import (
 	"fmt"
 
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
+	"tailscale.com/types/key"
 )
 
 const (
@@ -57,7 +57,7 @@ var debugCmd = &cobra.Command{
 
 var createNodeCmd = &cobra.Command{
 	Use:   "create-node",
-	Short: "Create a node (machine) that can be registered with `nodes register <>` command",
+	Short: "Create a node that can be registered with `nodes register <>` command",
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
@@ -93,11 +93,13 @@ var createNodeCmd = &cobra.Command{
 
 			return
 		}
-		if !headscale.NodePublicKeyRegex.Match([]byte(machineKey)) {
-			err = errPreAuthKeyMalformed
+
+		var mkey key.MachinePublic
+		err = mkey.UnmarshalText([]byte(machineKey))
+		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", err),
+				fmt.Sprintf("Failed to parse machine key from flag: %s", err),
 				output,
 			)
 
@@ -115,24 +117,24 @@ var createNodeCmd = &cobra.Command{
 			return
 		}
 
-		request := &v1.DebugCreateMachineRequest{
+		request := &v1.DebugCreateNodeRequest{
 			Key:    machineKey,
 			Name:   name,
 			User:   user,
 			Routes: routes,
 		}
 
-		response, err := client.DebugCreateMachine(ctx, request)
+		response, err := client.DebugCreateNode(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot create machine: %s", status.Convert(err).Message()),
+				fmt.Sprintf("Cannot create node: %s", status.Convert(err).Message()),
 				output,
 			)
 
 			return
 		}
 
-		SuccessOutput(response.Machine, "Machine created", output)
+		SuccessOutput(response.GetNode(), "Node created", output)
 	},
 }

cmd/headscale/cli/nodes.go

@@ -9,8 +9,8 @@ import (
 	"time"
 
 	survey "github.com/AlecAivazis/survey/v2"
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/pterm/pterm"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
@@ -107,7 +107,7 @@ var nodeCmd = &cobra.Command{
 
 var registerNodeCmd = &cobra.Command{
 	Use:   "register",
-	Short: "Registers a machine to your network",
+	Short: "Registers a node to your network",
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 		user, err := cmd.Flags().GetString("user")
@@ -132,17 +132,17 @@ var registerNodeCmd = &cobra.Command{
 			return
 		}
 
-		request := &v1.RegisterMachineRequest{
+		request := &v1.RegisterNodeRequest{
 			Key:  machineKey,
 			User: user,
 		}
 
-		response, err := client.RegisterMachine(ctx, request)
+		response, err := client.RegisterNode(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf(
-					"Cannot register machine: %s\n",
+					"Cannot register node: %s\n",
 					status.Convert(err).Message(),
 				),
 				output,
@@ -152,8 +152,8 @@ var registerNodeCmd = &cobra.Command{
 		}
 
 		SuccessOutput(
-			response.Machine,
-			fmt.Sprintf("Machine %s registered", response.Machine.GivenName), output)
+			response.GetNode(),
+			fmt.Sprintf("Node %s registered", response.GetNode().GetGivenName()), output)
 	},
 }
 
@@ -180,11 +180,11 @@ var listNodesCmd = &cobra.Command{
 		defer cancel()
 		defer conn.Close()
 
-		request := &v1.ListMachinesRequest{
+		request := &v1.ListNodesRequest{
 			User: user,
 		}
 
-		response, err := client.ListMachines(ctx, request)
+		response, err := client.ListNodes(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
@@ -196,12 +196,12 @@ var listNodesCmd = &cobra.Command{
 		}
 
 		if output != "" {
-			SuccessOutput(response.Machines, "", output)
+			SuccessOutput(response.GetNodes(), "", output)
 
 			return
 		}
 
-		tableData, err := nodesToPtables(user, showTags, response.Machines)
+		tableData, err := nodesToPtables(user, showTags, response.GetNodes())
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error converting to table: %s", err), output)
 
@@ -223,7 +223,7 @@ var listNodesCmd = &cobra.Command{
 
 var expireNodeCmd = &cobra.Command{
 	Use:     "expire",
-	Short:   "Expire (log out) a machine in your network",
+	Short:   "Expire (log out) a node in your network",
 	Long:    "Expiring a node will keep the node in the database and force it to reauthenticate.",
 	Aliases: []string{"logout", "exp", "e"},
 	Run: func(cmd *cobra.Command, args []string) {
@@ -244,16 +244,16 @@ var expireNodeCmd = &cobra.Command{
 		defer cancel()
 		defer conn.Close()
 
-		request := &v1.ExpireMachineRequest{
-			MachineId: identifier,
+		request := &v1.ExpireNodeRequest{
+			NodeId: identifier,
 		}
 
-		response, err := client.ExpireMachine(ctx, request)
+		response, err := client.ExpireNode(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf(
-					"Cannot expire machine: %s\n",
+					"Cannot expire node: %s\n",
 					status.Convert(err).Message(),
 				),
 				output,
@@ -262,13 +262,13 @@ var expireNodeCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(response.Machine, "Machine expired", output)
+		SuccessOutput(response.GetNode(), "Node expired", output)
 	},
 }
 
 var renameNodeCmd = &cobra.Command{
 	Use:   "rename NEW_NAME",
-	Short: "Renames a machine in your network",
+	Short: "Renames a node in your network",
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
@@ -291,17 +291,17 @@ var renameNodeCmd = &cobra.Command{
 		if len(args) > 0 {
 			newName = args[0]
 		}
-		request := &v1.RenameMachineRequest{
-			MachineId: identifier,
-			NewName:   newName,
+		request := &v1.RenameNodeRequest{
+			NodeId:  identifier,
+			NewName: newName,
 		}
 
-		response, err := client.RenameMachine(ctx, request)
+		response, err := client.RenameNode(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf(
-					"Cannot rename machine: %s\n",
+					"Cannot rename node: %s\n",
 					status.Convert(err).Message(),
 				),
 				output,
@@ -310,7 +310,7 @@ var renameNodeCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(response.Machine, "Machine renamed", output)
+		SuccessOutput(response.GetNode(), "Node renamed", output)
 	},
 }
 
@@ -336,11 +336,11 @@ var deleteNodeCmd = &cobra.Command{
 		defer cancel()
 		defer conn.Close()
 
-		getRequest := &v1.GetMachineRequest{
-			MachineId: identifier,
+		getRequest := &v1.GetNodeRequest{
+			NodeId: identifier,
 		}
 
-		getResponse, err := client.GetMachine(ctx, getRequest)
+		getResponse, err := client.GetNode(ctx, getRequest)
 		if err != nil {
 			ErrorOutput(
 				err,
@@ -354,8 +354,8 @@ var deleteNodeCmd = &cobra.Command{
 			return
 		}
 
-		deleteRequest := &v1.DeleteMachineRequest{
-			MachineId: identifier,
+		deleteRequest := &v1.DeleteNodeRequest{
+			NodeId: identifier,
 		}
 
 		confirm := false
@@ -364,7 +364,7 @@ var deleteNodeCmd = &cobra.Command{
 			prompt := &survey.Confirm{
 				Message: fmt.Sprintf(
 					"Do you want to remove the node %s?",
-					getResponse.GetMachine().Name,
+					getResponse.GetNode().GetName(),
 				),
 			}
 			err = survey.AskOne(prompt, &confirm)
@@ -374,7 +374,7 @@ var deleteNodeCmd = &cobra.Command{
 		}
 
 		if confirm || force {
-			response, err := client.DeleteMachine(ctx, deleteRequest)
+			response, err := client.DeleteNode(ctx, deleteRequest)
 			if output != "" {
 				SuccessOutput(response, "", output)
 
@@ -436,11 +436,11 @@ var moveNodeCmd = &cobra.Command{
 		defer cancel()
 		defer conn.Close()
 
-		getRequest := &v1.GetMachineRequest{
-			MachineId: identifier,
+		getRequest := &v1.GetNodeRequest{
+			NodeId: identifier,
 		}
 
-		_, err = client.GetMachine(ctx, getRequest)
+		_, err = client.GetNode(ctx, getRequest)
 		if err != nil {
 			ErrorOutput(
 				err,
@@ -454,12 +454,12 @@ var moveNodeCmd = &cobra.Command{
 			return
 		}
 
-		moveRequest := &v1.MoveMachineRequest{
-			MachineId: identifier,
-			User:      user,
+		moveRequest := &v1.MoveNodeRequest{
+			NodeId: identifier,
+			User:   user,
 		}
 
-		moveResponse, err := client.MoveMachine(ctx, moveRequest)
+		moveResponse, err := client.MoveNode(ctx, moveRequest)
 		if err != nil {
 			ErrorOutput(
 				err,
@@ -473,14 +473,14 @@ var moveNodeCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(moveResponse.Machine, "Node moved to another user", output)
+		SuccessOutput(moveResponse.GetNode(), "Node moved to another user", output)
 	},
 }
 
 func nodesToPtables(
 	currentUser string,
 	showTags bool,
-	machines []*v1.Machine,
+	nodes []*v1.Node,
 ) (pterm.TableData, error) {
 	tableHeader := []string{
 		"ID",
@@ -493,7 +493,7 @@ func nodesToPtables(
 		"Ephemeral",
 		"Last seen",
 		"Expiration",
-		"Online",
+		"Connected",
 		"Expired",
 	}
 	if showTags {
@@ -505,23 +505,23 @@ func nodesToPtables(
 	}
 	tableData := pterm.TableData{tableHeader}
 
-	for _, machine := range machines {
+	for _, node := range nodes {
 		var ephemeral bool
-		if machine.PreAuthKey != nil && machine.PreAuthKey.Ephemeral {
+		if node.GetPreAuthKey() != nil && node.GetPreAuthKey().GetEphemeral() {
 			ephemeral = true
 		}
 
 		var lastSeen time.Time
 		var lastSeenTime string
-		if machine.LastSeen != nil {
-			lastSeen = machine.LastSeen.AsTime()
+		if node.GetLastSeen() != nil {
+			lastSeen = node.GetLastSeen().AsTime()
 			lastSeenTime = lastSeen.Format("2006-01-02 15:04:05")
 		}
 
 		var expiry time.Time
 		var expiryTime string
-		if machine.Expiry != nil {
-			expiry = machine.Expiry.AsTime()
+		if node.GetExpiry() != nil {
+			expiry = node.GetExpiry().AsTime()
 			expiryTime = expiry.Format("2006-01-02 15:04:05")
 		} else {
 			expiryTime = "N/A"
@@ -529,7 +529,7 @@ func nodesToPtables(
 
 		var machineKey key.MachinePublic
 		err := machineKey.UnmarshalText(
-			[]byte(headscale.MachinePublicKeyEnsurePrefix(machine.MachineKey)),
+			[]byte(node.GetMachineKey()),
 		)
 		if err != nil {
 			machineKey = key.MachinePublic{}
@@ -537,14 +537,14 @@ func nodesToPtables(
 
 		var nodeKey key.NodePublic
 		err = nodeKey.UnmarshalText(
-			[]byte(headscale.NodePublicKeyEnsurePrefix(machine.NodeKey)),
+			[]byte(node.GetNodeKey()),
 		)
 		if err != nil {
 			return nil, err
 		}
 
 		var online string
-		if machine.Online {
+		if node.GetOnline() {
 			online = pterm.LightGreen("online")
 		} else {
 			online = pterm.LightRed("offline")
@@ -558,36 +558,36 @@ func nodesToPtables(
 		}
 
 		var forcedTags string
-		for _, tag := range machine.ForcedTags {
+		for _, tag := range node.GetForcedTags() {
 			forcedTags += "," + tag
 		}
 		forcedTags = strings.TrimLeft(forcedTags, ",")
 		var invalidTags string
-		for _, tag := range machine.InvalidTags {
-			if !contains(machine.ForcedTags, tag) {
+		for _, tag := range node.GetInvalidTags() {
+			if !contains(node.GetForcedTags(), tag) {
 				invalidTags += "," + pterm.LightRed(tag)
 			}
 		}
 		invalidTags = strings.TrimLeft(invalidTags, ",")
 		var validTags string
-		for _, tag := range machine.ValidTags {
-			if !contains(machine.ForcedTags, tag) {
+		for _, tag := range node.GetValidTags() {
+			if !contains(node.GetForcedTags(), tag) {
 				validTags += "," + pterm.LightGreen(tag)
 			}
 		}
 		validTags = strings.TrimLeft(validTags, ",")
 
 		var user string
-		if currentUser == "" || (currentUser == machine.User.Name) {
-			user = pterm.LightMagenta(machine.User.Name)
+		if currentUser == "" || (currentUser == node.GetUser().GetName()) {
+			user = pterm.LightMagenta(node.GetUser().GetName())
 		} else {
 			// Shared into this user
-			user = pterm.LightYellow(machine.User.Name)
+			user = pterm.LightYellow(node.GetUser().GetName())
 		}
 
 		var IPV4Address string
 		var IPV6Address string
-		for _, addr := range machine.IpAddresses {
+		for _, addr := range node.GetIpAddresses() {
 			if netip.MustParseAddr(addr).Is4() {
 				IPV4Address = addr
 			} else {
@@ -596,9 +596,9 @@ func nodesToPtables(
 		}
 
 		nodeData := []string{
-			strconv.FormatUint(machine.Id, headscale.Base10),
-			machine.Name,
-			machine.GetGivenName(),
+			strconv.FormatUint(node.GetId(), util.Base10),
+			node.GetName(),
+			node.GetGivenName(),
 			machineKey.ShortString(),
 			nodeKey.ShortString(),
 			user,
@@ -646,17 +646,17 @@ var tagCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error retrieving list of tags to add to machine, %v", err),
+				fmt.Sprintf("Error retrieving list of tags to add to node, %v", err),
 				output,
 			)
 
 			return
 		}
 
-		// Sending tags to machine
+		// Sending tags to node
 		request := &v1.SetTagsRequest{
-			MachineId: identifier,
-			Tags:      tagsToSet,
+			NodeId: identifier,
+			Tags:   tagsToSet,
 		}
 		resp, err := client.SetTags(ctx, request)
 		if err != nil {
@@ -671,8 +671,8 @@ var tagCmd = &cobra.Command{
 
 		if resp != nil {
 			SuccessOutput(
-				resp.GetMachine(),
-				"Machine updated",
+				resp.GetNode(),
+				"Node updated",
 				output,
 			)
 		}

cmd/headscale/cli/preauthkeys.go

@@ -84,7 +84,7 @@ var listPreAuthKeys = &cobra.Command{
 		}
 
 		if output != "" {
-			SuccessOutput(response.PreAuthKeys, "", output)
+			SuccessOutput(response.GetPreAuthKeys(), "", output)
 
 			return
 		}
@@ -101,10 +101,10 @@ var listPreAuthKeys = &cobra.Command{
 				"Tags",
 			},
 		}
-		for _, key := range response.PreAuthKeys {
+		for _, key := range response.GetPreAuthKeys() {
 			expiration := "-"
 			if key.GetExpiration() != nil {
-				expiration = ColourTime(key.Expiration.AsTime())
+				expiration = ColourTime(key.GetExpiration().AsTime())
 			}
 
 			var reusable string
@@ -116,7 +116,7 @@ var listPreAuthKeys = &cobra.Command{
 
 			aclTags := ""
 
-			for _, tag := range key.AclTags {
+			for _, tag := range key.GetAclTags() {
 				aclTags += "," + tag
 			}
 
@@ -214,7 +214,7 @@ var createPreAuthKeyCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(response.PreAuthKey, response.PreAuthKey.Key, output)
+		SuccessOutput(response.GetPreAuthKey(), response.GetPreAuthKey().GetKey(), output)
 	},
 }
 

cmd/headscale/cli/root.go

@@ -5,7 +5,7 @@ import (
 	"os"
 	"runtime"
 
-	"github.com/juanfont/headscale"
+	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/rs/zerolog"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
@@ -38,33 +38,33 @@ func initConfig() {
 		cfgFile = os.Getenv("HEADSCALE_CONFIG")
 	}
 	if cfgFile != "" {
-		err := headscale.LoadConfig(cfgFile, true)
+		err := types.LoadConfig(cfgFile, true)
 		if err != nil {
 			log.Fatal().Caller().Err(err).Msgf("Error loading config file %s", cfgFile)
 		}
 	} else {
-		err := headscale.LoadConfig("", false)
+		err := types.LoadConfig("", false)
 		if err != nil {
 			log.Fatal().Caller().Err(err).Msgf("Error loading config")
 		}
 	}
 
-	cfg, err := headscale.GetHeadscaleConfig()
+	cfg, err := types.GetHeadscaleConfig()
 	if err != nil {
-		log.Fatal().Caller().Err(err)
+		log.Fatal().Caller().Err(err).Msg("Failed to get headscale configuration")
 	}
 
 	machineOutput := HasMachineOutputFlag()
 
 	zerolog.SetGlobalLevel(cfg.Log.Level)
 
-	// If the user has requested a "machine" readable format,
+	// If the user has requested a "node" readable format,
 	// then disable login so the output remains valid.
 	if machineOutput {
 		zerolog.SetGlobalLevel(zerolog.Disabled)
 	}
 
-	if cfg.Log.Format == headscale.JSONLogFormat {
+	if cfg.Log.Format == types.JSONLogFormat {
 		log.Logger = log.Output(os.Stdout)
 	}
 

cmd/headscale/cli/routes.go

@@ -6,8 +6,8 @@ import (
 	"net/netip"
 	"strconv"
 
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/pterm/pterm"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
@@ -87,20 +87,20 @@ var listRoutesCmd = &cobra.Command{
 			}
 
 			if output != "" {
-				SuccessOutput(response.Routes, "", output)
+				SuccessOutput(response.GetRoutes(), "", output)
 
 				return
 			}
 
-			routes = response.Routes
+			routes = response.GetRoutes()
 		} else {
-			response, err := client.GetMachineRoutes(ctx, &v1.GetMachineRoutesRequest{
-				MachineId: machineID,
+			response, err := client.GetNodeRoutes(ctx, &v1.GetNodeRoutesRequest{
+				NodeId: machineID,
 			})
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf("Cannot get routes for machine %d: %s", machineID, status.Convert(err).Message()),
+					fmt.Sprintf("Cannot get routes for node %d: %s", machineID, status.Convert(err).Message()),
 					output,
 				)
 
@@ -108,12 +108,12 @@ var listRoutesCmd = &cobra.Command{
 			}
 
 			if output != "" {
-				SuccessOutput(response.Routes, "", output)
+				SuccessOutput(response.GetRoutes(), "", output)
 
 				return
 			}
 
-			routes = response.Routes
+			routes = response.GetRoutes()
 		}
 
 		tableData := routesToPtables(routes)
@@ -267,29 +267,29 @@ var deleteRouteCmd = &cobra.Command{
 
 // routesToPtables converts the list of routes to a nice table.
 func routesToPtables(routes []*v1.Route) pterm.TableData {
-	tableData := pterm.TableData{{"ID", "Machine", "Prefix", "Advertised", "Enabled", "Primary"}}
+	tableData := pterm.TableData{{"ID", "Node", "Prefix", "Advertised", "Enabled", "Primary"}}
 
 	for _, route := range routes {
 		var isPrimaryStr string
-		prefix, err := netip.ParsePrefix(route.Prefix)
+		prefix, err := netip.ParsePrefix(route.GetPrefix())
 		if err != nil {
-			log.Printf("Error parsing prefix %s: %s", route.Prefix, err)
+			log.Printf("Error parsing prefix %s: %s", route.GetPrefix(), err)
 
 			continue
 		}
-		if prefix == headscale.ExitRouteV4 || prefix == headscale.ExitRouteV6 {
+		if prefix == types.ExitRouteV4 || prefix == types.ExitRouteV6 {
 			isPrimaryStr = "-"
 		} else {
-			isPrimaryStr = strconv.FormatBool(route.IsPrimary)
+			isPrimaryStr = strconv.FormatBool(route.GetIsPrimary())
 		}
 
 		tableData = append(tableData,
 			[]string{
-				strconv.FormatUint(route.Id, Base10),
-				route.Machine.GivenName,
-				route.Prefix,
-				strconv.FormatBool(route.Advertised),
-				strconv.FormatBool(route.Enabled),
+				strconv.FormatUint(route.GetId(), Base10),
+				route.GetNode().GetGivenName(),
+				route.GetPrefix(),
+				strconv.FormatBool(route.GetAdvertised()),
+				strconv.FormatBool(route.GetEnabled()),
 				isPrimaryStr,
 			})
 	}

cmd/headscale/cli/users.go

@@ -1,10 +1,10 @@
 package cli
 
 import (
+	"errors"
 	"fmt"
 
 	survey "github.com/AlecAivazis/survey/v2"
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/pterm/pterm"
 	"github.com/rs/zerolog/log"
@@ -20,9 +20,7 @@ func init() {
 	userCmd.AddCommand(renameUserCmd)
 }
 
-const (
-	errMissingParameter = headscale.Error("missing parameters")
-)
+var errMissingParameter = errors.New("missing parameters")
 
 var userCmd = &cobra.Command{
 	Use:     "users",
@@ -69,7 +67,7 @@ var createUserCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(response.User, "User created", output)
+		SuccessOutput(response.GetUser(), "User created", output)
 	},
 }
 
@@ -171,7 +169,7 @@ var listUsersCmd = &cobra.Command{
 		}
 
 		if output != "" {
-			SuccessOutput(response.Users, "", output)
+			SuccessOutput(response.GetUsers(), "", output)
 
 			return
 		}
@@ -238,6 +236,6 @@ var renameUserCmd = &cobra.Command{
 			return
 		}
 
-		SuccessOutput(response.User, "User renamed", output)
+		SuccessOutput(response.GetUser(), "User renamed", output)
 	},
 }

cmd/headscale/cli/utils.go

@@ -8,8 +8,11 @@ import (
 	"os"
 	"reflect"
 
-	"github.com/juanfont/headscale"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol"
+	"github.com/juanfont/headscale/hscontrol/policy"
+	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/rs/zerolog/log"
 	"google.golang.org/grpc"
 	"google.golang.org/grpc/credentials"
@@ -22,8 +25,8 @@ const (
 	SocketWritePermissions  = 0o666
 )
 
-func getHeadscaleApp() (*headscale.Headscale, error) {
-	cfg, err := headscale.GetHeadscaleConfig()
+func getHeadscaleApp() (*hscontrol.Headscale, error) {
+	cfg, err := types.GetHeadscaleConfig()
 	if err != nil {
 		return nil, fmt.Errorf(
 			"failed to load configuration while creating headscale instance: %w",
@@ -31,7 +34,7 @@ func getHeadscaleApp() (*headscale.Headscale, error) {
 		)
 	}
 
-	app, err := headscale.NewHeadscale(cfg)
+	app, err := hscontrol.NewHeadscale(cfg)
 	if err != nil {
 		return nil, err
 	}
@@ -39,21 +42,23 @@ func getHeadscaleApp() (*headscale.Headscale, error) {
 	// We are doing this here, as in the future could be cool to have it also hot-reload
 
 	if cfg.ACL.PolicyPath != "" {
-		aclPath := headscale.AbsolutePathFromConfigPath(cfg.ACL.PolicyPath)
-		err = app.LoadACLPolicy(aclPath)
+		aclPath := util.AbsolutePathFromConfigPath(cfg.ACL.PolicyPath)
+		pol, err := policy.LoadACLPolicyFromPath(aclPath)
 		if err != nil {
 			log.Fatal().
 				Str("path", aclPath).
 				Err(err).
 				Msg("Could not load the ACL policy")
 		}
+
+		app.ACLPolicy = pol
 	}
 
 	return app, nil
 }
 
 func getHeadscaleCLIClient() (context.Context, v1.HeadscaleServiceClient, *grpc.ClientConn, context.CancelFunc) {
-	cfg, err := headscale.GetHeadscaleConfig()
+	cfg, err := types.GetHeadscaleConfig()
 	if err != nil {
 		log.Fatal().
 			Err(err).
@@ -74,7 +79,7 @@ func getHeadscaleCLIClient() (context.Context, v1.HeadscaleServiceClient, *grpc.
 
 	address := cfg.CLI.Address
 
-	// If the address is not set, we assume that we are on the server hosting headscale.
+	// If the address is not set, we assume that we are on the server hosting hscontrol.
 	if address == "" {
 		log.Debug().
 			Str("socket", cfg.UnixSocket).
@@ -98,7 +103,7 @@ func getHeadscaleCLIClient() (context.Context, v1.HeadscaleServiceClient, *grpc.
 		grpcOptions = append(
 			grpcOptions,
 			grpc.WithTransportCredentials(insecure.NewCredentials()),
-			grpc.WithContextDialer(headscale.GrpcSocketDialer),
+			grpc.WithContextDialer(util.GrpcSocketDialer),
 		)
 	} else {
 		// If we are not connecting to a local server, require an API key for authentication
@@ -149,17 +154,17 @@ func SuccessOutput(result interface{}, override string, outputFormat string) {
 	case "json":
 		jsonBytes, err = json.MarshalIndent(result, "", "\t")
 		if err != nil {
-			log.Fatal().Err(err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
 	case "json-line":
 		jsonBytes, err = json.Marshal(result)
 		if err != nil {
-			log.Fatal().Err(err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
 	case "yaml":
 		jsonBytes, err = yaml.Marshal(result)
 		if err != nil {
-			log.Fatal().Err(err)
+			log.Fatal().Err(err).Msg("failed to unmarshal output")
 		}
 	default:
 		//nolint

cmd/headscale/headscale.go

@@ -6,11 +6,25 @@ import (
 
 	"github.com/efekarakus/termcolor"
 	"github.com/juanfont/headscale/cmd/headscale/cli"
+	"github.com/pkg/profile"
 	"github.com/rs/zerolog"
 	"github.com/rs/zerolog/log"
 )
 
 func main() {
+	if _, enableProfile := os.LookupEnv("HEADSCALE_PROFILING_ENABLED"); enableProfile {
+		if profilePath, ok := os.LookupEnv("HEADSCALE_PROFILING_PATH"); ok {
+			err := os.MkdirAll(profilePath, os.ModePerm)
+			if err != nil {
+				log.Fatal().Err(err).Msg("failed to create profiling directory")
+			}
+
+			defer profile.Start(profile.ProfilePath(profilePath)).Stop()
+		} else {
+			defer profile.Start().Stop()
+		}
+	}
+
 	var colors bool
 	switch l := termcolor.SupportLevel(os.Stderr); l {
 	case termcolor.Level16M:
@@ -34,7 +48,7 @@ func main() {
 
 	zerolog.TimeFieldFormat = zerolog.TimeFormatUnix
 	log.Logger = log.Output(zerolog.ConsoleWriter{
-		Out:        os.Stdout,
+		Out:        os.Stderr,
 		TimeFormat: time.RFC3339,
 		NoColor:    !colors,
 	})

cmd/headscale/headscale_test.go

@@ -7,7 +7,8 @@ import (
 	"strings"
 	"testing"
 
-	"github.com/juanfont/headscale"
+	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/spf13/viper"
 	"gopkg.in/check.v1"
 )
@@ -50,7 +51,7 @@ func (*Suite) TestConfigFileLoading(c *check.C) {
 	}
 
 	// Load example config, it should load without validation errors
-	err = headscale.LoadConfig(cfgFile, true)
+	err = types.LoadConfig(cfgFile, true)
 	c.Assert(err, check.IsNil)
 
 	// Test that config file was interpreted correctly
@@ -58,13 +59,13 @@ func (*Suite) TestConfigFileLoading(c *check.C) {
 	c.Assert(viper.GetString("listen_addr"), check.Equals, "127.0.0.1:8080")
 	c.Assert(viper.GetString("metrics_listen_addr"), check.Equals, "127.0.0.1:9090")
 	c.Assert(viper.GetString("db_type"), check.Equals, "sqlite3")
-	c.Assert(viper.GetString("db_path"), check.Equals, "./db.sqlite")
+	c.Assert(viper.GetString("db_path"), check.Equals, "/var/lib/headscale/db.sqlite")
 	c.Assert(viper.GetString("tls_letsencrypt_hostname"), check.Equals, "")
 	c.Assert(viper.GetString("tls_letsencrypt_listen"), check.Equals, ":http")
 	c.Assert(viper.GetString("tls_letsencrypt_challenge_type"), check.Equals, "HTTP-01")
 	c.Assert(viper.GetStringSlice("dns_config.nameservers")[0], check.Equals, "1.1.1.1")
 	c.Assert(
-		headscale.GetFileMode("unix_socket_permission"),
+		util.GetFileMode("unix_socket_permission"),
 		check.Equals,
 		fs.FileMode(0o770),
 	)
@@ -93,7 +94,7 @@ func (*Suite) TestConfigLoading(c *check.C) {
 	}
 
 	// Load example config, it should load without validation errors
-	err = headscale.LoadConfig(tmpDir, false)
+	err = types.LoadConfig(tmpDir, false)
 	c.Assert(err, check.IsNil)
 
 	// Test that config file was interpreted correctly
@@ -101,13 +102,13 @@ func (*Suite) TestConfigLoading(c *check.C) {
 	c.Assert(viper.GetString("listen_addr"), check.Equals, "127.0.0.1:8080")
 	c.Assert(viper.GetString("metrics_listen_addr"), check.Equals, "127.0.0.1:9090")
 	c.Assert(viper.GetString("db_type"), check.Equals, "sqlite3")
-	c.Assert(viper.GetString("db_path"), check.Equals, "./db.sqlite")
+	c.Assert(viper.GetString("db_path"), check.Equals, "/var/lib/headscale/db.sqlite")
 	c.Assert(viper.GetString("tls_letsencrypt_hostname"), check.Equals, "")
 	c.Assert(viper.GetString("tls_letsencrypt_listen"), check.Equals, ":http")
 	c.Assert(viper.GetString("tls_letsencrypt_challenge_type"), check.Equals, "HTTP-01")
 	c.Assert(viper.GetStringSlice("dns_config.nameservers")[0], check.Equals, "1.1.1.1")
 	c.Assert(
-		headscale.GetFileMode("unix_socket_permission"),
+		util.GetFileMode("unix_socket_permission"),
 		check.Equals,
 		fs.FileMode(0o770),
 	)
@@ -137,10 +138,10 @@ func (*Suite) TestDNSConfigLoading(c *check.C) {
 	}
 
 	// Load example config, it should load without validation errors
-	err = headscale.LoadConfig(tmpDir, false)
+	err = types.LoadConfig(tmpDir, false)
 	c.Assert(err, check.IsNil)
 
-	dnsConfig, baseDomain := headscale.GetDNSConfig()
+	dnsConfig, baseDomain := types.GetDNSConfig()
 
 	c.Assert(dnsConfig.Nameservers[0].String(), check.Equals, "1.1.1.1")
 	c.Assert(dnsConfig.Resolvers[0].Addr, check.Equals, "1.1.1.1")
@@ -172,7 +173,7 @@ noise:
 	writeConfig(c, tmpDir, configYaml)
 
 	// Check configuration validation errors (1)
-	err = headscale.LoadConfig(tmpDir, false)
+	err = types.LoadConfig(tmpDir, false)
 	c.Assert(err, check.NotNil)
 	// check.Matches can not handle multiline strings
 	tmp := strings.ReplaceAll(err.Error(), "\n", "***")
@@ -201,6 +202,6 @@ tls_letsencrypt_hostname: example.com
 tls_letsencrypt_challenge_type: TLS-ALPN-01
 `)
 	writeConfig(c, tmpDir, configYaml)
-	err = headscale.LoadConfig(tmpDir, false)
+	err = types.LoadConfig(tmpDir, false)
 	c.Assert(err, check.IsNil)
 }

config-example.yaml

@@ -40,34 +40,23 @@ grpc_listen_addr: 127.0.0.1:50443
 # are doing.
 grpc_allow_insecure: false
 
-# Private key used to encrypt the traffic between headscale
-# and Tailscale clients.
-# The private key file will be autogenerated if it's missing.
-#
-# For production:
-# /var/lib/headscale/private.key
-private_key_path: ./private.key
-
 # The Noise section includes specific configuration for the
 # TS2021 Noise protocol
 noise:
   # The Noise private key is used to encrypt the
   # traffic between headscale and Tailscale clients when
-  # using the new Noise-based protocol. It must be different
-  # from the legacy private key.
-  #
-  # For production:
-  # private_key_path: /var/lib/headscale/noise_private.key
-  private_key_path: ./noise_private.key
+  # using the new Noise-based protocol.
+  private_key_path: /var/lib/headscale/noise_private.key
 
 # List of IP prefixes to allocate tailaddresses from.
 # Each prefix consists of either an IPv4 or IPv6 address,
 # and the associated prefix length, delimited by a slash.
-# While this looks like it can take arbitrary values, it
-# needs to be within IP ranges supported by the Tailscale
-# client.
+# It must be within IP ranges supported by the Tailscale
+# client - i.e., subnets of 100.64.0.0/10 and fd7a:115c:a1e0::/48.
+# See below:
 # IPv6: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#LL81C52-L81C71
 # IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33
+# Any other range is NOT supported, and it will cause unexpected issues.
 ip_prefixes:
   - fd7a:115c:a1e0::/48
   - 100.64.0.0/10
@@ -99,6 +88,22 @@ derp:
     # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/
     stun_listen_addr: "0.0.0.0:3478"
 
+    # Private key used to encrypt the traffic between headscale DERP
+    # and Tailscale clients.
+    # The private key file will be autogenerated if it's missing.
+    #
+    private_key_path: /var/lib/headscale/derp_server_private.key
+
+    # This flag can be used, so the DERP map entry for the embedded DERP server is not written automatically,
+    # it enables the creation of your very own DERP map entry using a locally available file with the parameter DERP.paths
+    # If you enable the DERP server and set this to false, it is required to add the DERP server to the DERP map using DERP.paths
+    automatically_add_embedded_derp_region: true
+
+    # For better connection stability (especially when using an Exit-Node and DNS is not working),
+    # it is possible to optionall add the public IPv4 and IPv6 address to the Derp-Map using:
+    ipv4: 1.2.3.4
+    ipv6: 2001:db8::1
+
   # List of externally available DERP maps encoded in JSON
   urls:
     - https://controlplane.tailscale.com/derpmap/default
@@ -137,8 +142,7 @@ node_update_check_interval: 10s
 db_type: sqlite3
 
 # For production:
-# db_path: /var/lib/headscale/db.sqlite
-db_path: ./db.sqlite
+db_path: /var/lib/headscale/db.sqlite
 
 # # Postgres config
 # If using a Unix socket to connect to Postgres, set the socket path in the 'host' field and leave 'port' blank.
@@ -172,8 +176,7 @@ tls_letsencrypt_hostname: ""
 # Path to store certificates and metadata needed by
 # letsencrypt
 # For production:
-# tls_letsencrypt_cache_dir: /var/lib/headscale/cache
-tls_letsencrypt_cache_dir: ./cache
+tls_letsencrypt_cache_dir: /var/lib/headscale/cache
 
 # Type of ACME challenge to use, currently supported types:
 # HTTP-01 or TLS-ALPN-01
@@ -263,8 +266,7 @@ dns_config:
 
 # Unix socket used for the CLI to connect without authentication
 # Note: for production you will want to set this to something like:
-# unix_socket: /var/run/headscale.sock
-unix_socket: ./headscale.sock
+unix_socket: /var/run/headscale/headscale.sock
 unix_socket_permission: "0770"
 #
 # headscale supports experimental OpenID connect support,

db.go

@@ -1,404 +0,0 @@
-package headscale
-
-import (
-	"context"
-	"database/sql/driver"
-	"encoding/json"
-	"errors"
-	"fmt"
-	"net/netip"
-	"time"
-
-	"github.com/glebarez/sqlite"
-	"github.com/rs/zerolog/log"
-	"gorm.io/driver/postgres"
-	"gorm.io/gorm"
-	"gorm.io/gorm/logger"
-	"tailscale.com/tailcfg"
-)
-
-const (
-	dbVersion = "1"
-
-	errValueNotFound     = Error("not found")
-	ErrCannotParsePrefix = Error("cannot parse prefix")
-)
-
-// KV is a key-value store in a psql table. For future use...
-type KV struct {
-	Key   string
-	Value string
-}
-
-func (h *Headscale) initDB() error {
-	db, err := h.openDB()
-	if err != nil {
-		return err
-	}
-	h.db = db
-
-	if h.dbType == Postgres {
-		db.Exec(`create extension if not exists "uuid-ossp";`)
-	}
-
-	_ = db.Migrator().RenameTable("namespaces", "users")
-
-	err = db.AutoMigrate(&User{})
-	if err != nil {
-		return err
-	}
-
-	_ = db.Migrator().RenameColumn(&Machine{}, "namespace_id", "user_id")
-	_ = db.Migrator().RenameColumn(&PreAuthKey{}, "namespace_id", "user_id")
-
-	_ = db.Migrator().RenameColumn(&Machine{}, "ip_address", "ip_addresses")
-	_ = db.Migrator().RenameColumn(&Machine{}, "name", "hostname")
-
-	// GivenName is used as the primary source of DNS names, make sure
-	// the field is populated and normalized if it was not when the
-	// machine was registered.
-	_ = db.Migrator().RenameColumn(&Machine{}, "nickname", "given_name")
-
-	// If the Machine table has a column for registered,
-	// find all occourences of "false" and drop them. Then
-	// remove the column.
-	if db.Migrator().HasColumn(&Machine{}, "registered") {
-		log.Info().
-			Msg(`Database has legacy "registered" column in machine, removing...`)
-
-		machines := Machines{}
-		if err := h.db.Not("registered").Find(&machines).Error; err != nil {
-			log.Error().Err(err).Msg("Error accessing db")
-		}
-
-		for _, machine := range machines {
-			log.Info().
-				Str("machine", machine.Hostname).
-				Str("machine_key", machine.MachineKey).
-				Msg("Deleting unregistered machine")
-			if err := h.db.Delete(&Machine{}, machine.ID).Error; err != nil {
-				log.Error().
-					Err(err).
-					Str("machine", machine.Hostname).
-					Str("machine_key", machine.MachineKey).
-					Msg("Error deleting unregistered machine")
-			}
-		}
-
-		err := db.Migrator().DropColumn(&Machine{}, "registered")
-		if err != nil {
-			log.Error().Err(err).Msg("Error dropping registered column")
-		}
-	}
-
-	err = db.AutoMigrate(&Route{})
-	if err != nil {
-		return err
-	}
-
-	if db.Migrator().HasColumn(&Machine{}, "enabled_routes") {
-		log.Info().Msgf("Database has legacy enabled_routes column in machine, migrating...")
-
-		type MachineAux struct {
-			ID            uint64
-			EnabledRoutes IPPrefixes
-		}
-
-		machinesAux := []MachineAux{}
-		err := db.Table("machines").Select("id, enabled_routes").Scan(&machinesAux).Error
-		if err != nil {
-			log.Fatal().Err(err).Msg("Error accessing db")
-		}
-		for _, machine := range machinesAux {
-			for _, prefix := range machine.EnabledRoutes {
-				if err != nil {
-					log.Error().
-						Err(err).
-						Str("enabled_route", prefix.String()).
-						Msg("Error parsing enabled_route")
-
-					continue
-				}
-
-				err = db.Preload("Machine").
-					Where("machine_id = ? AND prefix = ?", machine.ID, IPPrefix(prefix)).
-					First(&Route{}).
-					Error
-				if err == nil {
-					log.Info().
-						Str("enabled_route", prefix.String()).
-						Msg("Route already migrated to new table, skipping")
-
-					continue
-				}
-
-				route := Route{
-					MachineID:  machine.ID,
-					Advertised: true,
-					Enabled:    true,
-					Prefix:     IPPrefix(prefix),
-				}
-				if err := h.db.Create(&route).Error; err != nil {
-					log.Error().Err(err).Msg("Error creating route")
-				} else {
-					log.Info().
-						Uint64("machine_id", route.MachineID).
-						Str("prefix", prefix.String()).
-						Msg("Route migrated")
-				}
-			}
-		}
-
-		err = db.Migrator().DropColumn(&Machine{}, "enabled_routes")
-		if err != nil {
-			log.Error().Err(err).Msg("Error dropping enabled_routes column")
-		}
-	}
-
-	err = db.AutoMigrate(&Machine{})
-	if err != nil {
-		return err
-	}
-
-	if db.Migrator().HasColumn(&Machine{}, "given_name") {
-		machines := Machines{}
-		if err := h.db.Find(&machines).Error; err != nil {
-			log.Error().Err(err).Msg("Error accessing db")
-		}
-
-		for item, machine := range machines {
-			if machine.GivenName == "" {
-				normalizedHostname, err := NormalizeToFQDNRules(
-					machine.Hostname,
-					h.cfg.OIDC.StripEmaildomain,
-				)
-				if err != nil {
-					log.Error().
-						Caller().
-						Str("hostname", machine.Hostname).
-						Err(err).
-						Msg("Failed to normalize machine hostname in DB migration")
-				}
-
-				err = h.RenameMachine(&machines[item], normalizedHostname)
-				if err != nil {
-					log.Error().
-						Caller().
-						Str("hostname", machine.Hostname).
-						Err(err).
-						Msg("Failed to save normalized machine name in DB migration")
-				}
-			}
-		}
-	}
-
-	err = db.AutoMigrate(&KV{})
-	if err != nil {
-		return err
-	}
-
-	err = db.AutoMigrate(&PreAuthKey{})
-	if err != nil {
-		return err
-	}
-
-	err = db.AutoMigrate(&PreAuthKeyACLTag{})
-	if err != nil {
-		return err
-	}
-
-	_ = db.Migrator().DropTable("shared_machines")
-
-	err = db.AutoMigrate(&APIKey{})
-	if err != nil {
-		return err
-	}
-
-	err = h.setValue("db_version", dbVersion)
-
-	return err
-}
-
-func (h *Headscale) openDB() (*gorm.DB, error) {
-	var db *gorm.DB
-	var err error
-
-	var log logger.Interface
-	if h.dbDebug {
-		log = logger.Default
-	} else {
-		log = logger.Default.LogMode(logger.Silent)
-	}
-
-	switch h.dbType {
-	case Sqlite:
-		db, err = gorm.Open(
-			sqlite.Open(h.dbString+"?_synchronous=1&_journal_mode=WAL"),
-			&gorm.Config{
-				DisableForeignKeyConstraintWhenMigrating: true,
-				Logger:                                   log,
-			},
-		)
-
-		db.Exec("PRAGMA foreign_keys=ON")
-
-		// The pure Go SQLite library does not handle locking in
-		// the same way as the C based one and we cant use the gorm
-		// connection pool as of 2022/02/23.
-		sqlDB, _ := db.DB()
-		sqlDB.SetMaxIdleConns(1)
-		sqlDB.SetMaxOpenConns(1)
-		sqlDB.SetConnMaxIdleTime(time.Hour)
-
-	case Postgres:
-		db, err = gorm.Open(postgres.Open(h.dbString), &gorm.Config{
-			DisableForeignKeyConstraintWhenMigrating: true,
-			Logger:                                   log,
-		})
-	}
-
-	if err != nil {
-		return nil, err
-	}
-
-	return db, nil
-}
-
-// getValue returns the value for the given key in KV.
-func (h *Headscale) getValue(key string) (string, error) {
-	var row KV
-	if result := h.db.First(&row, "key = ?", key); errors.Is(
-		result.Error,
-		gorm.ErrRecordNotFound,
-	) {
-		return "", errValueNotFound
-	}
-
-	return row.Value, nil
-}
-
-// setValue sets value for the given key in KV.
-func (h *Headscale) setValue(key string, value string) error {
-	keyValue := KV{
-		Key:   key,
-		Value: value,
-	}
-
-	if _, err := h.getValue(key); err == nil {
-		h.db.Model(&keyValue).Where("key = ?", key).Update("value", value)
-
-		return nil
-	}
-
-	if err := h.db.Create(keyValue).Error; err != nil {
-		return fmt.Errorf("failed to create key value pair in the database: %w", err)
-	}
-
-	return nil
-}
-
-func (h *Headscale) pingDB(ctx context.Context) error {
-	ctx, cancel := context.WithTimeout(ctx, time.Second)
-	defer cancel()
-	db, err := h.db.DB()
-	if err != nil {
-		return err
-	}
-
-	return db.PingContext(ctx)
-}
-
-// This is a "wrapper" type around tailscales
-// Hostinfo to allow us to add database "serialization"
-// methods. This allows us to use a typed values throughout
-// the code and not have to marshal/unmarshal and error
-// check all over the code.
-type HostInfo tailcfg.Hostinfo
-
-func (hi *HostInfo) Scan(destination interface{}) error {
-	switch value := destination.(type) {
-	case []byte:
-		return json.Unmarshal(value, hi)
-
-	case string:
-		return json.Unmarshal([]byte(value), hi)
-
-	default:
-		return fmt.Errorf("%w: unexpected data type %T", ErrMachineAddressesInvalid, destination)
-	}
-}
-
-// Value return json value, implement driver.Valuer interface.
-func (hi HostInfo) Value() (driver.Value, error) {
-	bytes, err := json.Marshal(hi)
-
-	return string(bytes), err
-}
-
-type IPPrefix netip.Prefix
-
-func (i *IPPrefix) Scan(destination interface{}) error {
-	switch value := destination.(type) {
-	case string:
-		prefix, err := netip.ParsePrefix(value)
-		if err != nil {
-			return err
-		}
-		*i = IPPrefix(prefix)
-
-		return nil
-	default:
-		return fmt.Errorf("%w: unexpected data type %T", ErrCannotParsePrefix, destination)
-	}
-}
-
-// Value return json value, implement driver.Valuer interface.
-func (i IPPrefix) Value() (driver.Value, error) {
-	prefixStr := netip.Prefix(i).String()
-
-	return prefixStr, nil
-}
-
-type IPPrefixes []netip.Prefix
-
-func (i *IPPrefixes) Scan(destination interface{}) error {
-	switch value := destination.(type) {
-	case []byte:
-		return json.Unmarshal(value, i)
-
-	case string:
-		return json.Unmarshal([]byte(value), i)
-
-	default:
-		return fmt.Errorf("%w: unexpected data type %T", ErrMachineAddressesInvalid, destination)
-	}
-}
-
-// Value return json value, implement driver.Valuer interface.
-func (i IPPrefixes) Value() (driver.Value, error) {
-	bytes, err := json.Marshal(i)
-
-	return string(bytes), err
-}
-
-type StringList []string
-
-func (i *StringList) Scan(destination interface{}) error {
-	switch value := destination.(type) {
-	case []byte:
-		return json.Unmarshal(value, i)
-
-	case string:
-		return json.Unmarshal([]byte(value), i)
-
-	default:
-		return fmt.Errorf("%w: unexpected data type %T", ErrMachineAddressesInvalid, destination)
-	}
-}
-
-// Value return json value, implement driver.Valuer interface.
-func (i StringList) Value() (driver.Value, error) {
-	bytes, err := json.Marshal(i)
-
-	return string(bytes), err
-}

dns_test.go

@@ -1,394 +0,0 @@
-package headscale
-
-import (
-	"fmt"
-	"net/netip"
-
-	"gopkg.in/check.v1"
-	"tailscale.com/tailcfg"
-	"tailscale.com/types/dnstype"
-)
-
-func (s *Suite) TestMagicDNSRootDomains100(c *check.C) {
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix("100.64.0.0/10"),
-	}
-	domains := generateMagicDNSRootDomains(prefixes)
-
-	found := false
-	for _, domain := range domains {
-		if domain == "64.100.in-addr.arpa." {
-			found = true
-
-			break
-		}
-	}
-	c.Assert(found, check.Equals, true)
-
-	found = false
-	for _, domain := range domains {
-		if domain == "100.100.in-addr.arpa." {
-			found = true
-
-			break
-		}
-	}
-	c.Assert(found, check.Equals, true)
-
-	found = false
-	for _, domain := range domains {
-		if domain == "127.100.in-addr.arpa." {
-			found = true
-
-			break
-		}
-	}
-	c.Assert(found, check.Equals, true)
-}
-
-func (s *Suite) TestMagicDNSRootDomains172(c *check.C) {
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix("172.16.0.0/16"),
-	}
-	domains := generateMagicDNSRootDomains(prefixes)
-
-	found := false
-	for _, domain := range domains {
-		if domain == "0.16.172.in-addr.arpa." {
-			found = true
-
-			break
-		}
-	}
-	c.Assert(found, check.Equals, true)
-
-	found = false
-	for _, domain := range domains {
-		if domain == "255.16.172.in-addr.arpa." {
-			found = true
-
-			break
-		}
-	}
-	c.Assert(found, check.Equals, true)
-}
-
-// Happens when netmask is a multiple of 4 bits (sounds likely).
-func (s *Suite) TestMagicDNSRootDomainsIPv6Single(c *check.C) {
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix("fd7a:115c:a1e0::/48"),
-	}
-	domains := generateMagicDNSRootDomains(prefixes)
-
-	c.Assert(len(domains), check.Equals, 1)
-	c.Assert(
-		domains[0].WithTrailingDot(),
-		check.Equals,
-		"0.e.1.a.c.5.1.1.a.7.d.f.ip6.arpa.",
-	)
-}
-
-func (s *Suite) TestMagicDNSRootDomainsIPv6SingleMultiple(c *check.C) {
-	prefixes := []netip.Prefix{
-		netip.MustParsePrefix("fd7a:115c:a1e0::/50"),
-	}
-	domains := generateMagicDNSRootDomains(prefixes)
-
-	yieldsRoot := func(dom string) bool {
-		for _, candidate := range domains {
-			if candidate.WithTrailingDot() == dom {
-				return true
-			}
-		}
-
-		return false
-	}
-
-	c.Assert(len(domains), check.Equals, 4)
-	c.Assert(yieldsRoot("0.0.e.1.a.c.5.1.1.a.7.d.f.ip6.arpa."), check.Equals, true)
-	c.Assert(yieldsRoot("1.0.e.1.a.c.5.1.1.a.7.d.f.ip6.arpa."), check.Equals, true)
-	c.Assert(yieldsRoot("2.0.e.1.a.c.5.1.1.a.7.d.f.ip6.arpa."), check.Equals, true)
-	c.Assert(yieldsRoot("3.0.e.1.a.c.5.1.1.a.7.d.f.ip6.arpa."), check.Equals, true)
-}
-
-func (s *Suite) TestDNSConfigMapResponseWithMagicDNS(c *check.C) {
-	userShared1, err := app.CreateUser("shared1")
-	c.Assert(err, check.IsNil)
-
-	userShared2, err := app.CreateUser("shared2")
-	c.Assert(err, check.IsNil)
-
-	userShared3, err := app.CreateUser("shared3")
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared1, err := app.CreatePreAuthKey(
-		userShared1.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared2, err := app.CreatePreAuthKey(
-		userShared2.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared3, err := app.CreatePreAuthKey(
-		userShared3.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	PreAuthKey2InShared1, err := app.CreatePreAuthKey(
-		userShared1.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	_, err = app.GetMachine(userShared1.Name, "test_get_shared_nodes_1")
-	c.Assert(err, check.NotNil)
-
-	machineInShared1 := &Machine{
-		ID:             1,
-		MachineKey:     "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		NodeKey:        "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		DiscoKey:       "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		Hostname:       "test_get_shared_nodes_1",
-		UserID:         userShared1.ID,
-		User:           *userShared1,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.1")},
-		AuthKeyID:      uint(preAuthKeyInShared1.ID),
-	}
-	app.db.Save(machineInShared1)
-
-	_, err = app.GetMachine(userShared1.Name, machineInShared1.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machineInShared2 := &Machine{
-		ID:             2,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_2",
-		UserID:         userShared2.ID,
-		User:           *userShared2,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.2")},
-		AuthKeyID:      uint(preAuthKeyInShared2.ID),
-	}
-	app.db.Save(machineInShared2)
-
-	_, err = app.GetMachine(userShared2.Name, machineInShared2.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machineInShared3 := &Machine{
-		ID:             3,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_3",
-		UserID:         userShared3.ID,
-		User:           *userShared3,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.3")},
-		AuthKeyID:      uint(preAuthKeyInShared3.ID),
-	}
-	app.db.Save(machineInShared3)
-
-	_, err = app.GetMachine(userShared3.Name, machineInShared3.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machine2InShared1 := &Machine{
-		ID:             4,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_4",
-		UserID:         userShared1.ID,
-		User:           *userShared1,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.4")},
-		AuthKeyID:      uint(PreAuthKey2InShared1.ID),
-	}
-	app.db.Save(machine2InShared1)
-
-	baseDomain := "foobar.headscale.net"
-	dnsConfigOrig := tailcfg.DNSConfig{
-		Routes:  make(map[string][]*dnstype.Resolver),
-		Domains: []string{baseDomain},
-		Proxied: true,
-	}
-
-	peersOfMachineInShared1, err := app.getPeers(machineInShared1)
-	c.Assert(err, check.IsNil)
-
-	dnsConfig := getMapResponseDNSConfig(
-		&dnsConfigOrig,
-		baseDomain,
-		*machineInShared1,
-		peersOfMachineInShared1,
-	)
-	c.Assert(dnsConfig, check.NotNil)
-
-	c.Assert(len(dnsConfig.Routes), check.Equals, 3)
-
-	domainRouteShared1 := fmt.Sprintf("%s.%s", userShared1.Name, baseDomain)
-	_, ok := dnsConfig.Routes[domainRouteShared1]
-	c.Assert(ok, check.Equals, true)
-
-	domainRouteShared2 := fmt.Sprintf("%s.%s", userShared2.Name, baseDomain)
-	_, ok = dnsConfig.Routes[domainRouteShared2]
-	c.Assert(ok, check.Equals, true)
-
-	domainRouteShared3 := fmt.Sprintf("%s.%s", userShared3.Name, baseDomain)
-	_, ok = dnsConfig.Routes[domainRouteShared3]
-	c.Assert(ok, check.Equals, true)
-}
-
-func (s *Suite) TestDNSConfigMapResponseWithoutMagicDNS(c *check.C) {
-	userShared1, err := app.CreateUser("shared1")
-	c.Assert(err, check.IsNil)
-
-	userShared2, err := app.CreateUser("shared2")
-	c.Assert(err, check.IsNil)
-
-	userShared3, err := app.CreateUser("shared3")
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared1, err := app.CreatePreAuthKey(
-		userShared1.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared2, err := app.CreatePreAuthKey(
-		userShared2.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	preAuthKeyInShared3, err := app.CreatePreAuthKey(
-		userShared3.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	preAuthKey2InShared1, err := app.CreatePreAuthKey(
-		userShared1.Name,
-		false,
-		false,
-		nil,
-		nil,
-	)
-	c.Assert(err, check.IsNil)
-
-	_, err = app.GetMachine(userShared1.Name, "test_get_shared_nodes_1")
-	c.Assert(err, check.NotNil)
-
-	machineInShared1 := &Machine{
-		ID:             1,
-		MachineKey:     "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		NodeKey:        "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		DiscoKey:       "686824e749f3b7f2a5927ee6c1e422aee5292592d9179a271ed7b3e659b44a66",
-		Hostname:       "test_get_shared_nodes_1",
-		UserID:         userShared1.ID,
-		User:           *userShared1,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.1")},
-		AuthKeyID:      uint(preAuthKeyInShared1.ID),
-	}
-	app.db.Save(machineInShared1)
-
-	_, err = app.GetMachine(userShared1.Name, machineInShared1.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machineInShared2 := &Machine{
-		ID:             2,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_2",
-		UserID:         userShared2.ID,
-		User:           *userShared2,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.2")},
-		AuthKeyID:      uint(preAuthKeyInShared2.ID),
-	}
-	app.db.Save(machineInShared2)
-
-	_, err = app.GetMachine(userShared2.Name, machineInShared2.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machineInShared3 := &Machine{
-		ID:             3,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_3",
-		UserID:         userShared3.ID,
-		User:           *userShared3,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.3")},
-		AuthKeyID:      uint(preAuthKeyInShared3.ID),
-	}
-	app.db.Save(machineInShared3)
-
-	_, err = app.GetMachine(userShared3.Name, machineInShared3.Hostname)
-	c.Assert(err, check.IsNil)
-
-	machine2InShared1 := &Machine{
-		ID:             4,
-		MachineKey:     "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		NodeKey:        "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		DiscoKey:       "dec46ef9dc45c7d2f03bfcd5a640d9e24e3cc68ce3d9da223867c9bc6d5e9863",
-		Hostname:       "test_get_shared_nodes_4",
-		UserID:         userShared1.ID,
-		User:           *userShared1,
-		RegisterMethod: RegisterMethodAuthKey,
-		IPAddresses:    []netip.Addr{netip.MustParseAddr("100.64.0.4")},
-		AuthKeyID:      uint(preAuthKey2InShared1.ID),
-	}
-	app.db.Save(machine2InShared1)
-
-	baseDomain := "foobar.headscale.net"
-	dnsConfigOrig := tailcfg.DNSConfig{
-		Routes:  make(map[string][]*dnstype.Resolver),
-		Domains: []string{baseDomain},
-		Proxied: false,
-	}
-
-	peersOfMachine1Shared1, err := app.getPeers(machineInShared1)
-	c.Assert(err, check.IsNil)
-
-	dnsConfig := getMapResponseDNSConfig(
-		&dnsConfigOrig,
-		baseDomain,
-		*machineInShared1,
-		peersOfMachine1Shared1,
-	)
-	c.Assert(dnsConfig, check.NotNil)
-	c.Assert(len(dnsConfig.Routes), check.Equals, 0)
-	c.Assert(len(dnsConfig.Domains), check.Equals, 1)
-}

docs/README.md

@@ -1,56 +0,0 @@
-# headscale documentation
-
-This page contains the official and community contributed documentation for `headscale`.
-
-If you are having trouble with following the documentation or get unexpected results,
-please ask on [Discord](https://discord.gg/c84AZQhmpx) instead of opening an Issue.
-
-## Official documentation
-
-### How-to
-
-- [Running headscale on Linux](running-headscale-linux.md)
-- [Control headscale remotely](remote-cli.md)
-- [Using a Windows client with headscale](windows-client.md)
-- [Configuring OIDC](oidc.md)
-
-### References
-
-- [Configuration](../config-example.yaml)
-- [Glossary](glossary.md)
-- [TLS](tls.md)
-
-## Community documentation
-
-Community documentation is not actively maintained by the headscale authors and is
-written by community members. It is _not_ verified by `headscale` developers.
-
-**It might be outdated and it might miss necessary steps**.
-
-- [Running headscale in a container](running-headscale-container.md)
-- [Running headscale on OpenBSD](running-headscale-openbsd.md)
-- [Running headscale behind a reverse proxy](reverse-proxy.md)
-- [Set Custom DNS records](dns-records.md)
-
-## Misc
-
-### Policy ACLs
-
-Headscale implements the same policy ACLs as Tailscale.com, adapted to the self-hosted environment.
-
-For instance, instead of referring to users when defining groups you must
-use users (which are the equivalent to user/logins in Tailscale.com).
-
-Please check https://tailscale.com/kb/1018/acls/, and `./tests/acls/` in this repo for working examples.
-
-When using ACL's the User borders are no longer applied. All machines
-whichever the User have the ability to communicate with other hosts as
-long as the ACL's permits this exchange.
-
-The [ACLs](acls.md) document should help understand a fictional case of setting
-up ACLs in a small company. All concepts presented in this document could be
-applied outside of business oriented usage.
-
-### Apple devices
-
-An endpoint with information on how to connect your Apple devices (currently macOS only) is available at `/apple` on your running instance.

docs/acls.md

@@ -1,4 +1,15 @@
-# ACLs use case example
+Headscale implements the same policy ACLs as Tailscale.com, adapted to the self-hosted environment.
+
+For instance, instead of referring to users when defining groups you must
+use users (which are the equivalent to user/logins in Tailscale.com).
+
+Please check https://tailscale.com/kb/1018/acls/, and `./tests/acls/` in this repo for working examples.
+
+When using ACL's the User borders are no longer applied. All machines
+whichever the User have the ability to communicate with other hosts as
+long as the ACL's permits this exchange.
+
+## ACLs use case example
 
 Let's build an example use case for a small business (It may be the place where
 ACL's are the most useful).

docs/dns-records.md

@@ -1,5 +1,12 @@
 # Setting custom DNS records
 
+!!! warning "Community documentation"
+
+    This page is not actively maintained by the headscale authors and is
+    written by community members. It is _not_ verified by `headscale` developers.
+
+    **It might be outdated and it might miss necessary steps**.
+
 ## Goal
 
 This documentation has the goal of showing how a user can set custom DNS records with `headscale`s magic dns.

docs/exit-node.md

@@ -14,6 +14,8 @@ If the node is already registered, it can advertise exit capabilities like this:
 $ sudo tailscale set --advertise-exit-node
 ```
 
+To use a node as an exit node, IP forwarding must be enabled on the node. Check the official [Tailscale documentation](https://tailscale.com/kb/1019/subnets/?tab=linux#enable-ip-forwarding) for how to enable IP fowarding.
+
 ## On the control server
 
 ```console

docs/faq.md

@@ -0,0 +1,53 @@
+---
+hide:
+  - navigation
+---
+
+# Frequently Asked Questions
+
+## What is the design goal of headscale?
+
+`headscale` aims to implement a self-hosted, open source alternative to the [Tailscale](https://tailscale.com/)
+control server.
+`headscale`'s goal is to provide self-hosters and hobbyists with an open-source
+server they can use for their projects and labs.
+It implements a narrow scope, a _single_ Tailnet, suitable for a personal use, or a small
+open-source organisation.
+
+## How can I contribute?
+
+Headscale is "Open Source, acknowledged contribution", this means that any
+contribution will have to be discussed with the Maintainers before being submitted.
+
+Headscale is open to code contributions for bug fixes without discussion.
+
+If you find mistakes in the documentation, please also submit a fix to the documentation.
+
+## Why is 'acknowledged contribution' the chosen model?
+
+Both maintainers have full-time jobs and families, and we want to avoid burnout. We also want to avoid frustration from contributors when their PRs are not accepted.
+
+We are more than happy to exchange emails, or to have dedicated calls before a PR is submitted.
+
+## When/Why is Feature X going to be implemented?
+
+We don't know. We might be working on it. If you want to help, please send us a PR.
+
+Please be aware that there are a number of reasons why we might not accept specific contributions:
+
+- It is not possible to implement the feature in a way that makes sense in a self-hosted environment.
+- Given that we are reverse-engineering Tailscale to satify our own curiosity, we might be interested in implementing the feature ourselves.
+- You are not sending unit and integration tests with it.
+
+## Do you support Y method of deploying Headscale?
+
+We currently support deploying `headscale` using our binaries and the DEB packages. Both can be found in the
+[GitHub releases page](https://github.com/juanfont/headscale/releases).
+
+In addition to that, there are semi-official RPM packages by the Fedora infra team https://copr.fedorainfracloud.org/coprs/jonathanspw/headscale/
+
+For convenience, we also build Docker images with `headscale`. But **please be aware that we don't officially support deploying `headscale` using Docker**. We have a [Discord channel](https://discord.com/channels/896711691637780480/1070619770942148618) where you can ask for Docker-specific help to the community.
+
+## Why is my reverse proxy not working with Headscale?
+
+We don't know. We don't use reverse proxies with `headscale` ourselves, so we don't have any experience with them. We have [community documentation](https://headscale.net/reverse-proxy/) on how to configure various reverse proxies, and a dedicated [Discord channel](https://discord.com/channels/896711691637780480/1070619818346164324) where you can ask for help to the community.

docs/iOS-client.md

@@ -12,6 +12,11 @@ Ensure that the installed version is at least 1.38.1, as that is the first relea
 
 ## Configuring the headscale URL
 
+!!! info "Apple devices"
+
+    An endpoint with information on how to connect your Apple devices
+    (currently macOS only) is available at `/apple` on your running instance.
+
 Ensure that the tailscale app is logged out before proceeding.
 
 Go to iOS settings, scroll down past game center and tv provider to the tailscale app and select it. The headscale URL can be entered into the _"ALTERNATE COORDINATION SERVER URL"_ box.

docs/index.md

@@ -0,0 +1,43 @@
+---
+hide:
+  - navigation
+  - toc
+---
+
+# headscale
+
+`headscale` is an open source, self-hosted implementation of the Tailscale control server.
+
+This page contains the documentation for the latest version of headscale. Please also check our [FAQ](/faq/).
+
+Join our [Discord](https://discord.gg/c84AZQhmpx) server for a chat and community support.
+
+## Design goal
+
+Headscale aims to implement a self-hosted, open source alternative to the Tailscale
+control server.
+Headscale's goal is to provide self-hosters and hobbyists with an open-source
+server they can use for their projects and labs.
+It implements a narrower scope, a single Tailnet, suitable for a personal use, or a small
+open-source organisation.
+
+## Supporting headscale
+
+If you like `headscale` and find it useful, there is a sponsorship and donation
+buttons available in the repo.
+
+## Contributing
+
+Headscale is "Open Source, acknowledged contribution", this means that any
+contribution will have to be discussed with the Maintainers before being submitted.
+
+This model has been chosen to reduce the risk of burnout by limiting the
+maintenance overhead of reviewing and validating third-party code.
+
+Headscale is open to code contributions for bug fixes without discussion.
+
+If you find mistakes in the documentation, please submit a fix to the documentation.
+
+## About
+
+`headscale` is maintained by [Kristoffer Dalby](https://kradalby.no/) and [Juan Font](https://font.eu).

docs/packaging/README.md

@@ -0,0 +1,5 @@
+# Packaging
+
+We use [nFPM](https://nfpm.goreleaser.com/) for making `.deb`, `.rpm` and `.apk`.
+
+This folder contains files we need to package with these releases.

docs/packaging/headscale.systemd.service

@@ -0,0 +1,51 @@
+[Unit]
+After=syslog.target
+After=network.target
+Description=headscale coordination server for Tailscale
+X-Restart-Triggers=/etc/headscale/config.yaml
+
+[Service]
+Type=simple
+User=headscale
+Group=headscale
+ExecStart=/usr/bin/headscale serve
+Restart=always
+RestartSec=5
+
+WorkingDirectory=/var/lib/headscale
+ReadWritePaths=/var/lib/headscale /var/run
+
+AmbientCapabilities=CAP_NET_BIND_SERVICE CAP_CHOWN
+CapabilityBoundingSet=CAP_NET_BIND_SERVICE CAP_CHOWN
+LockPersonality=true
+NoNewPrivileges=true
+PrivateDevices=true
+PrivateMounts=true
+PrivateTmp=true
+ProcSubset=pid
+ProtectClock=true
+ProtectControlGroups=true
+ProtectHome=true
+ProtectHostname=true
+ProtectKernelLogs=true
+ProtectKernelModules=true
+ProtectKernelTunables=true
+ProtectProc=invisible
+ProtectSystem=strict
+RemoveIPC=true
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+RestrictNamespaces=true
+RestrictRealtime=true
+RestrictSUIDSGID=true
+RuntimeDirectory=headscale
+RuntimeDirectoryMode=0750
+StateDirectory=headscale
+StateDirectoryMode=0750
+SystemCallArchitectures=native
+SystemCallFilter=@chown
+SystemCallFilter=@system-service
+SystemCallFilter=~@privileged
+UMask=0077
+
+[Install]
+WantedBy=multi-user.target

docs/packaging/postinstall.sh

@@ -0,0 +1,86 @@
+#!/bin/sh
+# Determine OS platform
+# shellcheck source=/dev/null
+. /etc/os-release
+
+HEADSCALE_EXE="/usr/bin/headscale"
+BSD_HIER=""
+HEADSCALE_RUN_DIR="/var/run/headscale"
+HEADSCALE_USER="headscale"
+HEADSCALE_GROUP="headscale"
+
+ensure_sudo() {
+	if [ "$(id -u)" = "0" ]; then
+		echo "Sudo permissions detected"
+	else
+		echo "No sudo permission detected, please run as sudo"
+		exit 1
+	fi
+}
+
+ensure_headscale_path() {
+	if [ ! -f "$HEADSCALE_EXE" ]; then
+		echo "headscale not in default path, exiting..."
+		exit 1
+	fi
+
+	printf "Found headscale %s\n" "$HEADSCALE_EXE"
+}
+
+create_headscale_user() {
+	printf "PostInstall: Adding headscale user %s\n" "$HEADSCALE_USER"
+	useradd -s /bin/sh -c "headscale default user" headscale
+}
+
+create_headscale_group() {
+	if command -V systemctl >/dev/null 2>&1; then
+		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
+		groupadd "$HEADSCALE_GROUP"
+
+		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+		usermod -a -G "$HEADSCALE_GROUP" "$HEADSCALE_USER"
+	fi
+
+	if [ "$ID" = "alpine" ]; then
+		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
+		addgroup "$HEADSCALE_GROUP"
+
+		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+		addgroup "$HEADSCALE_USER" "$HEADSCALE_GROUP"
+	fi
+}
+
+create_run_dir() {
+	printf "PostInstall: Creating headscale run directory \n"
+	mkdir -p "$HEADSCALE_RUN_DIR"
+
+	printf "PostInstall: Modifying group ownership of headscale run directory \n"
+	chown "$HEADSCALE_USER":"$HEADSCALE_GROUP" "$HEADSCALE_RUN_DIR"
+}
+
+summary() {
+	echo "----------------------------------------------------------------------"
+	echo " headscale package has been successfully installed."
+	echo ""
+	echo " Please follow the next steps to start the software:"
+	echo ""
+	echo "    sudo systemctl enable headscale"
+	echo "    sudo systemctl start headscale"
+	echo ""
+	echo " Configuration settings can be adjusted here:"
+	echo "    ${BSD_HIER}/etc/headscale/config.yaml"
+	echo ""
+	echo "----------------------------------------------------------------------"
+}
+
+#
+# Main body of the script
+#
+{
+	ensure_sudo
+	ensure_headscale_path
+	create_headscale_user
+	create_headscale_group
+	create_run_dir
+	summary
+}

docs/packaging/postremove.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+# Determine OS platform
+# shellcheck source=/dev/null
+. /etc/os-release
+
+if command -V systemctl >/dev/null 2>&1; then
+	echo "Stop and disable headscale service"
+	systemctl stop headscale >/dev/null 2>&1 || true
+	systemctl disable headscale >/dev/null 2>&1 || true
+	echo "Running daemon-reload"
+	systemctl daemon-reload || true
+fi
+
+echo "Removing run directory"
+rm -rf "/var/run/headscale.sock"

docs/requirements.txt

@@ -0,0 +1,5 @@
+cairosvg~=2.7.1
+mkdocs-material~=9.4.14
+mkdocs-minify-plugin~=0.7.1
+pillow~=10.1.0
+

docs/reverse-proxy.md

@@ -1,5 +1,12 @@
 # Running headscale behind a reverse proxy
 
+!!! warning "Community documentation"
+
+    This page is not actively maintained by the headscale authors and is
+    written by community members. It is _not_ verified by `headscale` developers.
+
+    **It might be outdated and it might miss necessary steps**.
+
 Running headscale behind a reverse proxy is useful when running multiple applications on the same server, and you want to reuse the same external IP and port - usually tcp/443 for HTTPS.
 
 ### WebSockets