flake.lock: Update

Flake lock file updates:

• Updated input 'nixpkgs':
    'github:NixOS/nixpkgs/35f5903' (2025-10-15)
  → 'github:NixOS/nixpkgs/f997fa0' (2025-12-11)

.claude/agents/headscale-integration-tester.md

@@ -0,0 +1,763 @@
+---
+name: headscale-integration-tester
+description: Use this agent when you need to execute, analyze, or troubleshoot Headscale integration tests. This includes running specific test scenarios, investigating test failures, interpreting test artifacts, validating end-to-end functionality, or ensuring integration test quality before releases. Examples: <example>Context: User has made changes to the route management code and wants to validate the changes work correctly. user: 'I've updated the route advertisement logic in poll.go. Can you run the relevant integration tests to make sure everything still works?' assistant: 'I'll use the headscale-integration-tester agent to run the subnet routing integration tests and analyze the results.' <commentary>Since the user wants to validate route-related changes with integration tests, use the headscale-integration-tester agent to execute the appropriate tests and analyze results.</commentary></example> <example>Context: A CI pipeline integration test is failing and the user needs help understanding why. user: 'The TestSubnetRouterMultiNetwork test is failing in CI. The logs show some timing issues but I can't figure out what's wrong.' assistant: 'Let me use the headscale-integration-tester agent to analyze the test failure and examine the artifacts.' <commentary>Since this involves analyzing integration test failures and interpreting test artifacts, use the headscale-integration-tester agent to investigate the issue.</commentary></example>
+color: green
+---
+
+You are a specialist Quality Assurance Engineer with deep expertise in Headscale's integration testing system. You understand the Docker-based test infrastructure, real Tailscale client interactions, and the complex timing considerations involved in end-to-end network testing.
+
+## Integration Test System Overview
+
+The Headscale integration test system uses Docker containers running real Tailscale clients against a Headscale server. Tests validate end-to-end functionality including routing, ACLs, node lifecycle, and network coordination. The system is built around the `hi` (Headscale Integration) test runner in `cmd/hi/`.
+
+## Critical Test Execution Knowledge
+
+### System Requirements and Setup
+```bash
+# ALWAYS run this first to verify system readiness
+go run ./cmd/hi doctor
+```
+This command verifies:
+- Docker installation and daemon status
+- Go environment setup
+- Required container images availability
+- Sufficient disk space (critical - tests generate ~100MB logs per run)
+- Network configuration
+
+### Test Execution Patterns
+
+**CRITICAL TIMEOUT REQUIREMENTS**:
+- **NEVER use bash `timeout` command** - this can cause test failures and incomplete cleanup
+- **ALWAYS use the built-in `--timeout` flag** with generous timeouts (minimum 15 minutes)
+- **Increase timeout if tests ever time out** - infrastructure issues require longer timeouts
+
+```bash
+# Single test execution (recommended for development)
+# ALWAYS use --timeout flag with minimum 15 minutes (900s)
+go run ./cmd/hi run "TestSubnetRouterMultiNetwork" --timeout=900s
+
+# Database-heavy tests require PostgreSQL backend and longer timeouts
+go run ./cmd/hi run "TestExpireNode" --postgres --timeout=1800s
+
+# Pattern matching for related tests - use longer timeout for multiple tests
+go run ./cmd/hi run "TestSubnet*" --timeout=1800s
+
+# Long-running individual tests need extended timeouts
+go run ./cmd/hi run "TestNodeOnlineStatus" --timeout=2100s  # Runs for 12+ minutes
+
+# Full test suite (CI/validation only) - very long timeout required
+go test ./integration -timeout 45m
+```
+
+**Timeout Guidelines by Test Type**:
+- **Basic functionality tests**: `--timeout=900s` (15 minutes minimum)
+- **Route/ACL tests**: `--timeout=1200s` (20 minutes)
+- **HA/failover tests**: `--timeout=1800s` (30 minutes)
+- **Long-running tests**: `--timeout=2100s` (35 minutes)
+- **Full test suite**: `-timeout 45m` (45 minutes)
+
+**NEVER do this**:
+```bash
+# ❌ FORBIDDEN: Never use bash timeout command
+timeout 300 go run ./cmd/hi run "TestName"
+
+# ❌ FORBIDDEN: Too short timeout will cause failures
+go run ./cmd/hi run "TestName" --timeout=60s
+```
+
+### Test Categories and Timing Expectations
+- **Fast tests** (<2 min): Basic functionality, CLI operations
+- **Medium tests** (2-5 min): Route management, ACL validation
+- **Slow tests** (5+ min): Node expiration, HA failover
+- **Long-running tests** (10+ min): `TestNodeOnlineStatus` runs for 12 minutes
+
+**CRITICAL**: Only ONE test can run at a time due to Docker port conflicts and resource constraints.
+
+## Test Artifacts and Log Analysis
+
+### Artifact Structure
+All test runs save comprehensive artifacts to `control_logs/TIMESTAMP-ID/`:
+```
+control_logs/20250713-213106-iajsux/
+├── hs-testname-abc123.stderr.log     # Headscale server error logs
+├── hs-testname-abc123.stdout.log     # Headscale server output logs
+├── hs-testname-abc123.db             # Database snapshot for post-mortem
+├── hs-testname-abc123_metrics.txt    # Prometheus metrics dump
+├── hs-testname-abc123-mapresponses/  # Protocol-level debug data
+├── ts-client-xyz789.stderr.log       # Tailscale client error logs
+├── ts-client-xyz789.stdout.log       # Tailscale client output logs
+└── ts-client-xyz789_status.json      # Client network status dump
+```
+
+### Log Analysis Priority Order
+When tests fail, examine artifacts in this specific order:
+
+1. **Headscale server stderr logs** (`hs-*.stderr.log`): Look for errors, panics, database issues, policy evaluation failures
+2. **Tailscale client stderr logs** (`ts-*.stderr.log`): Check for authentication failures, network connectivity issues
+3. **MapResponse JSON files**: Protocol-level debugging for network map generation issues
+4. **Client status dumps** (`*_status.json`): Network state and peer connectivity information
+5. **Database snapshots** (`.db` files): For data consistency and state persistence issues
+
+## Common Failure Patterns and Root Cause Analysis
+
+### CRITICAL MINDSET: Code Issues vs Infrastructure Issues
+
+**⚠️ IMPORTANT**: When tests fail, it is ALMOST ALWAYS a code issue with Headscale, NOT infrastructure problems. Do not immediately blame disk space, Docker issues, or timing unless you have thoroughly investigated the actual error logs first.
+
+### Systematic Debugging Process
+
+1. **Read the actual error message**: Don't assume - read the stderr logs completely
+2. **Check Headscale server logs first**: Most issues originate from server-side logic
+3. **Verify client connectivity**: Only after ruling out server issues
+4. **Check timing patterns**: Use proper `EventuallyWithT` patterns
+5. **Infrastructure as last resort**: Only blame infrastructure after code analysis
+
+### Real Failure Patterns
+
+#### 1. Timing Issues (Common but fixable)
+```go
+// ❌ Wrong: Immediate assertions after async operations
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+nodes, _ := headscale.ListNodes()
+require.Len(t, nodes[0].GetAvailableRoutes(), 1) // WILL FAIL
+
+// ✅ Correct: Wait for async operations
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes[0].GetAvailableRoutes(), 1)
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+```
+
+**Timeout Guidelines**:
+- Route operations: 3-5 seconds
+- Node state changes: 5-10 seconds
+- Complex scenarios: 10-15 seconds
+- Policy recalculation: 5-10 seconds
+
+#### 2. NodeStore Synchronization Issues
+Route advertisements must propagate through poll requests (`poll.go:420`). NodeStore updates happen at specific synchronization points after Hostinfo changes.
+
+#### 3. Test Data Management Issues
+```go
+// ❌ Wrong: Assuming array ordering
+require.Len(t, nodes[0].GetAvailableRoutes(), 1)
+
+// ✅ Correct: Identify nodes by properties
+expectedRoutes := map[string]string{"1": "10.33.0.0/16"}
+for _, node := range nodes {
+    nodeIDStr := fmt.Sprintf("%d", node.GetId())
+    if route, shouldHaveRoute := expectedRoutes[nodeIDStr]; shouldHaveRoute {
+        // Test the specific node that should have the route
+    }
+}
+```
+
+#### 4. Database Backend Differences
+SQLite vs PostgreSQL have different timing characteristics:
+- Use `--postgres` flag for database-intensive tests
+- PostgreSQL generally has more consistent timing
+- Some race conditions only appear with specific backends
+
+## Resource Management and Cleanup
+
+### Disk Space Management
+Tests consume significant disk space (~100MB per run):
+```bash
+# Check available space before running tests
+df -h
+
+# Clean up test artifacts periodically
+rm -rf control_logs/older-timestamp-dirs/
+
+# Clean Docker resources
+docker system prune -f
+docker volume prune -f
+```
+
+### Container Cleanup
+- Successful tests clean up automatically
+- Failed tests may leave containers running
+- Manually clean if needed: `docker ps -a` and `docker rm -f <containers>`
+
+## Advanced Debugging Techniques
+
+### Protocol-Level Debugging
+MapResponse JSON files in `control_logs/*/hs-*-mapresponses/` contain:
+- Network topology as sent to clients
+- Peer relationships and visibility
+- Route distribution and primary route selection
+- Policy evaluation results
+
+### Database State Analysis
+Use the database snapshots for post-mortem analysis:
+```bash
+# SQLite examination
+sqlite3 control_logs/TIMESTAMP/hs-*.db
+.tables
+.schema nodes
+SELECT * FROM nodes WHERE name LIKE '%problematic%';
+```
+
+### Performance Analysis
+Prometheus metrics dumps show:
+- Request latencies and error rates
+- NodeStore operation timing
+- Database query performance
+- Memory usage patterns
+
+## Test Development and Quality Guidelines
+
+### Proper Test Patterns
+```go
+// Always use EventuallyWithT for async operations
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Test condition that may take time to become true
+}, timeout, interval, "descriptive failure message")
+
+// Handle node identification correctly
+var targetNode *v1.Node
+for _, node := range nodes {
+    if node.GetName() == expectedNodeName {
+        targetNode = node
+        break
+    }
+}
+require.NotNil(t, targetNode, "should find expected node")
+```
+
+### Quality Validation Checklist
+- ✅ Tests use `EventuallyWithT` for asynchronous operations
+- ✅ Tests don't rely on array ordering for node identification
+- ✅ Proper cleanup and resource management
+- ✅ Tests handle both success and failure scenarios
+- ✅ Timing assumptions are realistic for operations being tested
+- ✅ Error messages are descriptive and actionable
+
+## Real-World Test Failure Patterns from HA Debugging
+
+### Infrastructure vs Code Issues - Detailed Examples
+
+**INFRASTRUCTURE FAILURES (Rare but Real)**:
+1. **DNS Resolution in Auth Tests**: `failed to resolve "hs-pingallbyip-jax97k": no DNS fallback candidates remain`
+   - **Pattern**: Client containers can't resolve headscale server hostname during logout
+   - **Detection**: Error messages specifically mention DNS/hostname resolution
+   - **Solution**: Docker networking reset, not code changes
+
+2. **Container Creation Timeouts**: Test gets stuck during client container setup
+   - **Pattern**: Tests hang indefinitely at container startup phase
+   - **Detection**: No progress in logs for >2 minutes during initialization
+   - **Solution**: `docker system prune -f` and retry
+
+3. **Docker Port Conflicts**: Multiple tests trying to use same ports
+   - **Pattern**: "bind: address already in use" errors
+   - **Detection**: Port binding failures in Docker logs
+   - **Solution**: Only run ONE test at a time
+
+**CODE ISSUES (99% of failures)**:
+1. **Route Approval Process Failures**: Routes not getting approved when they should be
+   - **Pattern**: Tests expecting approved routes but finding none
+   - **Detection**: `SubnetRoutes()` returns empty when `AnnouncedRoutes()` shows routes
+   - **Root Cause**: Auto-approval logic bugs, policy evaluation issues
+
+2. **NodeStore Synchronization Issues**: State updates not propagating correctly
+   - **Pattern**: Route changes not reflected in NodeStore or Primary Routes
+   - **Detection**: Logs show route announcements but no tracking updates
+   - **Root Cause**: Missing synchronization points in `poll.go:420` area
+
+3. **HA Failover Architecture Issues**: Routes removed when nodes go offline
+   - **Pattern**: `TestHASubnetRouterFailover` fails because approved routes disappear
+   - **Detection**: Routes available on online nodes but lost when nodes disconnect
+   - **Root Cause**: Conflating route approval with node connectivity
+
+### Critical Test Environment Setup
+
+**Pre-Test Cleanup (MANDATORY)**:
+```bash
+# ALWAYS run this before each test
+rm -rf control_logs/202507*
+docker system prune -f
+df -h  # Verify sufficient disk space
+```
+
+**Environment Verification**:
+```bash
+# Verify system readiness
+go run ./cmd/hi doctor
+
+# Check for running containers that might conflict
+docker ps
+```
+
+### Specific Test Categories and Known Issues
+
+#### Route-Related Tests (Primary Focus)
+```bash
+# Core route functionality - these should work first
+# Note: Generous timeouts are required for reliable execution
+go run ./cmd/hi run "TestSubnetRouteACL" --timeout=1200s
+go run ./cmd/hi run "TestAutoApproveMultiNetwork" --timeout=1800s
+go run ./cmd/hi run "TestHASubnetRouterFailover" --timeout=1800s
+```
+
+**Common Route Test Patterns**:
+- Tests validate route announcement, approval, and distribution workflows
+- Route state changes are asynchronous - may need `EventuallyWithT` wrappers
+- Route approval must respect ACL policies - test expectations encode security requirements
+- HA tests verify route persistence during node connectivity changes
+
+#### Authentication Tests (Infrastructure-Prone)
+```bash
+# These tests are more prone to infrastructure issues
+# Require longer timeouts due to auth flow complexity
+go run ./cmd/hi run "TestAuthKeyLogoutAndReloginSameUser" --timeout=1200s
+go run ./cmd/hi run "TestAuthWebFlowLogoutAndRelogin" --timeout=1200s
+go run ./cmd/hi run "TestOIDCExpireNodesBasedOnTokenExpiry" --timeout=1800s
+```
+
+**Common Auth Test Infrastructure Failures**:
+- DNS resolution during logout operations
+- Container creation timeouts
+- HTTP/2 stream errors (often symptoms, not root cause)
+
+### Security-Critical Debugging Rules
+
+**❌ FORBIDDEN CHANGES (Security & Test Integrity)**:
+1. **Never change expected test outputs** - Tests define correct behavior contracts
+   - Changing `require.Len(t, routes, 3)` to `require.Len(t, routes, 2)` because test fails
+   - Modifying expected status codes, node counts, or route counts
+   - Removing assertions that are "inconvenient"
+   - **Why forbidden**: Test expectations encode business requirements and security policies
+
+2. **Never bypass security mechanisms** - Security must never be compromised for convenience
+   - Using `AnnouncedRoutes()` instead of `SubnetRoutes()` in production code
+   - Skipping authentication or authorization checks
+   - **Why forbidden**: Security bypasses create vulnerabilities in production
+
+3. **Never reduce test coverage** - Tests prevent regressions
+   - Removing test cases or assertions
+   - Commenting out "problematic" test sections
+   - **Why forbidden**: Reduced coverage allows bugs to slip through
+
+**✅ ALLOWED CHANGES (Timing & Observability)**:
+1. **Fix timing issues with proper async patterns**
+   ```go
+   // ✅ GOOD: Add EventuallyWithT for async operations
+   require.EventuallyWithT(t, func(c *assert.CollectT) {
+       nodes, err := headscale.ListNodes()
+       assert.NoError(c, err)
+       assert.Len(c, nodes, expectedCount) // Keep original expectation
+   }, 10*time.Second, 100*time.Millisecond, "nodes should reach expected count")
+   ```
+   - **Why allowed**: Fixes race conditions without changing business logic
+
+2. **Add MORE observability and debugging**
+   - Additional logging statements
+   - More detailed error messages
+   - Extra assertions that verify intermediate states
+   - **Why allowed**: Better observability helps debug without changing behavior
+
+3. **Improve test documentation**
+   - Add godoc comments explaining test purpose and business logic
+   - Document timing requirements and async behavior
+   - **Why encouraged**: Helps future maintainers understand intent
+
+### Advanced Debugging Workflows
+
+#### Route Tracking Debug Flow
+```bash
+# Run test with detailed logging and proper timeout
+go run ./cmd/hi run "TestSubnetRouteACL" --timeout=1200s > test_output.log 2>&1
+
+# Check route approval process
+grep -E "(auto-approval|ApproveRoutesWithPolicy|PolicyManager)" test_output.log
+
+# Check route tracking
+tail -50 control_logs/*/hs-*.stderr.log | grep -E "(announced|tracking|SetNodeRoutes)"
+
+# Check for security violations
+grep -E "(AnnouncedRoutes.*SetNodeRoutes|bypass.*approval)" test_output.log
+```
+
+#### HA Failover Debug Flow
+```bash
+# Test HA failover specifically with adequate timeout
+go run ./cmd/hi run "TestHASubnetRouterFailover" --timeout=1800s
+
+# Check route persistence during disconnect
+grep -E "(Disconnect|NodeWentOffline|PrimaryRoutes)" control_logs/*/hs-*.stderr.log
+
+# Verify routes don't disappear inappropriately
+grep -E "(removing.*routes|SetNodeRoutes.*empty)" control_logs/*/hs-*.stderr.log
+```
+
+### Test Result Interpretation Guidelines
+
+#### Success Patterns to Look For
+- `"updating node routes for tracking"` in logs
+- Routes appearing in `announcedRoutes` logs
+- Proper `ApproveRoutesWithPolicy` calls for auto-approval
+- Routes persisting through node connectivity changes (HA tests)
+
+#### Failure Patterns to Investigate
+- `SubnetRoutes()` returning empty when `AnnouncedRoutes()` has routes
+- Routes disappearing when nodes go offline (HA architectural issue)
+- Missing `EventuallyWithT` causing timing race conditions
+- Security bypass attempts using wrong route methods
+
+### Critical Testing Methodology
+
+**Phase-Based Testing Approach**:
+1. **Phase 1**: Core route tests (ACL, auto-approval, basic functionality)
+2. **Phase 2**: HA and complex route scenarios
+3. **Phase 3**: Auth tests (infrastructure-sensitive, test last)
+
+**Per-Test Process**:
+1. Clean environment before each test
+2. Monitor logs for route tracking and approval messages
+3. Check artifacts in `control_logs/` if test fails
+4. Focus on actual error messages, not assumptions
+5. Document results and patterns discovered
+
+## Test Documentation and Code Quality Standards
+
+### Adding Missing Test Documentation
+When you understand a test's purpose through debugging, always add comprehensive godoc:
+
+```go
+// TestSubnetRoutes validates the complete subnet route lifecycle including
+// advertisement from clients, policy-based approval, and distribution to peers.
+// This test ensures that route security policies are properly enforced and that
+// only approved routes are distributed to the network.
+//
+// The test verifies:
+// - Route announcements are received and tracked
+// - ACL policies control route approval correctly
+// - Only approved routes appear in peer network maps
+// - Route state persists correctly in the database
+func TestSubnetRoutes(t *testing.T) {
+    // Test implementation...
+}
+```
+
+**Why add documentation**: Future maintainers need to understand business logic and security requirements encoded in tests.
+
+### Comment Guidelines - Focus on WHY, Not WHAT
+
+```go
+// ✅ GOOD: Explains reasoning and business logic
+// Wait for route propagation because NodeStore updates are asynchronous
+// and happen after poll requests complete processing
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Check that security policies are enforced...
+}, timeout, interval, "route approval must respect ACL policies")
+
+// ❌ BAD: Just describes what the code does
+// Wait for routes
+require.EventuallyWithT(t, func(c *assert.CollectT) {
+    // Get routes and check length
+}, timeout, interval, "checking routes")
+```
+
+**Why focus on WHY**: Helps maintainers understand architectural decisions and security requirements.
+
+## EventuallyWithT Pattern for External Calls
+
+### Overview
+EventuallyWithT is a testing pattern used to handle eventual consistency in distributed systems. In Headscale integration tests, many operations are asynchronous - clients advertise routes, the server processes them, updates propagate through the network. EventuallyWithT allows tests to wait for these operations to complete while making assertions.
+
+### External Calls That Must Be Wrapped
+The following operations are **external calls** that interact with the headscale server or tailscale clients and MUST be wrapped in EventuallyWithT:
+- `headscale.ListNodes()` - Queries server state
+- `client.Status()` - Gets client network status
+- `client.Curl()` - Makes HTTP requests through the network
+- `client.Traceroute()` - Performs network diagnostics
+- `client.Execute()` when running commands that query state
+- Any operation that reads from the headscale server or tailscale client
+
+### Five Key Rules for EventuallyWithT
+
+1. **One External Call Per EventuallyWithT Block**
+   - Each EventuallyWithT should make ONE external call (e.g., ListNodes OR Status)
+   - Related assertions based on that single call can be grouped together
+   - Unrelated external calls must be in separate EventuallyWithT blocks
+
+2. **Variable Scoping**
+   - Declare variables that need to be shared across EventuallyWithT blocks at function scope
+   - Use `=` for assignment inside EventuallyWithT, not `:=` (unless the variable is only used within that block)
+   - Variables declared with `:=` inside EventuallyWithT are not accessible outside
+
+3. **No Nested EventuallyWithT**
+   - NEVER put an EventuallyWithT inside another EventuallyWithT
+   - This is a critical anti-pattern that must be avoided
+
+4. **Use CollectT for Assertions**
+   - Inside EventuallyWithT, use `assert` methods with the CollectT parameter
+   - Helper functions called within EventuallyWithT must accept `*assert.CollectT`
+
+5. **Descriptive Messages**
+   - Always provide a descriptive message as the last parameter
+   - Message should explain what condition is being waited for
+
+### Correct Pattern Examples
+
+```go
+// CORRECT: Single external call with related assertions
+var nodes []*v1.Node
+var err error
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err = headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+    // These assertions are all based on the ListNodes() call
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+    requireNodeRouteCountWithCollect(c, nodes[1], 1, 1, 1)
+}, 10*time.Second, 500*time.Millisecond, "nodes should have expected route counts")
+
+// CORRECT: Separate EventuallyWithT for different external call
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+    // All these assertions are based on the single Status() call
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedPrefixes)
+    }
+}, 10*time.Second, 500*time.Millisecond, "client should see expected routes")
+
+// CORRECT: Variable scoping for sharing between blocks
+var routeNode *v1.Node
+var nodeKey key.NodePublic
+
+// First EventuallyWithT to get the node
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+
+    for _, node := range nodes {
+        if node.GetName() == "router" {
+            routeNode = node
+            nodeKey, _ = key.ParseNodePublicUntyped(mem.S(node.GetNodeKey()))
+            break
+        }
+    }
+    assert.NotNil(c, routeNode, "should find router node")
+}, 10*time.Second, 100*time.Millisecond, "router node should exist")
+
+// Second EventuallyWithT using the nodeKey from first block
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+
+    peerStatus, ok := status.Peer[nodeKey]
+    assert.True(c, ok, "peer should exist in status")
+    requirePeerSubnetRoutesWithCollect(c, peerStatus, expectedPrefixes)
+}, 10*time.Second, 100*time.Millisecond, "routes should be visible to client")
+```
+
+### Incorrect Patterns to Avoid
+
+```go
+// INCORRECT: Multiple unrelated external calls in same EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    // First external call
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 2)
+
+    // Second unrelated external call - WRONG!
+    status, err := client.Status()
+    assert.NoError(c, err)
+    assert.NotNil(c, status)
+}, 10*time.Second, 500*time.Millisecond, "mixed operations")
+
+// INCORRECT: Nested EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+
+    // NEVER do this!
+    assert.EventuallyWithT(t, func(c2 *assert.CollectT) {
+        status, _ := client.Status()
+        assert.NotNil(c2, status)
+    }, 5*time.Second, 100*time.Millisecond, "nested")
+}, 10*time.Second, 500*time.Millisecond, "outer")
+
+// INCORRECT: Variable scoping error
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes() // This shadows outer 'nodes' variable
+    assert.NoError(c, err)
+}, 10*time.Second, 500*time.Millisecond, "get nodes")
+
+// This will fail - nodes is nil because := created a new variable inside the block
+require.Len(t, nodes, 2) // COMPILATION ERROR or nil pointer
+
+// INCORRECT: Not wrapping external calls
+nodes, err := headscale.ListNodes() // External call not wrapped!
+require.NoError(t, err)
+```
+
+### Helper Functions for EventuallyWithT
+
+When creating helper functions for use within EventuallyWithT:
+
+```go
+// Helper function that accepts CollectT
+func requireNodeRouteCountWithCollect(c *assert.CollectT, node *v1.Node, available, approved, primary int) {
+    assert.Len(c, node.GetAvailableRoutes(), available, "available routes for node %s", node.GetName())
+    assert.Len(c, node.GetApprovedRoutes(), approved, "approved routes for node %s", node.GetName())
+    assert.Len(c, node.GetPrimaryRoutes(), primary, "primary routes for node %s", node.GetName())
+}
+
+// Usage within EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    requireNodeRouteCountWithCollect(c, nodes[0], 2, 2, 2)
+}, 10*time.Second, 500*time.Millisecond, "route counts should match expected")
+```
+
+### Operations That Must NOT Be Wrapped
+
+**CRITICAL**: The following operations are **blocking/mutating operations** that change state and MUST NOT be wrapped in EventuallyWithT:
+- `tailscale set` commands (e.g., `--advertise-routes`, `--accept-routes`)
+- `headscale.ApproveRoute()` - Approves routes on server
+- `headscale.CreateUser()` - Creates users
+- `headscale.CreatePreAuthKey()` - Creates authentication keys
+- `headscale.RegisterNode()` - Registers new nodes
+- Any `client.Execute()` that modifies configuration
+- Any operation that creates, updates, or deletes resources
+
+These operations:
+1. Complete synchronously or fail immediately
+2. Should not be retried automatically
+3. Need explicit error handling with `require.NoError()`
+
+### Correct Pattern for Blocking Operations
+
+```go
+// CORRECT: Blocking operation NOT wrapped
+status := client.MustStatus()
+command := []string{"tailscale", "set", "--advertise-routes=" + expectedRoutes[string(status.Self.ID)]}
+_, _, err = client.Execute(command)
+require.NoErrorf(t, err, "failed to advertise route: %s", err)
+
+// Then wait for the result with EventuallyWithT
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetAvailableRoutes(), expectedRoutes[string(status.Self.ID)])
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+
+// INCORRECT: Blocking operation wrapped (DON'T DO THIS)
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    _, _, err = client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+    assert.NoError(c, err) // This might retry the command multiple times!
+}, 10*time.Second, 100*time.Millisecond, "advertise routes")
+```
+
+### Assert vs Require Pattern
+
+When working within EventuallyWithT blocks where you need to prevent panics:
+
+```go
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+
+    // For array bounds - use require with t to prevent panic
+    assert.Len(c, nodes, 6)  // Test expectation
+    require.GreaterOrEqual(t, len(nodes), 3, "need at least 3 nodes to avoid panic")
+
+    // For nil pointer access - use require with t before dereferencing
+    assert.NotNil(c, srs1PeerStatus.PrimaryRoutes)  // Test expectation
+    require.NotNil(t, srs1PeerStatus.PrimaryRoutes, "primary routes must be set to avoid panic")
+    assert.Contains(c,
+        srs1PeerStatus.PrimaryRoutes.AsSlice(),
+        pref,
+    )
+}, 5*time.Second, 200*time.Millisecond, "checking route state")
+```
+
+**Key Principle**:
+- Use `assert` with `c` (*assert.CollectT) for test expectations that can be retried
+- Use `require` with `t` (*testing.T) for MUST conditions that prevent panics
+- Within EventuallyWithT, both are available - choose based on whether failure would cause a panic
+
+### Common Scenarios
+
+1. **Waiting for route advertisement**:
+```go
+client.Execute([]string{"tailscale", "set", "--advertise-routes=10.0.0.0/24"})
+
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetAvailableRoutes(), "10.0.0.0/24")
+}, 10*time.Second, 100*time.Millisecond, "route should be advertised")
+```
+
+2. **Checking client sees routes**:
+```go
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    status, err := client.Status()
+    assert.NoError(c, err)
+
+    // Check all peers have expected routes
+    for _, peerKey := range status.Peers() {
+        peerStatus := status.Peer[peerKey]
+        assert.Contains(c, peerStatus.AllowedIPs, expectedPrefix)
+    }
+}, 10*time.Second, 100*time.Millisecond, "all peers should see route")
+```
+
+3. **Sequential operations**:
+```go
+// First wait for node to appear
+var nodeID uint64
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Len(c, nodes, 1)
+    nodeID = nodes[0].GetId()
+}, 10*time.Second, 100*time.Millisecond, "node should register")
+
+// Then perform operation
+_, err := headscale.ApproveRoute(nodeID, "10.0.0.0/24")
+require.NoError(t, err)
+
+// Then wait for result
+assert.EventuallyWithT(t, func(c *assert.CollectT) {
+    nodes, err := headscale.ListNodes()
+    assert.NoError(c, err)
+    assert.Contains(c, nodes[0].GetApprovedRoutes(), "10.0.0.0/24")
+}, 10*time.Second, 100*time.Millisecond, "route should be approved")
+```
+
+## Your Core Responsibilities
+
+1. **Test Execution Strategy**: Execute integration tests with appropriate configurations, understanding when to use `--postgres` and timing requirements for different test categories. Follow phase-based testing approach prioritizing route tests.
+   - **Why this priority**: Route tests are less infrastructure-sensitive and validate core security logic
+
+2. **Systematic Test Analysis**: When tests fail, systematically examine artifacts starting with Headscale server logs, then client logs, then protocol data. Focus on CODE ISSUES first (99% of cases), not infrastructure. Use real-world failure patterns to guide investigation.
+   - **Why this approach**: Most failures are logic bugs, not environment issues - efficient debugging saves time
+
+3. **Timing & Synchronization Expertise**: Understand asynchronous Headscale operations, particularly route advertisements, NodeStore synchronization at `poll.go:420`, and policy propagation. Fix timing with `EventuallyWithT` while preserving original test expectations.
+   - **Why preserve expectations**: Test assertions encode business requirements and security policies
+   - **Key Pattern**: Apply the EventuallyWithT pattern correctly for all external calls as documented above
+
+4. **Root Cause Analysis**: Distinguish between actual code regressions (route approval logic, HA failover architecture), timing issues requiring `EventuallyWithT` patterns, and genuine infrastructure problems (DNS, Docker, container issues).
+   - **Why this distinction matters**: Different problem types require completely different solution approaches
+   - **EventuallyWithT Issues**: Often manifest as flaky tests or immediate assertion failures after async operations
+
+5. **Security-Aware Quality Validation**: Ensure tests properly validate end-to-end functionality with realistic timing expectations and proper error handling. Never suggest security bypasses or test expectation changes. Add comprehensive godoc when you understand test business logic.
+   - **Why security focus**: Integration tests are the last line of defense against security regressions
+   - **EventuallyWithT Usage**: Proper use prevents race conditions without weakening security assertions
+
+**CRITICAL PRINCIPLE**: Test expectations are sacred contracts that define correct system behavior. When tests fail, fix the code to match the test, never change the test to match broken code. Only timing and observability improvements are allowed - business logic expectations are immutable.
+
+**EventuallyWithT PRINCIPLE**: Every external call to headscale server or tailscale client must be wrapped in EventuallyWithT. Follow the five key rules strictly: one external call per block, proper variable scoping, no nesting, use CollectT for assertions, and provide descriptive messages.
+
+**Remember**: Test failures are usually code issues in Headscale that need to be fixed, not infrastructure problems to be ignored. Use the specific debugging workflows and failure patterns documented above to efficiently identify root causes. Infrastructure issues have very specific signatures - everything else is code-related.

.coderabbit.yaml

@@ -1,15 +0,0 @@
-# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
-language: "en-GB"
-early_access: false
-reviews:
-  profile: "chill"
-  request_changes_workflow: false
-  high_level_summary: true
-  poem: true
-  review_status: true
-  collapse_walkthrough: false
-  auto_review:
-    enabled: true
-    drafts: true
-chat:
-  auto_reply: true

.dockerignore

@@ -17,3 +17,7 @@ LICENSE
 .vscode
 
 *.sock
+
+node_modules/
+package-lock.json
+package.json

.github/CODEOWNERS

@@ -1,10 +1,10 @@
 * @juanfont @kradalby
 
-*.md @ohdearaugustin
-*.yml @ohdearaugustin
-*.yaml @ohdearaugustin
-Dockerfile* @ohdearaugustin
-.goreleaser.yaml @ohdearaugustin
-/docs/ @ohdearaugustin
-/.github/workflows/ @ohdearaugustin
-/.github/renovate.json @ohdearaugustin
+*.md @ohdearaugustin @nblock
+*.yml @ohdearaugustin @nblock
+*.yaml @ohdearaugustin @nblock
+Dockerfile* @ohdearaugustin @nblock
+.goreleaser.yaml @ohdearaugustin @nblock
+/docs/ @ohdearaugustin @nblock
+/.github/workflows/ @ohdearaugustin @nblock
+/.github/renovate.json @ohdearaugustin @nblock

.github/ISSUE_TEMPLATE/bug_report.yaml

@@ -6,14 +6,18 @@ body:
   - type: checkboxes
     attributes:
       label: Is this a support request?
-      description: This issue tracker is for bugs and feature requests only. If you need help, please use ask in our Discord community
+      description:
+        This issue tracker is for bugs and feature requests only. If you need
+        help, please use ask in our Discord community
       options:
         - label: This is not a support request
           required: true
   - type: checkboxes
     attributes:
       label: Is there an existing issue for this?
-      description: Please search to see if an issue already exists for the bug you encountered.
+      description:
+        Please search to see if an issue already exists for the bug you
+        encountered.
       options:
         - label: I have searched the existing issues
           required: true
@@ -44,10 +48,19 @@ body:
     attributes:
       label: Environment
       description: |
+        Please provide information about your environment.
+        If you are using a container, always provide the headscale version and not only the Docker image version.
+        Please do not put "latest".
+
+        Describe your "headscale network". Is there a lot of nodes, are the nodes all interconnected, are some subnet routers?
+
+        If you are experiencing a problem during an upgrade, please provide the versions of the old and new versions of Headscale and Tailscale.
+
         examples:
-          - **OS**: Ubuntu 20.04
-          - **Headscale version**: 0.22.3
-          - **Tailscale version**: 1.64.0
+          - **OS**: Ubuntu 24.04
+          - **Headscale version**: 0.24.3
+          - **Tailscale version**: 1.80.0
+          - **Number of nodes**: 20
       value: |
         - OS:
         - Headscale version:
@@ -65,19 +78,31 @@ body:
           required: false
   - type: textarea
     attributes:
-      label: Anything else?
+      label: Debug information
       description: |
-        Links? References? Anything that will give us more context about the issue you are encountering!
+        Please have a look at our [Debugging and troubleshooting
+        guide](https://headscale.net/development/ref/debug/) to learn about
+        common debugging techniques.
+
+        Links? References? Anything that will give us more context about the issue you are encountering.
+        If **any** of these are omitted we will likely close your issue, do **not** ignore them.
 
         - Client netmap dump (see below)
-        - ACL configuration
+        - Policy configuration
         - Headscale configuration
+        - Headscale log (with `trace` enabled)
 
         Dump the netmap of tailscale clients:
         `tailscale debug netmap > DESCRIPTIVE_NAME.json`
 
-        Please provide information describing the netmap, which client, which headscale version etc.
+        Dump the status of tailscale clients:
+        `tailscale status --json > DESCRIPTIVE_NAME.json`
+
+        Get the logs of a Tailscale client that is not working as expected.
+        `tailscale debug daemon-logs`
 
         Tip: You can attach images or log files by clicking this area to highlight it and then dragging files in.
+        **Ensure** you use formatting for files you attach.
+        Do **not** paste in long files.
     validations:
-      required: false
+      required: true

.github/ISSUE_TEMPLATE/feature_request.yaml

@@ -25,9 +25,9 @@ body:
       description: Are you willing to contribute to the implementation of this feature?
       options:
         - label: I can write the design doc for this feature
-          required: true
+          required: false
         - label: I can contribute this feature
-          required: true
+          required: false
   - type: textarea
     attributes:
       label: How can it be implemented?

.github/workflows/build.yml

@@ -5,24 +5,22 @@ on:
     branches:
       - main
   pull_request:
-    branches:
-      - main
 
 concurrency:
   group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 
 jobs:
-  build:
+  build-nix:
     runs-on: ubuntu-latest
     permissions: write-all
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -31,12 +29,16 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
-      - name: Run build
+      - name: Run nix build
         id: build
         if: steps.changed-files.outputs.files == 'true'
         run: |
@@ -52,7 +54,7 @@ jobs:
           exit $BUILD_STATUS
 
       - name: Nix gosum diverging
-        uses: actions/github-script@v6
+        uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
         if: failure() && steps.build.outcome == 'failure'
         with:
           github-token: ${{secrets.GITHUB_TOKEN}}
@@ -64,8 +66,35 @@ jobs:
               body: 'Nix build failed with wrong gosum, please update "vendorSha256" (${{ steps.build.outputs.OLD_HASH }}) for the "headscale" package in flake.nix with the new SHA: ${{ steps.build.outputs.NEW_HASH }}'
             })
 
-      - uses: actions/upload-artifact@v4
+      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
         if: steps.changed-files.outputs.files == 'true'
         with:
           name: headscale-linux
           path: result/bin/headscale
+  build-cross:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        env:
+          - "GOARCH=arm64 GOOS=linux"
+          - "GOARCH=amd64 GOOS=linux"
+          - "GOARCH=arm64 GOOS=darwin"
+          - "GOARCH=amd64 GOOS=darwin"
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+
+      - name: Run go cross compile
+        env:
+          CGO_ENABLED: 0
+        run: env ${{ matrix.env }} nix develop --command -- go build -o "headscale"
+          ./cmd/headscale
+      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: "headscale-${{ matrix.env }}"
+          path: "headscale"

.github/workflows/check-generated.yml

@@ -0,0 +1,55 @@
+name: Check Generated Files
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  check-generated:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          fetch-depth: 2
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        with:
+          filters: |
+            files:
+              - '*.nix'
+              - 'go.*'
+              - '**/*.go'
+              - '**/*.proto'
+              - 'buf.gen.yaml'
+              - 'tools/**'
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+
+      - name: Run make generate
+        if: steps.changed-files.outputs.files == 'true'
+        run: nix develop --command -- make generate
+
+      - name: Check for uncommitted changes
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          if ! git diff --exit-code; then
+            echo "❌ Generated files are not up to date!"
+            echo "Please run 'make generate' and commit the changes."
+            exit 1
+          else
+            echo "✅ All generated files are up to date."
+          fi

.github/workflows/check-tests.yaml

@@ -10,12 +10,12 @@ jobs:
   check-tests:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -24,15 +24,19 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Generate and check integration tests
         if: steps.changed-files.outputs.files == 'true'
         run: |
-          nix develop --command bash -c "cd cmd/gh-action-integration-generator/ && go generate"
+          nix develop --command bash -c "cd .github/workflows && go generate"
           git diff --exit-code .github/workflows/test-integration.yaml
 
       - name: Show missing tests

.github/workflows/docs-deploy.yml

@@ -0,0 +1,51 @@
+name: Deploy docs
+
+on:
+  push:
+    branches:
+      # Main branch for development docs
+      - main
+
+      # Doc maintenance branches
+      - doc/[0-9]+.[0-9]+.[0-9]+
+    tags:
+      # Stable release tags
+      - v[0-9]+.[0-9]+.[0-9]+
+    paths:
+      - "docs/**"
+      - "mkdocs.yml"
+  workflow_dispatch:
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout repository
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          fetch-depth: 0
+      - name: Install python
+        uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
+        with:
+          python-version: 3.x
+      - name: Setup cache
+        uses: actions/cache@a7833574556fa59680c1b7cb190c1735db73ebf0 # v5.0.0
+        with:
+          key: ${{ github.ref }}
+          path: .cache
+      - name: Setup dependencies
+        run: pip install -r docs/requirements.txt
+      - name: Configure git
+        run: |
+          git config user.name github-actions
+          git config user.email github-actions@github.com
+      - name: Deploy development docs
+        if: github.ref == 'refs/heads/main'
+        run: mike deploy --push development unstable
+      - name: Deploy stable docs from doc branches
+        if: startsWith(github.ref, 'refs/heads/doc/')
+        run: mike deploy --push ${GITHUB_REF_NAME##*/}
+      - name: Deploy stable docs from tag
+        if: startsWith(github.ref, 'refs/tags/v')
+        # This assumes that only newer tags are pushed
+        run: mike deploy --push --update-aliases ${GITHUB_REF_NAME#v} stable latest

.github/workflows/docs-test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
       - name: Install python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@83679a892e2d95755f2dac6acb0bfd1e9ac5d548 # v6.1.0
         with:
           python-version: 3.x
       - name: Setup cache
-        uses: actions/cache@v2
+        uses: actions/cache@a7833574556fa59680c1b7cb190c1735db73ebf0 # v5.0.0
         with:
           key: ${{ github.ref }}
           path: .cache

.github/workflows/docs.yml

@@ -1,52 +0,0 @@
-name: Build documentation
-
-on:
-  push:
-    branches:
-      - main
-  workflow_dispatch:
-
-permissions:
-  contents: read
-  pages: write
-  id-token: write
-
-jobs:
-  build:
-    runs-on: ubuntu-latest
-    steps:
-      - name: Checkout repository
-        uses: actions/checkout@v4
-      - name: Install python
-        uses: actions/setup-python@v4
-        with:
-          python-version: 3.x
-      - name: Setup cache
-        uses: actions/cache@v2
-        with:
-          key: ${{ github.ref }}
-          path: .cache
-      - name: Setup dependencies
-        run: pip install -r docs/requirements.txt
-      - name: Build docs
-        run: mkdocs build --strict
-      - name: Upload artifact
-        uses: actions/upload-pages-artifact@v3
-        with:
-          path: ./site
-
-  deploy:
-    environment:
-      name: github-pages
-      url: ${{ steps.deployment.outputs.page_url }}
-    permissions:
-      pages: write
-      id-token: write
-    runs-on: ubuntu-latest
-    needs: build
-    steps:
-      - name: Configure Pages
-        uses: actions/configure-pages@v4
-      - name: Deploy to GitHub Pages
-        id: deployment
-        uses: actions/deploy-pages@v4

.github/workflows/gh-action-integration-generator.go

@@ -1,6 +1,6 @@
 package main
 
-//go:generate go run ./main.go
+//go:generate go run ./gh-action-integration-generator.go
 
 import (
 	"bytes"
@@ -38,23 +38,29 @@ func findTests() []string {
 	return tests
 }
 
-func updateYAML(tests []string) {
+func updateYAML(tests []string, jobName string, testPath string) {
 	testsForYq := fmt.Sprintf("[%s]", strings.Join(tests, ", "))
 
 	yqCommand := fmt.Sprintf(
-		"yq eval '.jobs.integration-test.strategy.matrix.test = %s' ../../.github/workflows/test-integration.yaml -i",
+		"yq eval '.jobs.%s.strategy.matrix.test = %s' %s -i",
+		jobName,
 		testsForYq,
+		testPath,
 	)
 	cmd := exec.Command("bash", "-c", yqCommand)
 
-	var out bytes.Buffer
-	cmd.Stdout = &out
+	var stdout bytes.Buffer
+	var stderr bytes.Buffer
+	cmd.Stdout = &stdout
+	cmd.Stderr = &stderr
 	err := cmd.Run()
 	if err != nil {
+		log.Printf("stdout: %s", stdout.String())
+		log.Printf("stderr: %s", stderr.String())
 		log.Fatalf("failed to run yq command: %s", err)
 	}
 
-	fmt.Println("YAML file updated successfully")
+	fmt.Printf("YAML file (%s) job %s updated successfully\n", testPath, jobName)
 }
 
 func main() {
@@ -65,5 +71,21 @@ func main() {
 		quotedTests[i] = fmt.Sprintf("\"%s\"", test)
 	}
 
-	updateYAML(quotedTests)
+	// Define selected tests for PostgreSQL
+	postgresTestNames := []string{
+		"TestACLAllowUserDst",
+		"TestPingAllByIP",
+		"TestEphemeral2006DeletedTooQuickly",
+		"TestPingAllByIPManyUpDown",
+		"TestSubnetRouterMultiNetwork",
+	}
+
+	quotedPostgresTests := make([]string, len(postgresTestNames))
+	for i, test := range postgresTestNames {
+		quotedPostgresTests[i] = fmt.Sprintf("\"%s\"", test)
+	}
+
+	// Update both SQLite and PostgreSQL job matrices
+	updateYAML(quotedTests, "sqlite", "./test-integration.yaml")
+	updateYAML(quotedPostgresTests, "postgres", "./test-integration.yaml")
 }

.github/workflows/gh-actions-updater.yaml

@@ -7,16 +7,17 @@ on:
 
 jobs:
   build:
+    if: github.repository == 'juanfont/headscale'
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}
 
       - name: Run GitHub Actions Version Updater
-        uses: saadmk11/github-actions-version-updater@v0.8.1
+        uses: saadmk11/github-actions-version-updater@d8781caf11d11168579c8e5e94f62b068038f442 # v0.9.0
         with:
           # [Required] Access token with `workflow` scope.
           token: ${{ secrets.WORKFLOW_SECRET }}

.github/workflows/integration-test-template.yml

@@ -0,0 +1,105 @@
+name: Integration Test Template
+
+on:
+  workflow_call:
+    inputs:
+      test:
+        required: true
+        type: string
+      postgres_flag:
+        required: false
+        type: string
+        default: ""
+      database_name:
+        required: true
+        type: string
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    env:
+      # Github does not allow us to access secrets in pull requests,
+      # so this env var is used to check if we have the secret or not.
+      # If we have the secrets, meaning we are running on push in a fork,
+      # there might be secrets available for more debugging.
+      # If TS_OAUTH_CLIENT_ID and TS_OAUTH_SECRET is set, then the job
+      # will join a debug tailscale network, set up SSH and a tmux session.
+      # The SSH will be configured to use the SSH key of the Github user
+      # that triggered the build.
+      HAS_TAILSCALE_SECRET: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          fetch-depth: 2
+      - name: Tailscale
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: tailscale/github-action@a392da0a182bba0e9613b6243ebd69529b1878aa # v4.1.0
+        with:
+          oauth-client-id: ${{ secrets.TS_OAUTH_CLIENT_ID }}
+          oauth-secret: ${{ secrets.TS_OAUTH_SECRET }}
+          tags: tag:gh
+      - name: Setup SSH server for Actor
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/setup-sshd-actor@master
+      - name: Download headscale image
+        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+        with:
+          name: headscale-image
+          path: /tmp/artifacts
+      - name: Download tailscale HEAD image
+        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+        with:
+          name: tailscale-head-image
+          path: /tmp/artifacts
+      - name: Download hi binary
+        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+        with:
+          name: hi-binary
+          path: /tmp/artifacts
+      - name: Download Go cache
+        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+        with:
+          name: go-cache
+          path: /tmp/artifacts
+      - name: Download postgres image
+        if: ${{ inputs.postgres_flag == '--postgres=1' }}
+        uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v6.0.0
+        with:
+          name: postgres-image
+          path: /tmp/artifacts
+      - name: Load Docker images, Go cache, and prepare binary
+        run: |
+          gunzip -c /tmp/artifacts/headscale-image.tar.gz | docker load
+          gunzip -c /tmp/artifacts/tailscale-head-image.tar.gz | docker load
+          if [ -f /tmp/artifacts/postgres-image.tar.gz ]; then
+            gunzip -c /tmp/artifacts/postgres-image.tar.gz | docker load
+          fi
+          chmod +x /tmp/artifacts/hi
+          docker images
+          # Extract Go cache to host directories for bind mounting
+          mkdir -p /tmp/go-cache
+          tar -xzf /tmp/artifacts/go-cache.tar.gz -C /tmp/go-cache
+          ls -la /tmp/go-cache/ /tmp/go-cache/.cache/
+      - name: Run Integration Test
+        env:
+          HEADSCALE_INTEGRATION_HEADSCALE_IMAGE: headscale:${{ github.sha }}
+          HEADSCALE_INTEGRATION_TAILSCALE_IMAGE: tailscale-head:${{ github.sha }}
+          HEADSCALE_INTEGRATION_POSTGRES_IMAGE: ${{ inputs.postgres_flag == '--postgres=1' && format('postgres:{0}', github.sha) || '' }}
+          HEADSCALE_INTEGRATION_GO_CACHE: /tmp/go-cache/go
+          HEADSCALE_INTEGRATION_GO_BUILD_CACHE: /tmp/go-cache/.cache/go-build
+        run: /tmp/artifacts/hi run --stats --ts-memory-limit=300 --hs-memory-limit=1500 "^${{ inputs.test }}$" \
+          --timeout=120m \
+          ${{ inputs.postgres_flag }}
+      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        if: always()
+        with:
+          name: ${{ inputs.database_name }}-${{ inputs.test }}-logs
+          path: "control_logs/*/*.log"
+      - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        if: always()
+        with:
+          name: ${{ inputs.database_name }}-${{ inputs.test }}-artifacts
+          path: control_logs/
+      - name: Setup a blocking tmux session
+        if: ${{ env.HAS_TAILSCALE_SECRET }}
+        uses: alexellis/block-with-tmux-action@master

.github/workflows/lint.yml

@@ -10,12 +10,12 @@ jobs:
   golangci-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -24,24 +24,33 @@ jobs:
               - '**/*.go'
               - 'integration_test/'
               - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: golangci-lint
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- golangci-lint run --new-from-rev=${{github.event.pull_request.base.sha}} --out-format=github-actions .
+        run: nix develop --command -- golangci-lint run
+          --new-from-rev=${{github.event.pull_request.base.sha}}
+          --output.text.path=stdout
+          --output.text.print-linter-name
+          --output.text.print-issued-lines
+          --output.text.colors
 
   prettier-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 2
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -55,21 +64,30 @@ jobs:
               - '**/*.css'
               - '**/*.scss'
               - '**/*.html'
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Prettify code
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --command -- prettier --no-error-on-unmatched-pattern --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
+        run: nix develop --command -- prettier --no-error-on-unmatched-pattern
+          --ignore-unknown --check **/*.{ts,js,md,yaml,yml,sass,css,scss,html}
 
   proto-lint:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: DeterminateSystems/nix-installer-action@main
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Buf lint
         run: nix develop --command -- buf lint proto

.github/workflows/nix-module-test.yml

@@ -0,0 +1,55 @@
+name: NixOS Module Tests
+
+on:
+  push:
+    branches:
+      - main
+  pull_request:
+    branches:
+      - main
+
+concurrency:
+  group: ${{ github.workflow }}-$${{ github.head_ref || github.run_id }}
+  cancel-in-progress: true
+
+jobs:
+  nix-module-check:
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          fetch-depth: 2
+
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        with:
+          filters: |
+            nix:
+              - 'nix/**'
+              - 'flake.nix'
+              - 'flake.lock'
+            go:
+              - 'go.*'
+              - '**/*.go'
+              - 'cmd/**'
+              - 'hscontrol/**'
+
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
+
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+
+      - name: Run NixOS module tests
+        if: steps.changed-files.outputs.nix == 'true' || steps.changed-files.outputs.go == 'true'
+        run: |
+          echo "Running NixOS module integration test..."
+          nix build .#checks.x86_64-linux.headscale -L

.github/workflows/release.yml

@@ -9,28 +9,33 @@ on:
 
 jobs:
   goreleaser:
+    if: github.repository == 'juanfont/headscale'
     runs-on: ubuntu-latest
     steps:
       - name: Checkout
-        uses: actions/checkout@v4
+        uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 0
 
       - name: Login to DockerHub
-        uses: docker/login-action@v3
+        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
         with:
           username: ${{ secrets.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_TOKEN }}
 
       - name: Login to GHCR
-        uses: docker/login-action@v3
+        uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0
         with:
           registry: ghcr.io
           username: ${{ github.repository_owner }}
           password: ${{ secrets.GITHUB_TOKEN }}
 
-      - uses: DeterminateSystems/nix-installer-action@main
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run goreleaser
         run: nix develop --command -- goreleaser release --clean

.github/workflows/stale.yml

@@ -6,18 +6,22 @@ on:
 
 jobs:
   close-issues:
+    if: github.repository == 'juanfont/headscale'
     runs-on: ubuntu-latest
     permissions:
       issues: write
       pull-requests: write
     steps:
-      - uses: actions/stale@v9
+      - uses: actions/stale@997185467fa4f803885201cee163a9f38240193d # v10.1.1
         with:
           days-before-issue-stale: 90
           days-before-issue-close: 7
           stale-issue-label: "stale"
-          stale-issue-message: "This issue is stale because it has been open for 90 days with no activity."
-          close-issue-message: "This issue was closed because it has been inactive for 14 days since being marked as stale."
+          stale-issue-message: "This issue is stale because it has been open for 90 days with no
+            activity."
+          close-issue-message: "This issue was closed because it has been inactive for 14 days
+            since being marked as stale."
           days-before-pr-stale: -1
           days-before-pr-close: -1
+          exempt-issue-labels: "no-stale-bot"
           repo-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/test-integration.yaml

@@ -1,11 +1,123 @@
-name: Integration Tests
+name: integration
+# To debug locally on a branch, and when needing secrets
+# change this to include `push` so the build is ran on
+# the main repository.
 on: [pull_request]
 concurrency:
   group: ${{ github.workflow }}-${{ github.head_ref || github.run_id }}
   cancel-in-progress: true
 jobs:
-  integration-test:
+  # build: Builds binaries and Docker images once, uploads as artifacts for reuse.
+  # build-postgres: Pulls postgres image separately to avoid Docker Hub rate limits.
+  # sqlite: Runs all integration tests with SQLite backend.
+  # postgres: Runs a subset of tests with PostgreSQL to verify database compatibility.
+  build:
     runs-on: ubuntu-latest
+    outputs:
+      files-changed: ${{ steps.changed-files.outputs.files }}
+    steps:
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
+        with:
+          fetch-depth: 2
+      - name: Get changed files
+        id: changed-files
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
+        with:
+          filters: |
+            files:
+              - '*.nix'
+              - 'go.*'
+              - '**/*.go'
+              - 'integration/**'
+              - 'config-example.yaml'
+              - '.github/workflows/test-integration.yaml'
+              - '.github/workflows/integration-test-template.yml'
+              - 'Dockerfile.*'
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
+        if: steps.changed-files.outputs.files == 'true'
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
+        if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix', '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
+      - name: Build binaries and warm Go cache
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          # Build all Go binaries in one nix shell to maximize cache reuse
+          nix develop --command -- bash -c '
+            go build -o hi ./cmd/hi
+            CGO_ENABLED=0 GOOS=linux go build -o headscale ./cmd/headscale
+            # Build integration test binary to warm the cache with all dependencies
+            go test -c ./integration -o /dev/null 2>/dev/null || true
+          '
+      - name: Upload hi binary
+        if: steps.changed-files.outputs.files == 'true'
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: hi-binary
+          path: hi
+          retention-days: 10
+      - name: Package Go cache
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          # Package Go module cache and build cache
+          tar -czf go-cache.tar.gz -C ~ go .cache/go-build
+      - name: Upload Go cache
+        if: steps.changed-files.outputs.files == 'true'
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: go-cache
+          path: go-cache.tar.gz
+          retention-days: 10
+      - name: Build headscale image
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          docker build \
+            --file Dockerfile.integration-ci \
+            --tag headscale:${{ github.sha }} \
+            .
+          docker save headscale:${{ github.sha }} | gzip > headscale-image.tar.gz
+      - name: Build tailscale HEAD image
+        if: steps.changed-files.outputs.files == 'true'
+        run: |
+          docker build \
+            --file Dockerfile.tailscale-HEAD \
+            --tag tailscale-head:${{ github.sha }} \
+            .
+          docker save tailscale-head:${{ github.sha }} | gzip > tailscale-head-image.tar.gz
+      - name: Upload headscale image
+        if: steps.changed-files.outputs.files == 'true'
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: headscale-image
+          path: headscale-image.tar.gz
+          retention-days: 10
+      - name: Upload tailscale HEAD image
+        if: steps.changed-files.outputs.files == 'true'
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: tailscale-head-image
+          path: tailscale-head-image.tar.gz
+          retention-days: 10
+  build-postgres:
+    runs-on: ubuntu-latest
+    needs: build
+    if: needs.build.outputs.files-changed == 'true'
+    steps:
+      - name: Pull and save postgres image
+        run: |
+          docker pull postgres:latest
+          docker tag postgres:latest postgres:${{ github.sha }}
+          docker save postgres:${{ github.sha }} | gzip > postgres-image.tar.gz
+      - name: Upload postgres image
+        uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+        with:
+          name: postgres-image
+          path: postgres-image.tar.gz
+          retention-days: 10
+  sqlite:
+    needs: build
+    if: needs.build.outputs.files-changed == 'true'
     strategy:
       fail-fast: false
       matrix:
@@ -19,101 +131,126 @@ jobs:
           - TestACLNamedHostsCanReach
           - TestACLDevice1CanAccessDevice2
           - TestPolicyUpdateWhileRunningWithCLIInDatabase
+          - TestACLAutogroupMember
+          - TestACLAutogroupTagged
+          - TestACLAutogroupSelf
+          - TestACLPolicyPropagationOverTime
+          - TestAPIAuthenticationBypass
+          - TestAPIAuthenticationBypassCurl
+          - TestGRPCAuthenticationBypass
+          - TestCLIWithConfigAuthenticationBypass
+          - TestAuthKeyLogoutAndReloginSameUser
+          - TestAuthKeyLogoutAndReloginNewUser
+          - TestAuthKeyLogoutAndReloginSameUserExpiredKey
+          - TestAuthKeyDeleteKey
+          - TestAuthKeyLogoutAndReloginRoutesPreserved
           - TestOIDCAuthenticationPingAll
           - TestOIDCExpireNodesBasedOnTokenExpiry
+          - TestOIDC024UserCreation
+          - TestOIDCAuthenticationWithPKCE
+          - TestOIDCReloginSameNodeNewUser
+          - TestOIDCFollowUpUrl
+          - TestOIDCMultipleOpenedLoginUrls
+          - TestOIDCReloginSameNodeSameUser
+          - TestOIDCExpiryAfterRestart
+          - TestOIDCACLPolicyOnJoin
+          - TestOIDCReloginSameUserRoutesPreserved
           - TestAuthWebFlowAuthenticationPingAll
-          - TestAuthWebFlowLogoutAndRelogin
+          - TestAuthWebFlowLogoutAndReloginSameUser
+          - TestAuthWebFlowLogoutAndReloginNewUser
           - TestUserCommand
           - TestPreAuthKeyCommand
           - TestPreAuthKeyCommandWithoutExpiry
           - TestPreAuthKeyCommandReusableEphemeral
           - TestPreAuthKeyCorrectUserLoggedInCommand
           - TestApiKeyCommand
-          - TestNodeTagCommand
-          - TestNodeAdvertiseTagNoACLCommand
-          - TestNodeAdvertiseTagWithACLCommand
           - TestNodeCommand
           - TestNodeExpireCommand
           - TestNodeRenameCommand
-          - TestNodeMoveCommand
           - TestPolicyCommand
+          - TestPolicyBrokenConfigCommand
+          - TestDERPVerifyEndpoint
           - TestResolveMagicDNS
-          - TestValidateResolvConf
+          - TestResolveMagicDNSExtraRecordsPath
           - TestDERPServerScenario
+          - TestDERPServerWebsocketScenario
           - TestPingAllByIP
           - TestPingAllByIPPublicDERP
-          - TestAuthKeyLogoutAndRelogin
           - TestEphemeral
+          - TestEphemeralInAlternateTimezone
           - TestEphemeral2006DeletedTooQuickly
           - TestPingAllByHostname
           - TestTaildrop
+          - TestUpdateHostnameFromClient
           - TestExpireNode
+          - TestSetNodeExpiryInFuture
           - TestNodeOnlineStatus
           - TestPingAllByIPManyUpDown
+          - Test2118DeletingOnlineNodePanics
           - TestEnablingRoutes
           - TestHASubnetRouterFailover
-          - TestEnableDisableAutoApprovedRoute
           - TestSubnetRouteACL
+          - TestEnablingExitRoutes
+          - TestSubnetRouterMultiNetwork
+          - TestSubnetRouterMultiNetworkExitNode
+          - TestAutoApproveMultiNetwork
+          - TestSubnetRouteACLFiltering
           - TestHeadscale
-          - TestCreateTailscale
           - TestTailscaleNodesJoiningHeadcale
           - TestSSHOneUserToAll
           - TestSSHMultipleUsersAllToAll
           - TestSSHNoSSHConfigured
           - TestSSHIsBlockedInACL
           - TestSSHUserOnlyIsolation
-        database: [postgres, sqlite]
-    steps:
-      - uses: actions/checkout@v4
-        with:
-          fetch-depth: 2
-      - name: Get changed files
-        id: changed-files
-        uses: dorny/paths-filter@v3
-        with:
-          filters: |
-            files:
-              - '*.nix'
-              - 'go.*'
-              - '**/*.go'
-              - 'integration_test/'
-              - 'config-example.yaml'
-      - uses: DeterminateSystems/nix-installer-action@main
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
-        if: steps.changed-files.outputs.files == 'true'
-      - uses: satackey/action-docker-layer-caching@main
-        if: steps.changed-files.outputs.files == 'true'
-        continue-on-error: true
-      - name: Run Integration Test
-        uses: Wandalen/wretry.action@master
-        if: steps.changed-files.outputs.files == 'true'
-        env:
-          USE_POSTGRES: ${{ matrix.database == 'postgres' && '1' || '0' }}
-        with:
-          attempt_limit: 5
-          command: |
-            nix develop --command -- docker run \
-              --tty --rm \
-              --volume ~/.cache/hs-integration-go:/go \
-              --name headscale-test-suite \
-              --volume $PWD:$PWD -w $PWD/integration \
-              --volume /var/run/docker.sock:/var/run/docker.sock \
-              --volume $PWD/control_logs:/tmp/control \
-              --env HEADSCALE_INTEGRATION_POSTGRES=${{env.USE_POSTGRES}} \
-              golang:1 \
-                go run gotest.tools/gotestsum@latest -- ./... \
-                  -failfast \
-                  -timeout 120m \
-                  -parallel 1 \
-                  -run "^${{ matrix.test }}$"
-      - uses: actions/upload-artifact@v4
-        if: always() && steps.changed-files.outputs.files == 'true'
-        with:
-          name: ${{ matrix.test }}-${{matrix.database}}-logs
-          path: "control_logs/*.log"
-      - uses: actions/upload-artifact@v4
-        if: always() && steps.changed-files.outputs.files == 'true'
-        with:
-          name: ${{ matrix.test }}-${{matrix.database}}-pprof
-          path: "control_logs/*.pprof.tar"
+          - TestSSHAutogroupSelf
+          - TestTagsAuthKeyWithTagRequestDifferentTag
+          - TestTagsAuthKeyWithTagNoAdvertiseFlag
+          - TestTagsAuthKeyWithTagCannotAddViaCLI
+          - TestTagsAuthKeyWithTagCannotChangeViaCLI
+          - TestTagsAuthKeyWithTagAdminOverrideReauthPreserves
+          - TestTagsAuthKeyWithTagCLICannotModifyAdminTags
+          - TestTagsAuthKeyWithoutTagCannotRequestTags
+          - TestTagsAuthKeyWithoutTagRegisterNoTags
+          - TestTagsAuthKeyWithoutTagCannotAddViaCLI
+          - TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithReset
+          - TestTagsAuthKeyWithoutTagCLINoOpAfterAdminWithEmptyAdvertise
+          - TestTagsAuthKeyWithoutTagCLICannotReduceAdminMultiTag
+          - TestTagsUserLoginOwnedTagAtRegistration
+          - TestTagsUserLoginNonExistentTagAtRegistration
+          - TestTagsUserLoginUnownedTagAtRegistration
+          - TestTagsUserLoginAddTagViaCLIReauth
+          - TestTagsUserLoginRemoveTagViaCLIReauth
+          - TestTagsUserLoginCLINoOpAfterAdminAssignment
+          - TestTagsUserLoginCLICannotRemoveAdminTags
+          - TestTagsAuthKeyWithTagRequestNonExistentTag
+          - TestTagsAuthKeyWithTagRequestUnownedTag
+          - TestTagsAuthKeyWithoutTagRequestNonExistentTag
+          - TestTagsAuthKeyWithoutTagRequestUnownedTag
+          - TestTagsAdminAPICannotSetNonExistentTag
+          - TestTagsAdminAPICanSetUnownedTag
+          - TestTagsAdminAPICannotRemoveAllTags
+          - TestTagsAdminAPICannotSetInvalidFormat
+    uses: ./.github/workflows/integration-test-template.yml
+    secrets: inherit
+    with:
+      test: ${{ matrix.test }}
+      postgres_flag: "--postgres=0"
+      database_name: "sqlite"
+  postgres:
+    needs: [build, build-postgres]
+    if: needs.build.outputs.files-changed == 'true'
+    strategy:
+      fail-fast: false
+      matrix:
+        test:
+          - TestACLAllowUserDst
+          - TestPingAllByIP
+          - TestEphemeral2006DeletedTooQuickly
+          - TestPingAllByIPManyUpDown
+          - TestSubnetRouterMultiNetwork
+    uses: ./.github/workflows/integration-test-template.yml
+    secrets: inherit
+    with:
+      test: ${{ matrix.test }}
+      postgres_flag: "--postgres=1"
+      database_name: "postgres"

.github/workflows/test.yml

@@ -11,13 +11,13 @@ jobs:
     runs-on: ubuntu-latest
 
     steps:
-      - uses: actions/checkout@v4
+      - uses: actions/checkout@8e8c483db84b4bee98b60c0593521ed34d9990e8 # v6.0.1
         with:
           fetch-depth: 2
 
       - name: Get changed files
         id: changed-files
-        uses: dorny/paths-filter@v3
+        uses: dorny/paths-filter@de90cc6fb38fc0963ad72b210f1f284cd68cea36 # v3.0.2
         with:
           filters: |
             files:
@@ -27,11 +27,21 @@ jobs:
               - 'integration_test/'
               - 'config-example.yaml'
 
-      - uses: DeterminateSystems/nix-installer-action@main
+      - uses: nixbuild/nix-quick-install-action@2c9db80fb984ceb1bcaa77cdda3fdf8cfba92035 # v34
         if: steps.changed-files.outputs.files == 'true'
-      - uses: DeterminateSystems/magic-nix-cache-action@main
+      - uses: nix-community/cache-nix-action@135667ec418502fa5a3598af6fb9eb733888ce6a # v6.1.3
         if: steps.changed-files.outputs.files == 'true'
+        with:
+          primary-key: nix-${{ runner.os }}-${{ runner.arch }}-${{ hashFiles('**/*.nix',
+            '**/flake.lock') }}
+          restore-prefixes-first-match: nix-${{ runner.os }}-${{ runner.arch }}
 
       - name: Run tests
         if: steps.changed-files.outputs.files == 'true'
-        run: nix develop --check
+        env:
+          # As of 2025-01-06, these env vars was not automatically
+          # set anymore which breaks the initdb for postgres on
+          # some of the database migration tests.
+          LC_ALL: "en_US.UTF-8"
+          LC_CTYPE: "en_US.UTF-8"
+        run: nix develop --command -- gotestsum

.github/workflows/update-flake.yml

@@ -6,13 +6,14 @@ on:
 
 jobs:
   lockfile:
+    if: github.repository == 'juanfont/headscale'
     runs-on: ubuntu-latest
     steps:
       - name: Checkout repository
-        uses: actions/checkout@v4
+        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
       - name: Install Nix
-        uses: DeterminateSystems/nix-installer-action@main
+        uses: DeterminateSystems/nix-installer-action@21a544727d0c62386e78b4befe52d19ad12692e3 # v17
       - name: Update flake.lock
-        uses: DeterminateSystems/update-flake-lock@main
+        uses: DeterminateSystems/update-flake-lock@428c2b58a4b7414dabd372acb6a03dba1084d3ab # v25
         with:
           pr-title: "Update flake.lock"

.gitignore

@@ -1,6 +1,10 @@
 ignored/
 tailscale/
 .vscode/
+.claude/
+logs/
+
+*.prof
 
 # Binaries for programs and plugins
 *.exe
@@ -20,8 +24,9 @@ vendor/
 
 dist/
 /headscale
-config.json
 config.yaml
+config*.yaml
+!config-example.yaml
 derp.yaml
 *.hujson
 *.key
@@ -45,3 +50,7 @@ integration_test/etc/config.dump.yaml
 /site
 
 __debug_bin
+
+node_modules/
+package-lock.json
+package.json

.golangci.yaml

@@ -1,64 +1,90 @@
 ---
-run:
-  timeout: 10m
-  build-tags:
-    - ts2019
-
-issues:
-  skip-dirs:
-    - gen
+version: "2"
 linters:
-  enable-all: true
+  default: all
   disable:
+    - cyclop
     - depguard
-
-    - revive
-    - lll
-    - gofmt
+    - dupl
+    - exhaustruct
+    - funcorder
+    - funlen
     - gochecknoglobals
     - gochecknoinits
     - gocognit
-    - funlen
-    - tagliatelle
     - godox
-    - ireturn
-    - execinquery
-    - exhaustruct
-    - nolintlint
-    - musttag # causes issues with imported libs
-    - depguard
-
-    # We should strive to enable these:
-    - wrapcheck
-    - dupl
-    - makezero
-    - maintidx
-
-    # Limits the methods of an interface to 10. We have more in integration tests
     - interfacebloat
-
-    # We might want to enable this, but it might be a lot of work
-    - cyclop
+    - ireturn
+    - lll
+    - maintidx
+    - makezero
+    - musttag
     - nestif
-    - wsl # might be incompatible with gofumpt
-    - testpackage
+    - nolintlint
     - paralleltest
+    - revive
+    - tagliatelle
+    - testpackage
+    - varnamelen
+    - wrapcheck
+    - wsl
+  settings:
+    forbidigo:
+      forbid:
+        # Forbid time.Sleep everywhere with context-appropriate alternatives
+        - pattern: 'time\.Sleep'
+          msg: >-
+            time.Sleep is forbidden.
+            In tests: use assert.EventuallyWithT for polling/waiting patterns.
+            In production code: use a backoff strategy (e.g., cenkalti/backoff) or proper synchronization primitives.
+      analyze-types: true
+    gocritic:
+      disabled-checks:
+        - appendAssign
+        - ifElseChain
+    nlreturn:
+      block-size: 4
+    varnamelen:
+      ignore-names:
+        - err
+        - db
+        - id
+        - ip
+        - ok
+        - c
+        - tt
+        - tx
+        - rx
+        - sb
+        - wg
+        - pr
+        - p
+        - p2
+      ignore-type-assert-ok: true
+      ignore-map-index-ok: true
+  exclusions:
+    generated: lax
+    presets:
+      - comments
+      - common-false-positives
+      - legacy
+      - std-error-handling
+    paths:
+      - third_party$
+      - builtin$
+      - examples$
+      - gen
 
-linters-settings:
-  varnamelen:
-    ignore-type-assert-ok: true
-    ignore-map-index-ok: true
-    ignore-names:
-      - err
-      - db
-      - id
-      - ip
-      - ok
-      - c
-      - tt
-
-  gocritic:
-    disabled-checks:
-      - appendAssign
-      # TODO(kradalby): Remove this
-      - ifElseChain
+formatters:
+  enable:
+    - gci
+    - gofmt
+    - gofumpt
+    - goimports
+  exclusions:
+    generated: lax
+    paths:
+      - third_party$
+      - builtin$
+      - examples$
+      - gen

.goreleaser.yml

@@ -1,11 +1,40 @@
 ---
+version: 2
 before:
   hooks:
-    - go mod tidy -compat=1.22
+    - go mod tidy -compat=1.25
     - go mod vendor
 
 release:
   prerelease: auto
+  draft: true
+  header: |
+    ## Upgrade
+
+    Please follow the steps outlined in the [upgrade guide](https://headscale.net/stable/setup/upgrade/) to update your existing Headscale installation.
+
+    **It's best to update from one stable version to the next** (e.g., 0.24.0 → 0.25.1 → 0.26.1) in case you are multiple releases behind. You should always pick the latest available patch release.
+
+    Be sure to check the changelog above for version-specific upgrade instructions and breaking changes.
+
+    ### Backup Your Database
+
+    **Always backup your database before upgrading.** Here's how to backup a SQLite database:
+
+    ```bash
+    # Stop headscale
+    systemctl stop headscale
+
+    # Backup sqlite database
+    cp /var/lib/headscale/db.sqlite /var/lib/headscale/db.sqlite.backup
+
+    # Backup sqlite WAL/SHM files (if they exist)
+    cp /var/lib/headscale/db.sqlite-wal /var/lib/headscale/db.sqlite-wal.backup
+    cp /var/lib/headscale/db.sqlite-shm /var/lib/headscale/db.sqlite-shm.backup
+
+    # Start headscale (migration will run automatically)
+    systemctl start headscale
+    ```
 
 builds:
   - id: headscale
@@ -17,23 +46,18 @@ builds:
       - darwin_amd64
       - darwin_arm64
       - freebsd_amd64
-      - linux_386
       - linux_amd64
       - linux_arm64
-      - linux_arm_5
-      - linux_arm_6
-      - linux_arm_7
     flags:
       - -mod=readonly
-    ldflags:
-      - -s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=v{{.Version}}
     tags:
       - ts2019
 
 archives:
   - id: golang-cross
     name_template: '{{ .ProjectName }}_{{ .Version }}_{{ .Os }}_{{ .Arch }}{{ with .Arm }}v{{ . }}{{ end }}{{ with .Mips }}_{{ . }}{{ end }}{{ if not (eq .Amd64 "v1") }}{{ .Amd64 }}{{ end }}'
-    format: binary
+    formats:
+      - binary
 
 source:
   enabled: true
@@ -52,15 +76,22 @@ nfpms:
   # List file contents: dpkg -c dist/headscale...deb
   # Package metadata: dpkg --info dist/headscale....deb
   #
-  - builds:
+  - ids:
       - headscale
     package_name: headscale
     priority: optional
     vendor: headscale
     maintainer: Kristoffer Dalby <kristoffer@dalby.cc>
     homepage: https://github.com/juanfont/headscale
-    license: BSD
+    description: |-
+      Open source implementation of the Tailscale control server.
+      Headscale aims to implement a self-hosted, open source alternative to the
+      Tailscale control server. Headscale's goal is to provide self-hosters and
+      hobbyists with an open-source server they can use for their projects and
+      labs. It implements a narrow scope, a single Tailscale network (tailnet),
+      suitable for a personal use, or a small open-source organisation.
     bindir: /usr/bin
+    section: net
     formats:
       - deb
     contents:
@@ -69,56 +100,39 @@ nfpms:
         type: config|noreplace
         file_info:
           mode: 0644
-      - src: ./docs/packaging/headscale.systemd.service
+      - src: ./packaging/systemd/headscale.service
         dst: /usr/lib/systemd/system/headscale.service
       - dst: /var/lib/headscale
         type: dir
-      - dst: /var/run/headscale
-        type: dir
+      - src: LICENSE
+        dst: /usr/share/doc/headscale/copyright
     scripts:
-      postinstall: ./docs/packaging/postinstall.sh
-      postremove: ./docs/packaging/postremove.sh
+      postinstall: ./packaging/deb/postinst
+      postremove: ./packaging/deb/postrm
+      preremove: ./packaging/deb/prerm
+    deb:
+      lintian_overrides:
+        - no-changelog # Our CHANGELOG.md uses a different formatting
+        - no-manual-page
+        - statically-linked-binary
 
 kos:
   - id: ghcr
-    repository: ghcr.io/juanfont/headscale
+    repositories:
+      - ghcr.io/juanfont/headscale
+      - headscale/headscale
 
     # bare tells KO to only use the repository
     # for tagging and naming the container.
     bare: true
-    base_image: gcr.io/distroless/base-debian12
+    base_image: gcr.io/distroless/base-debian13
     build: headscale
     main: ./cmd/headscale
     env:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
-      - linux/386
       - linux/arm64
-      - linux/arm/v7
-    tags:
-      - "{{ if not .Prerelease }}latest{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}.{{ .Patch }}{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}{{ end }}"
-      - "{{ if not .Prerelease }}stable{{ else }}unstable{{ end }}"
-      - "{{ .Tag }}"
-      - '{{ trimprefix .Tag "v" }}'
-      - "sha-{{ .ShortCommit }}"
-
-  - id: dockerhub
-    build: headscale
-    base_image: gcr.io/distroless/base-debian12
-    repository: headscale/headscale
-    bare: true
-    platforms:
-      - linux/amd64
-      - linux/386
-      - linux/arm64
-      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}{{ end }}"
@@ -131,20 +145,23 @@ kos:
       - "{{ .Tag }}"
       - '{{ trimprefix .Tag "v" }}'
       - "sha-{{ .ShortCommit }}"
+    creation_time: "{{.CommitTimestamp}}"
+    ko_data_creation_time: "{{.CommitTimestamp}}"
 
   - id: ghcr-debug
-    repository: ghcr.io/juanfont/headscale
+    repositories:
+      - ghcr.io/juanfont/headscale
+      - headscale/headscale
+
     bare: true
-    base_image: gcr.io/distroless/base-debian12:debug
+    base_image: gcr.io/distroless/base-debian13:debug
     build: headscale
     main: ./cmd/headscale
     env:
       - CGO_ENABLED=0
     platforms:
       - linux/amd64
-      - linux/386
       - linux/arm64
-      - linux/arm/v7
     tags:
       - "{{ if not .Prerelease }}latest-debug{{ end }}"
       - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"
@@ -153,30 +170,7 @@ kos:
       - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"
       - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}-debug{{ end }}"
       - "{{ if not .Prerelease }}v{{ .Major }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}stable{{ else }}unstable-debug{{ end }}"
-      - "{{ .Tag }}-debug"
-      - '{{ trimprefix .Tag "v" }}-debug'
-      - "sha-{{ .ShortCommit }}-debug"
-
-  - id: dockerhub-debug
-    build: headscale
-    base_image: gcr.io/distroless/base-debian12:debug
-    repository: headscale/headscale
-    bare: true
-    platforms:
-      - linux/amd64
-      - linux/386
-      - linux/arm64
-      - linux/arm/v7
-    tags:
-      - "{{ if not .Prerelease }}latest-debug{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}.{{ .Minor }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}{{ .Major }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}.{{ .Patch }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}.{{ .Minor }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}v{{ .Major }}-debug{{ end }}"
-      - "{{ if not .Prerelease }}stable{{ else }}unstable-debug{{ end }}"
+      - "{{ if not .Prerelease }}stable-debug{{ else }}unstable-debug{{ end }}"
       - "{{ .Tag }}-debug"
       - '{{ trimprefix .Tag "v" }}-debug'
       - "sha-{{ .ShortCommit }}-debug"
@@ -184,7 +178,7 @@ kos:
 checksum:
   name_template: "checksums.txt"
 snapshot:
-  name_template: "{{ .Tag }}-next"
+  version_template: "{{ .Tag }}-next"
 changelog:
   sort: asc
   filters:

.mcp.json

@@ -0,0 +1,34 @@
+{
+  "mcpServers": {
+    "claude-code-mcp": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@steipete/claude-code-mcp@latest"],
+      "env": {}
+    },
+    "sequential-thinking": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@modelcontextprotocol/server-sequential-thinking"],
+      "env": {}
+    },
+    "nixos": {
+      "type": "stdio",
+      "command": "uvx",
+      "args": ["mcp-nixos"],
+      "env": {}
+    },
+    "context7": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@upstash/context7-mcp"],
+      "env": {}
+    },
+    "git": {
+      "type": "stdio",
+      "command": "npx",
+      "args": ["-y", "@cyanheads/git-mcp-server"],
+      "env": {}
+    }
+  }
+}

.pre-commit-config.yaml

@@ -0,0 +1,68 @@
+# prek/pre-commit configuration for headscale
+# See: https://prek.j178.dev/quickstart/
+# See: https://prek.j178.dev/builtin/
+
+# Global exclusions - ignore generated code
+exclude: ^gen/
+
+repos:
+  # Built-in hooks from pre-commit/pre-commit-hooks
+  # prek will use fast-path optimized versions automatically
+  # See: https://prek.j178.dev/builtin/
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v6.0.0
+    hooks:
+      - id: check-added-large-files
+      - id: check-case-conflict
+      - id: check-executables-have-shebangs
+      - id: check-json
+      - id: check-merge-conflict
+      - id: check-symlinks
+      - id: check-toml
+      - id: check-xml
+      - id: check-yaml
+      - id: detect-private-key
+      - id: end-of-file-fixer
+      - id: fix-byte-order-marker
+      - id: mixed-line-ending
+      - id: trailing-whitespace
+
+  # Local hooks for project-specific tooling
+  - repo: local
+    hooks:
+      # nixpkgs-fmt for Nix files
+      - id: nixpkgs-fmt
+        name: nixpkgs-fmt
+        entry: nixpkgs-fmt
+        language: system
+        files: \.nix$
+
+      # Prettier for formatting
+      - id: prettier
+        name: prettier
+        entry: prettier --write --list-different
+        language: system
+        exclude: ^docs/
+        types_or:
+          [
+            javascript,
+            jsx,
+            ts,
+            tsx,
+            yaml,
+            json,
+            toml,
+            html,
+            css,
+            scss,
+            sass,
+            markdown,
+          ]
+
+      # golangci-lint for Go code quality
+      - id: golangci-lint
+        name: golangci-lint
+        entry: nix develop --command golangci-lint run --new-from-rev=HEAD~1 --timeout=5m --fix
+        language: system
+        types: [go]
+        pass_filenames: false

.prettierignore

@@ -1,6 +1,5 @@
 .github/workflows/test-integration-v2*
-docs/dns-records.md
-docs/running-headscale-container.md
-docs/running-headscale-linux-manual.md
-docs/running-headscale-linux.md
-docs/running-headscale-openbsd.md
+docs/about/features.md
+docs/ref/api.md
+docs/ref/configuration.md
+docs/ref/oidc.md

CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

CODE_OF_CONDUCT.md

@@ -62,7 +62,7 @@ event.
 
 Instances of abusive, harassing, or otherwise unacceptable behavior
 may be reported to the community leaders responsible for enforcement
-at our Discord channel. All complaints
+on our [Discord server](https://discord.gg/c84AZQhmpx). All complaints
 will be reviewed and investigated promptly and fairly.
 
 All community leaders are obligated to respect the privacy and

CONTRIBUTING.md

@@ -9,7 +9,7 @@ Headscale has a small maintainer team that tries to balance working on the proje
 
 When we work on issues ourselves, we develop first hand knowledge of the code and it makes it possible for us to maintain and own the code as the project develops.
 
-Code contributions are seen as a positive thing. People enjoy and engage with our project, but it also comes with some challenges; we have to understand the code, we have to understand the feature, we might have to become familiar with external libraries or services and we think about security implications. All those steps are required during the reviewing process. After the code has been merged, the feature has to be maintained. Any changes reliant on external services must be updated and expanded accordingly. 
+Code contributions are seen as a positive thing. People enjoy and engage with our project, but it also comes with some challenges; we have to understand the code, we have to understand the feature, we might have to become familiar with external libraries or services and we think about security implications. All those steps are required during the reviewing process. After the code has been merged, the feature has to be maintained. Any changes reliant on external services must be updated and expanded accordingly.
 
 The review and day-1 maintenance adds a significant burden on the maintainers. Often we hope that the contributor will help out, but we found that most of the time, they disappear after their new feature was added.
 

Dockerfile.debug

@@ -1,26 +0,0 @@
-# This Dockerfile and the images produced are for testing headscale,
-# and are in no way endorsed by Headscale's maintainers as an
-# official nor supported release or distribution.
-
-FROM docker.io/golang:1.22-bookworm
-ARG VERSION=dev
-ENV GOPATH /go
-WORKDIR /go/src/headscale
-
-RUN apt-get update \
-  && apt-get install --no-install-recommends --yes less jq \
-  && rm -rf /var/lib/apt/lists/* \
-  && apt-get clean
-RUN mkdir -p /var/run/headscale
-
-COPY go.mod go.sum /go/src/headscale/
-RUN go mod download
-
-COPY . .
-
-RUN CGO_ENABLED=0 GOOS=linux go install -ldflags="-s -w -X github.com/juanfont/headscale/cmd/headscale/cli.Version=$VERSION" -a ./cmd/headscale && test -e /go/bin/headscale
-
-# Need to reset the entrypoint or everything will run as a busybox script
-ENTRYPOINT []
-EXPOSE 8080/tcp
-CMD ["headscale"]

Dockerfile.derper

@@ -0,0 +1,19 @@
+# For testing purposes only
+
+FROM golang:alpine AS build-env
+
+WORKDIR /go/src
+
+RUN apk add --no-cache git
+ARG VERSION_BRANCH=main
+RUN git clone https://github.com/tailscale/tailscale.git --branch=$VERSION_BRANCH --depth=1
+WORKDIR /go/src/tailscale
+
+ARG TARGETARCH
+RUN GOARCH=$TARGETARCH go install -v ./cmd/derper
+
+FROM alpine:3.22
+RUN apk add --no-cache ca-certificates iptables iproute2 ip6tables curl
+
+COPY --from=build-env /go/bin/* /usr/local/bin/
+ENTRYPOINT [ "/usr/local/bin/derper" ]

Dockerfile.integration

@@ -0,0 +1,44 @@
+# This Dockerfile and the images produced are for testing headscale,
+# and are in no way endorsed by Headscale's maintainers as an
+# official nor supported release or distribution.
+
+FROM docker.io/golang:1.25-trixie AS builder
+ARG VERSION=dev
+ENV GOPATH /go
+WORKDIR /go/src/headscale
+
+# Install delve debugger first - rarely changes, good cache candidate
+RUN go install github.com/go-delve/delve/cmd/dlv@latest
+
+# Download dependencies - only invalidated when go.mod/go.sum change
+COPY go.mod go.sum /go/src/headscale/
+RUN go mod download
+
+# Copy source and build - invalidated on any source change
+COPY . .
+
+# Build debug binary with debug symbols for delve
+RUN CGO_ENABLED=0 GOOS=linux go build -gcflags="all=-N -l" -o /go/bin/headscale ./cmd/headscale
+
+# Runtime stage
+FROM debian:trixie-slim
+
+RUN apt-get --update install --no-install-recommends --yes \
+    less jq sqlite3 dnsutils ca-certificates procps bash findutils curl traceroute python3 \
+  && apt-get dist-clean
+
+RUN mkdir -p /var/run/headscale
+
+# Copy binaries from builder
+COPY --from=builder /go/bin/headscale /usr/local/bin/headscale
+COPY --from=builder /go/bin/dlv /usr/local/bin/dlv
+
+# Copy source code for delve source-level debugging
+COPY --from=builder /go/src/headscale /go/src/headscale
+
+WORKDIR /go/src/headscale
+
+# Need to reset the entrypoint or everything will run as a busybox script
+ENTRYPOINT []
+EXPOSE 8080/tcp 40000/tcp
+CMD ["dlv", "--listen=0.0.0.0:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/usr/local/bin/headscale", "--"]

Dockerfile.integration-ci

@@ -0,0 +1,17 @@
+# Minimal CI image - expects pre-built headscale binary in build context
+# For local development with delve debugging, use Dockerfile.integration instead
+
+FROM debian:trixie-slim
+
+RUN apt-get --update install --no-install-recommends --yes \
+    less jq sqlite3 dnsutils ca-certificates procps bash findutils curl traceroute \
+  && apt-get dist-clean
+
+RUN mkdir -p /var/run/headscale
+
+# Copy pre-built headscale binary from build context
+COPY headscale /usr/local/bin/headscale
+
+ENTRYPOINT []
+EXPOSE 8080/tcp
+CMD ["/usr/local/bin/headscale"]

Dockerfile.tailscale-HEAD

@@ -4,7 +4,7 @@
 # This Dockerfile is more or less lifted from tailscale/tailscale
 # to ensure a similar build process when testing the HEAD of tailscale.
 
-FROM golang:1.22-alpine AS build-env
+FROM golang:1.25-alpine AS build-env
 
 WORKDIR /go/src
 
@@ -28,14 +28,16 @@ ARG VERSION_GIT_HASH=""
 ENV VERSION_GIT_HASH=$VERSION_GIT_HASH
 ARG TARGETARCH
 
-RUN GOARCH=$TARGETARCH go install -ldflags="\
+ARG BUILD_TAGS=""
+
+RUN GOARCH=$TARGETARCH go install -tags="${BUILD_TAGS}" -ldflags="\
       -X tailscale.com/version.longStamp=$VERSION_LONG \
       -X tailscale.com/version.shortStamp=$VERSION_SHORT \
       -X tailscale.com/version.gitCommitStamp=$VERSION_GIT_HASH" \
       -v ./cmd/tailscale ./cmd/tailscaled ./cmd/containerboot
 
-FROM alpine:3.18
-RUN apk add --no-cache ca-certificates iptables iproute2 ip6tables curl
+FROM alpine:3.22
+RUN apk add --no-cache ca-certificates iptables iproute2 ip6tables curl traceroute
 
 COPY --from=build-env /go/bin/* /usr/local/bin/
 # For compat with the previous run.sh, although ideally you should be

Makefile

@@ -1,54 +1,128 @@
-# Calculate version
-version ?= $(shell git describe --always --tags --dirty)
+# Headscale Makefile
+# Modern Makefile following best practices
 
-rwildcard=$(foreach d,$(wildcard $1*),$(call rwildcard,$d/,$2) $(filter $(subst *,%,$2),$d))
+# Version calculation
+VERSION ?= $(shell git describe --always --tags --dirty)
 
-# Determine if OS supports pie
+# Build configuration
 GOOS ?= $(shell uname | tr '[:upper:]' '[:lower:]')
-ifeq ($(filter $(GOOS), openbsd netbsd soloaris plan9), )
-	pieflags = -buildmode=pie
-else
+ifeq ($(filter $(GOOS), openbsd netbsd solaris plan9), )
+	PIE_FLAGS = -buildmode=pie
 endif
 
-# GO_SOURCES = $(wildcard *.go)
-# PROTO_SOURCES = $(wildcard **/*.proto)
-GO_SOURCES = $(call rwildcard,,*.go)
-PROTO_SOURCES = $(call rwildcard,,*.proto)
+# Tool availability check with nix warning
+define check_tool
+	@command -v $(1) >/dev/null 2>&1 || { \
+		echo "Warning: $(1) not found. Run 'nix develop' to ensure all dependencies are available."; \
+		exit 1; \
+	}
+endef
+
+# Source file collections using shell find for better performance
+GO_SOURCES := $(shell find . -name '*.go' -not -path './gen/*' -not -path './vendor/*')
+PROTO_SOURCES := $(shell find . -name '*.proto' -not -path './gen/*' -not -path './vendor/*')
+DOC_SOURCES := $(shell find . \( -name '*.md' -o -name '*.yaml' -o -name '*.yml' -o -name '*.ts' -o -name '*.js' -o -name '*.html' -o -name '*.css' -o -name '*.scss' -o -name '*.sass' \) -not -path './gen/*' -not -path './vendor/*' -not -path './node_modules/*')
+
+# Default target
+.PHONY: all
+all: lint test build
+
+# Dependency checking
+.PHONY: check-deps
+check-deps:
+	$(call check_tool,go)
+	$(call check_tool,golangci-lint)
+	$(call check_tool,gofumpt)
+	$(call check_tool,prettier)
+	$(call check_tool,clang-format)
+	$(call check_tool,buf)
+
+# Build targets
+.PHONY: build
+build: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Building headscale..."
+	go build $(PIE_FLAGS) -ldflags "-X main.version=$(VERSION)" -o headscale ./cmd/headscale
+
+# Test targets
+.PHONY: test
+test: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Running Go tests..."
+	go test -race ./...
 
 
-build:
-	nix build
+# Formatting targets
+.PHONY: fmt
+fmt: fmt-go fmt-prettier fmt-proto
 
-dev: lint test build
+.PHONY: fmt-go
+fmt-go: check-deps $(GO_SOURCES)
+	@echo "Formatting Go code..."
+	gofumpt -l -w .
+	golangci-lint run --fix
 
-test:
-	gotestsum -- -short -coverprofile=coverage.out ./...
+.PHONY: fmt-prettier
+fmt-prettier: check-deps $(DOC_SOURCES)
+	@echo "Formatting documentation and config files..."
+	prettier --write '**/*.{ts,js,md,yaml,yml,sass,css,scss,html}'
 
-test_integration:
-	docker run \
-		-t --rm \
-		-v ~/.cache/hs-integration-go:/go \
-		--name headscale-test-suite \
-		-v $$PWD:$$PWD -w $$PWD/integration \
-		-v /var/run/docker.sock:/var/run/docker.sock \
-		-v $$PWD/control_logs:/tmp/control \
-		golang:1 \
-		go run gotest.tools/gotestsum@latest -- -failfast ./... -timeout 120m -parallel 8
+.PHONY: fmt-proto
+fmt-proto: check-deps $(PROTO_SOURCES)
+	@echo "Formatting Protocol Buffer files..."
+	clang-format -i $(PROTO_SOURCES)
 
-lint:
-	golangci-lint run --fix --timeout 10m
+# Linting targets
+.PHONY: lint
+lint: lint-go lint-proto
 
-fmt:
-	prettier --write '**/**.{ts,js,md,yaml,yml,sass,css,scss,html}'
-	golines --max-len=88 --base-formatter=gofumpt -w $(GO_SOURCES)
-	clang-format -style="{BasedOnStyle: Google, IndentWidth: 4, AlignConsecutiveDeclarations: true, AlignConsecutiveAssignments: true, ColumnLimit: 0}" -i $(PROTO_SOURCES)
+.PHONY: lint-go
+lint-go: check-deps $(GO_SOURCES) go.mod go.sum
+	@echo "Linting Go code..."
+	golangci-lint run --timeout 10m
 
-proto-lint:
-	cd proto/ && go run github.com/bufbuild/buf/cmd/buf lint
+.PHONY: lint-proto
+lint-proto: check-deps $(PROTO_SOURCES)
+	@echo "Linting Protocol Buffer files..."
+	cd proto/ && buf lint
 
-compress: build
-	upx --brute headscale
+# Code generation
+.PHONY: generate
+generate: check-deps
+	@echo "Generating code..."
+	go generate ./...
 
-generate:
-	rm -rf gen
-	buf generate proto
+# Clean targets
+.PHONY: clean
+clean:
+	rm -rf headscale gen
+
+# Development workflow
+.PHONY: dev
+dev: fmt lint test build
+
+# Help target
+.PHONY: help
+help:
+	@echo "Headscale Development Makefile"
+	@echo ""
+	@echo "Main targets:"
+	@echo "  all          - Run lint, test, and build (default)"
+	@echo "  build        - Build headscale binary"
+	@echo "  test         - Run Go tests"
+	@echo "  fmt          - Format all code (Go, docs, proto)"
+	@echo "  lint         - Lint all code (Go, proto)"
+	@echo "  generate     - Generate code from Protocol Buffers"
+	@echo "  dev          - Full development workflow (fmt + lint + test + build)"
+	@echo "  clean        - Clean build artifacts"
+	@echo ""
+	@echo "Specific targets:"
+	@echo "  fmt-go       - Format Go code only"
+	@echo "  fmt-prettier - Format documentation only"
+	@echo "  fmt-proto    - Format Protocol Buffer files only"
+	@echo "  lint-go      - Lint Go code only"
+	@echo "  lint-proto   - Lint Protocol Buffer files only"
+	@echo ""
+	@echo "Dependencies:"
+	@echo "  check-deps   - Verify required tools are available"
+	@echo ""
+	@echo "Note: If not running in a nix shell, ensure dependencies are available:"
+	@echo "  nix develop"

README.md

@@ -1,14 +1,18 @@
-![headscale logo](./docs/logo/headscale3_header_stacked_left.png)
+![headscale logo](./docs/assets/logo/headscale3_header_stacked_left.png)
 
 ![ci](https://github.com/juanfont/headscale/actions/workflows/test.yml/badge.svg)
 
 An open source, self-hosted implementation of the Tailscale control server.
 
-Join our [Discord](https://discord.gg/c84AZQhmpx) server for a chat.
+Join our [Discord server](https://discord.gg/c84AZQhmpx) for a chat.
 
 **Note:** Always select the same GitHub tag as the released version you use
-to ensure you have the correct example configuration and documentation.
-The `main` branch might contain unreleased changes.
+to ensure you have the correct example configuration. The `main` branch might
+contain unreleased changes. The documentation is available for stable and
+development versions:
+
+- [Documentation for the stable version](https://headscale.net/stable/)
+- [Documentation for the development version](https://headscale.net/development/)
 
 ## What is Tailscale
 
@@ -32,12 +36,12 @@ organisation.
 
 ## Design goal
 
-Headscale aims to implement a self-hosted, open source alternative to the Tailscale
-control server.
-Headscale's goal is to provide self-hosters and hobbyists with an open-source
-server they can use for their projects and labs.
-It implements a narrow scope, a single Tailnet, suitable for a personal use, or a small
-open-source organisation.
+Headscale aims to implement a self-hosted, open source alternative to the
+[Tailscale](https://tailscale.com/) control server. Headscale's goal is to
+provide self-hosters and hobbyists with an open-source server they can use for
+their projects and labs. It implements a narrow scope, a _single_ Tailscale
+network (tailnet), suitable for a personal use, or a small open-source
+organisation.
 
 ## Supporting Headscale
 
@@ -46,39 +50,20 @@ buttons available in the repo.
 
 ## Features
 
-- Full "base" support of Tailscale's features
-- Configurable DNS
-  - [Split DNS](https://tailscale.com/kb/1054/dns/#using-dns-settings-in-the-admin-console)
-- Node registration
-  - Single-Sign-On (via Open ID Connect)
-  - Pre authenticated key
-- Taildrop (File Sharing)
-- [Access control lists](https://tailscale.com/kb/1018/acls/)
-- [MagicDNS](https://tailscale.com/kb/1081/magicdns)
-- Support for multiple IP ranges in the tailnet
-- Dual stack (IPv4 and IPv6)
-- Routing advertising (including exit nodes)
-- Ephemeral nodes
-- Embedded [DERP server](https://tailscale.com/blog/how-tailscale-works/#encrypted-tcp-relays-derp)
+Please see ["Features" in the documentation](https://headscale.net/stable/about/features/).
 
 ## Client OS support
 
-| OS      | Supports headscale                                        |
-| ------- | --------------------------------------------------------- |
-| Linux   | Yes                                                       |
-| OpenBSD | Yes                                                       |
-| FreeBSD | Yes                                                       |
-| macOS   | Yes (see `/apple` on your headscale for more information) |
-| Windows | Yes [docs](./docs/windows-client.md)                      |
-| Android | Yes [docs](./docs/android-client.md)                      |
-| iOS     | Yes [docs](./docs/iOS-client.md)                          |
+Please see ["Client and operating system support" in the documentation](https://headscale.net/stable/about/clients/).
 
 ## Running headscale
 
 **Please note that we do not support nor encourage the use of reverse proxies
 and container to run Headscale.**
 
-Please have a look at the [`documentation`](https://headscale.net/).
+Please have a look at the [`documentation`](https://headscale.net/stable/).
+
+For NixOS users, a module is available in [`nix/`](./nix/).
 
 ## Talks
 
@@ -100,7 +85,7 @@ Please read the [CONTRIBUTING.md](./CONTRIBUTING.md) file.
 ### Requirements
 
 To contribute to headscale you would need the latest version of [Go](https://golang.org)
-and [Buf](https://buf.build)(Protobuf generator).
+and [Buf](https://buf.build) (Protobuf generator).
 
 We recommend using [Nix](https://nixos.org/) to setup a development environment. This can
 be done with `nix develop`, which will install the tools and give you a shell.
@@ -155,16 +140,31 @@ make test
 
 To build the program:
 
-```shell
-nix build
-```
-
-or
-
 ```shell
 make build
 ```
 
+### Development workflow
+
+We recommend using Nix for dependency management to ensure you have all required tools. If you prefer to manage dependencies yourself, you can use Make directly:
+
+**With Nix (recommended):**
+
+```shell
+nix develop
+make test
+make build
+```
+
+**With your own dependencies:**
+
+```shell
+make test
+make build
+```
+
+The Makefile will warn you if any required tools are missing and suggest running `nix develop`. Run `make help` to see all available targets.
+
 ## Contributors
 
 <a href="https://github.com/juanfont/headscale/graphs/contributors">

cmd/headscale/cli/api_key.go

@@ -54,7 +54,7 @@ var listAPIKeys = &cobra.Command{
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -67,14 +67,10 @@ var listAPIKeys = &cobra.Command{
 				fmt.Sprintf("Error getting the list of keys: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		if output != "" {
 			SuccessOutput(response.GetApiKeys(), "", output)
-
-			return
 		}
 
 		tableData := pterm.TableData{
@@ -102,8 +98,6 @@ var listAPIKeys = &cobra.Command{
 				fmt.Sprintf("Failed to render pterm table: %s", err),
 				output,
 			)
-
-			return
 		}
 	},
 }
@@ -119,9 +113,6 @@ If you loose a key, create a new one and revoke (expire) the old one.`,
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		log.Trace().
-			Msg("Preparing to create ApiKey")
-
 		request := &v1.CreateApiKeyRequest{}
 
 		durationStr, _ := cmd.Flags().GetString("expiration")
@@ -133,19 +124,13 @@ If you loose a key, create a new one and revoke (expire) the old one.`,
 				fmt.Sprintf("Could not parse duration: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		expiration := time.Now().UTC().Add(time.Duration(duration))
 
-		log.Trace().
-			Dur("expiration", time.Duration(duration)).
-			Msg("expiration has been set")
-
 		request.Expiration = timestamppb.New(expiration)
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -156,8 +141,6 @@ If you loose a key, create a new one and revoke (expire) the old one.`,
 				fmt.Sprintf("Cannot create Api Key: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetApiKey(), response.GetApiKey(), output)
@@ -178,11 +161,9 @@ var expireAPIKeyCmd = &cobra.Command{
 				fmt.Sprintf("Error getting prefix from CLI flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -197,8 +178,6 @@ var expireAPIKeyCmd = &cobra.Command{
 				fmt.Sprintf("Cannot expire Api Key: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response, "Key expired", output)
@@ -219,11 +198,9 @@ var deleteAPIKeyCmd = &cobra.Command{
 				fmt.Sprintf("Error getting prefix from CLI flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -238,8 +215,6 @@ var deleteAPIKeyCmd = &cobra.Command{
 				fmt.Sprintf("Cannot delete Api Key: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response, "Key deleted", output)

cmd/headscale/cli/configtest.go

@@ -14,7 +14,7 @@ var configTestCmd = &cobra.Command{
 	Short: "Test the configuration.",
 	Long:  "Run a test of the configuration and exit.",
 	Run: func(cmd *cobra.Command, args []string) {
-		_, err := getHeadscaleApp()
+		_, err := newHeadscaleServerWithConfig()
 		if err != nil {
 			log.Fatal().Caller().Err(err).Msg("Error initializing")
 		}

cmd/headscale/cli/debug.go

@@ -4,14 +4,10 @@ import (
 	"fmt"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
-	"tailscale.com/types/key"
-)
-
-const (
-	errPreAuthKeyMalformed = Error("key is malformed. expected 64 hex characters with `nodekey` prefix")
 )
 
 // Error is used to compare errors as per https://dave.cheney.net/2016/04/07/constant-errors
@@ -64,11 +60,9 @@ var createNodeCmd = &cobra.Command{
 		user, err := cmd.Flags().GetString("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -79,31 +73,24 @@ var createNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error getting node from flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		machineKey, err := cmd.Flags().GetString("key")
+		registrationID, err := cmd.Flags().GetString("key")
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf("Error getting key from flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		var mkey key.MachinePublic
-		err = mkey.UnmarshalText([]byte(machineKey))
+		_, err = types.RegistrationIDFromString(registrationID)
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf("Failed to parse machine key from flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		routes, err := cmd.Flags().GetStringSlice("route")
@@ -113,12 +100,10 @@ var createNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error getting routes from flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		request := &v1.DebugCreateNodeRequest{
-			Key:    machineKey,
+			Key:    registrationID,
 			Name:   name,
 			User:   user,
 			Routes: routes,
@@ -128,11 +113,9 @@ var createNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot create node: %s", status.Convert(err).Message()),
+				"Cannot create node: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetNode(), "Node created", output)

cmd/headscale/cli/health.go

@@ -0,0 +1,29 @@
+package cli
+
+import (
+	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/spf13/cobra"
+)
+
+func init() {
+	rootCmd.AddCommand(healthCmd)
+}
+
+var healthCmd = &cobra.Command{
+	Use:   "health",
+	Short: "Check the health of the Headscale server",
+	Long:  "Check the health of the Headscale server. This command will return an exit code of 0 if the server is healthy, or 1 if it is not.",
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		response, err := client.Health(ctx, &v1.HealthRequest{})
+		if err != nil {
+			ErrorOutput(err, "Error checking health", output)
+		}
+
+		SuccessOutput(response, "", output)
+	},
+}

cmd/headscale/cli/mockoidc.go

@@ -1,8 +1,11 @@
 package cli
 
 import (
+	"encoding/json"
+	"errors"
 	"fmt"
 	"net"
+	"net/http"
 	"os"
 	"strconv"
 	"time"
@@ -64,6 +67,19 @@ func mockOIDC() error {
 		accessTTL = newTTL
 	}
 
+	userStr := os.Getenv("MOCKOIDC_USERS")
+	if userStr == "" {
+		return errors.New("MOCKOIDC_USERS not defined")
+	}
+
+	var users []mockoidc.MockUser
+	err := json.Unmarshal([]byte(userStr), &users)
+	if err != nil {
+		return fmt.Errorf("unmarshalling users: %w", err)
+	}
+
+	log.Info().Interface("users", users).Msg("loading users from JSON")
+
 	log.Info().Msgf("Access token TTL: %s", accessTTL)
 
 	port, err := strconv.Atoi(portStr)
@@ -71,7 +87,7 @@ func mockOIDC() error {
 		return err
 	}
 
-	mock, err := getMockOIDC(clientID, clientSecret)
+	mock, err := getMockOIDC(clientID, clientSecret, users)
 	if err != nil {
 		return err
 	}
@@ -93,12 +109,18 @@ func mockOIDC() error {
 	return nil
 }
 
-func getMockOIDC(clientID string, clientSecret string) (*mockoidc.MockOIDC, error) {
+func getMockOIDC(clientID string, clientSecret string, users []mockoidc.MockUser) (*mockoidc.MockOIDC, error) {
 	keypair, err := mockoidc.NewKeypair(nil)
 	if err != nil {
 		return nil, err
 	}
 
+	userQueue := mockoidc.UserQueue{}
+
+	for _, user := range users {
+		userQueue.Push(&user)
+	}
+
 	mock := mockoidc.MockOIDC{
 		ClientID:                      clientID,
 		ClientSecret:                  clientSecret,
@@ -107,9 +129,19 @@ func getMockOIDC(clientID string, clientSecret string) (*mockoidc.MockOIDC, erro
 		CodeChallengeMethodsSupported: []string{"plain", "S256"},
 		Keypair:                       keypair,
 		SessionStore:                  mockoidc.NewSessionStore(),
-		UserQueue:                     &mockoidc.UserQueue{},
+		UserQueue:                     &userQueue,
 		ErrorQueue:                    &mockoidc.ErrorQueue{},
 	}
 
+	mock.AddMiddleware(func(h http.Handler) http.Handler {
+		return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+			log.Info().Msgf("Request: %+v", r)
+			h.ServeHTTP(w, r)
+			if r.Response != nil {
+				log.Info().Msgf("Response: %+v", r.Response)
+			}
+		})
+	})
+
 	return &mock, nil
 }

cmd/headscale/cli/nodes.go

@@ -9,12 +9,13 @@ import (
 	"strings"
 	"time"
 
-	survey "github.com/AlecAivazis/survey/v2"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/pterm/pterm"
+	"github.com/samber/lo"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
+	"google.golang.org/protobuf/types/known/timestamppb"
 	"tailscale.com/types/key"
 )
 
@@ -27,9 +28,11 @@ func init() {
 	listNodesNamespaceFlag := listNodesCmd.Flags().Lookup("namespace")
 	listNodesNamespaceFlag.Deprecated = deprecateNamespaceMessage
 	listNodesNamespaceFlag.Hidden = true
-
 	nodeCmd.AddCommand(listNodesCmd)
 
+	listNodeRoutesCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
+	nodeCmd.AddCommand(listNodeRoutesCmd)
+
 	registerNodeCmd.Flags().StringP("user", "u", "", "User")
 
 	registerNodeCmd.Flags().StringP("namespace", "n", "", "User")
@@ -39,66 +42,47 @@ func init() {
 
 	err := registerNodeCmd.MarkFlagRequired("user")
 	if err != nil {
-		log.Fatalf(err.Error())
+		log.Fatal(err.Error())
 	}
 	registerNodeCmd.Flags().StringP("key", "k", "", "Key")
 	err = registerNodeCmd.MarkFlagRequired("key")
 	if err != nil {
-		log.Fatalf(err.Error())
+		log.Fatal(err.Error())
 	}
 	nodeCmd.AddCommand(registerNodeCmd)
 
 	expireNodeCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
+	expireNodeCmd.Flags().StringP("expiry", "e", "", "Set expire to (RFC3339 format, e.g. 2025-08-27T10:00:00Z), or leave empty to expire immediately.")
 	err = expireNodeCmd.MarkFlagRequired("identifier")
 	if err != nil {
-		log.Fatalf(err.Error())
+		log.Fatal(err.Error())
 	}
 	nodeCmd.AddCommand(expireNodeCmd)
 
 	renameNodeCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
 	err = renameNodeCmd.MarkFlagRequired("identifier")
 	if err != nil {
-		log.Fatalf(err.Error())
+		log.Fatal(err.Error())
 	}
 	nodeCmd.AddCommand(renameNodeCmd)
 
 	deleteNodeCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
 	err = deleteNodeCmd.MarkFlagRequired("identifier")
 	if err != nil {
-		log.Fatalf(err.Error())
+		log.Fatal(err.Error())
 	}
 	nodeCmd.AddCommand(deleteNodeCmd)
 
-	moveNodeCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
-
-	err = moveNodeCmd.MarkFlagRequired("identifier")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-
-	moveNodeCmd.Flags().StringP("user", "u", "", "New user")
-
-	moveNodeCmd.Flags().StringP("namespace", "n", "", "User")
-	moveNodeNamespaceFlag := moveNodeCmd.Flags().Lookup("namespace")
-	moveNodeNamespaceFlag.Deprecated = deprecateNamespaceMessage
-	moveNodeNamespaceFlag.Hidden = true
-
-	err = moveNodeCmd.MarkFlagRequired("user")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-	nodeCmd.AddCommand(moveNodeCmd)
-
 	tagCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
-
-	err = tagCmd.MarkFlagRequired("identifier")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-	tagCmd.Flags().
-		StringSliceP("tags", "t", []string{}, "List of tags to add to the node")
+	tagCmd.MarkFlagRequired("identifier")
+	tagCmd.Flags().StringSliceP("tags", "t", []string{}, "List of tags to add to the node")
 	nodeCmd.AddCommand(tagCmd)
 
+	approveRoutesCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
+	approveRoutesCmd.MarkFlagRequired("identifier")
+	approveRoutesCmd.Flags().StringSliceP("routes", "r", []string{}, `List of routes that will be approved (comma-separated, e.g. "10.0.0.0/8,192.168.0.0/24" or empty string to remove all approved routes)`)
+	nodeCmd.AddCommand(approveRoutesCmd)
+
 	nodeCmd.AddCommand(backfillNodeIPsCmd)
 }
 
@@ -116,27 +100,23 @@ var registerNodeCmd = &cobra.Command{
 		user, err := cmd.Flags().GetString("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
-		machineKey, err := cmd.Flags().GetString("key")
+		registrationID, err := cmd.Flags().GetString("key")
 		if err != nil {
 			ErrorOutput(
 				err,
 				fmt.Sprintf("Error getting node key from flag: %s", err),
 				output,
 			)
-
-			return
 		}
 
 		request := &v1.RegisterNodeRequest{
-			Key:  machineKey,
+			Key:  registrationID,
 			User: user,
 		}
 
@@ -150,8 +130,6 @@ var registerNodeCmd = &cobra.Command{
 				),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(
@@ -169,17 +147,13 @@ var listNodesCmd = &cobra.Command{
 		user, err := cmd.Flags().GetString("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 		showTags, err := cmd.Flags().GetBool("tags")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting tags flag: %s", err), output)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -191,24 +165,18 @@ var listNodesCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot get nodes: %s", status.Convert(err).Message()),
+				"Cannot get nodes: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		if output != "" {
 			SuccessOutput(response.GetNodes(), "", output)
-
-			return
 		}
 
 		tableData, err := nodesToPtables(user, showTags, response.GetNodes())
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error converting to table: %s", err), output)
-
-			return
 		}
 
 		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
@@ -218,9 +186,72 @@ var listNodesCmd = &cobra.Command{
 				fmt.Sprintf("Failed to render pterm table: %s", err),
 				output,
 			)
+		}
+	},
+}
 
+var listNodeRoutesCmd = &cobra.Command{
+	Use:     "list-routes",
+	Short:   "List routes available on nodes",
+	Aliases: []string{"lsr", "routes"},
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		identifier, err := cmd.Flags().GetUint64("identifier")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error converting ID to integer: %s", err),
+				output,
+			)
+		}
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.ListNodesRequest{}
+
+		response, err := client.ListNodes(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				"Cannot get nodes: "+status.Convert(err).Message(),
+				output,
+			)
+		}
+
+		nodes := response.GetNodes()
+		if identifier != 0 {
+			for _, node := range response.GetNodes() {
+				if node.GetId() == identifier {
+					nodes = []*v1.Node{node}
+					break
+				}
+			}
+		}
+
+		nodes = lo.Filter(nodes, func(n *v1.Node, _ int) bool {
+			return (n.GetSubnetRoutes() != nil && len(n.GetSubnetRoutes()) > 0) || (n.GetApprovedRoutes() != nil && len(n.GetApprovedRoutes()) > 0) || (n.GetAvailableRoutes() != nil && len(n.GetAvailableRoutes()) > 0)
+		})
+
+		if output != "" {
+			SuccessOutput(nodes, "", output)
 			return
 		}
+
+		tableData, err := nodeRoutesToPtables(nodes)
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error converting to table: %s", err), output)
+		}
+
+		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Failed to render pterm table: %s", err),
+				output,
+			)
+		}
 	},
 }
 
@@ -239,16 +270,39 @@ var expireNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
+		}
+
+		expiry, err := cmd.Flags().GetString("expiry")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error converting expiry to string: %s", err),
+				output,
+			)
 
 			return
 		}
+		expiryTime := time.Now()
+		if expiry != "" {
+			expiryTime, err = time.Parse(time.RFC3339, expiry)
+			if err != nil {
+				ErrorOutput(
+					err,
+					fmt.Sprintf("Error converting expiry to string: %s", err),
+					output,
+				)
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+				return
+			}
+		}
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
 		request := &v1.ExpireNodeRequest{
 			NodeId: identifier,
+			Expiry: timestamppb.New(expiryTime),
 		}
 
 		response, err := client.ExpireNode(ctx, request)
@@ -261,8 +315,6 @@ var expireNodeCmd = &cobra.Command{
 				),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetNode(), "Node expired", output)
@@ -282,11 +334,9 @@ var renameNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -309,8 +359,6 @@ var renameNodeCmd = &cobra.Command{
 				),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetNode(), "Node renamed", output)
@@ -331,11 +379,9 @@ var deleteNodeCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -347,14 +393,9 @@ var deleteNodeCmd = &cobra.Command{
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Error getting node node: %s",
-					status.Convert(err).Message(),
-				),
+				"Error getting node node: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		deleteRequest := &v1.DeleteNodeRequest{
@@ -364,16 +405,10 @@ var deleteNodeCmd = &cobra.Command{
 		confirm := false
 		force, _ := cmd.Flags().GetBool("force")
 		if !force {
-			prompt := &survey.Confirm{
-				Message: fmt.Sprintf(
-					"Do you want to remove the node %s?",
-					getResponse.GetNode().GetName(),
-				),
-			}
-			err = survey.AskOne(prompt, &confirm)
-			if err != nil {
-				return
-			}
+			confirm = util.YesNo(fmt.Sprintf(
+				"Do you want to remove the node %s?",
+				getResponse.GetNode().GetName(),
+			))
 		}
 
 		if confirm || force {
@@ -386,14 +421,9 @@ var deleteNodeCmd = &cobra.Command{
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Error deleting node: %s",
-						status.Convert(err).Message(),
-					),
+					"Error deleting node: "+status.Convert(err).Message(),
 					output,
 				)
-
-				return
 			}
 			SuccessOutput(
 				map[string]string{"Result": "Node deleted"},
@@ -406,80 +436,6 @@ var deleteNodeCmd = &cobra.Command{
 	},
 }
 
-var moveNodeCmd = &cobra.Command{
-	Use:     "move",
-	Short:   "Move node to another user",
-	Aliases: []string{"mv"},
-	Run: func(cmd *cobra.Command, args []string) {
-		output, _ := cmd.Flags().GetString("output")
-
-		identifier, err := cmd.Flags().GetUint64("identifier")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error converting ID to integer: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		user, err := cmd.Flags().GetString("user")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error getting user: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
-
-		getRequest := &v1.GetNodeRequest{
-			NodeId: identifier,
-		}
-
-		_, err = client.GetNode(ctx, getRequest)
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf(
-					"Error getting node: %s",
-					status.Convert(err).Message(),
-				),
-				output,
-			)
-
-			return
-		}
-
-		moveRequest := &v1.MoveNodeRequest{
-			NodeId: identifier,
-			User:   user,
-		}
-
-		moveResponse, err := client.MoveNode(ctx, moveRequest)
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf(
-					"Error moving node: %s",
-					status.Convert(err).Message(),
-				),
-				output,
-			)
-
-			return
-		}
-
-		SuccessOutput(moveResponse.GetNode(), "Node moved to another user", output)
-	},
-}
-
 var backfillNodeIPsCmd = &cobra.Command{
 	Use:   "backfillips",
 	Short: "Backfill IPs missing from nodes",
@@ -496,34 +452,27 @@ If you remove IPv4 or IPv6 prefixes from the config,
 it can be run to remove the IPs that should no longer
 be assigned to nodes.`,
 	Run: func(cmd *cobra.Command, args []string) {
-		var err error
 		output, _ := cmd.Flags().GetString("output")
 
 		confirm := false
-		prompt := &survey.Confirm{
-			Message: "Are you sure that you want to assign/remove IPs to/from nodes?",
+
+		force, _ := cmd.Flags().GetBool("force")
+		if !force {
+			confirm = util.YesNo("Are you sure that you want to assign/remove IPs to/from nodes?")
 		}
-		err = survey.AskOne(prompt, &confirm)
-		if err != nil {
-			return
-		}
-		if confirm {
-			ctx, client, conn, cancel := getHeadscaleCLIClient()
+
+		if confirm || force {
+			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 			defer cancel()
 			defer conn.Close()
 
-			changes, err := client.BackfillNodeIPs(ctx, &v1.BackfillNodeIPsRequest{Confirmed: confirm})
+			changes, err := client.BackfillNodeIPs(ctx, &v1.BackfillNodeIPsRequest{Confirmed: confirm || force})
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Error backfilling IPs: %s",
-						status.Convert(err).Message(),
-					),
+					"Error backfilling IPs: "+status.Convert(err).Message(),
 					output,
 				)
-
-				return
 			}
 
 			SuccessOutput(changes, "Node IPs backfilled successfully", output)
@@ -613,23 +562,26 @@ func nodesToPtables(
 
 		var forcedTags string
 		for _, tag := range node.GetForcedTags() {
-			forcedTags += "," + tag
+			forcedTags += "\n" + tag
 		}
-		forcedTags = strings.TrimLeft(forcedTags, ",")
+
+		forcedTags = strings.TrimLeft(forcedTags, "\n")
 		var invalidTags string
 		for _, tag := range node.GetInvalidTags() {
 			if !slices.Contains(node.GetForcedTags(), tag) {
-				invalidTags += "," + pterm.LightRed(tag)
+				invalidTags += "\n" + pterm.LightRed(tag)
 			}
 		}
-		invalidTags = strings.TrimLeft(invalidTags, ",")
+
+		invalidTags = strings.TrimLeft(invalidTags, "\n")
 		var validTags string
 		for _, tag := range node.GetValidTags() {
 			if !slices.Contains(node.GetForcedTags(), tag) {
-				validTags += "," + pterm.LightGreen(tag)
+				validTags += "\n" + pterm.LightGreen(tag)
 			}
 		}
-		validTags = strings.TrimLeft(validTags, ",")
+
+		validTags = strings.TrimLeft(validTags, "\n")
 
 		var user string
 		if currentUser == "" || (currentUser == node.GetUser().GetName()) {
@@ -675,13 +627,42 @@ func nodesToPtables(
 	return tableData, nil
 }
 
+func nodeRoutesToPtables(
+	nodes []*v1.Node,
+) (pterm.TableData, error) {
+	tableHeader := []string{
+		"ID",
+		"Hostname",
+		"Approved",
+		"Available",
+		"Serving (Primary)",
+	}
+	tableData := pterm.TableData{tableHeader}
+
+	for _, node := range nodes {
+		nodeData := []string{
+			strconv.FormatUint(node.GetId(), util.Base10),
+			node.GetGivenName(),
+			strings.Join(node.GetApprovedRoutes(), "\n"),
+			strings.Join(node.GetAvailableRoutes(), "\n"),
+			strings.Join(node.GetSubnetRoutes(), "\n"),
+		}
+		tableData = append(
+			tableData,
+			nodeData,
+		)
+	}
+
+	return tableData, nil
+}
+
 var tagCmd = &cobra.Command{
 	Use:     "tag",
 	Short:   "Manage the tags of a node",
 	Aliases: []string{"tags", "t"},
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -693,8 +674,6 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error converting ID to integer: %s", err),
 				output,
 			)
-
-			return
 		}
 		tagsToSet, err := cmd.Flags().GetStringSlice("tags")
 		if err != nil {
@@ -703,8 +682,6 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error retrieving list of tags to add to node, %v", err),
 				output,
 			)
-
-			return
 		}
 
 		// Sending tags to node
@@ -719,8 +696,57 @@ var tagCmd = &cobra.Command{
 				fmt.Sprintf("Error while sending tags to headscale: %s", err),
 				output,
 			)
-
-			return
+		}
+
+		if resp != nil {
+			SuccessOutput(
+				resp.GetNode(),
+				"Node updated",
+				output,
+			)
+		}
+	},
+}
+
+var approveRoutesCmd = &cobra.Command{
+	Use:   "approve-routes",
+	Short: "Manage the approved routes of a node",
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		// retrieve flags from CLI
+		identifier, err := cmd.Flags().GetUint64("identifier")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error converting ID to integer: %s", err),
+				output,
+			)
+		}
+		routes, err := cmd.Flags().GetStringSlice("routes")
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error retrieving list of routes to add to node, %v", err),
+				output,
+			)
+		}
+
+		// Sending routes to node
+		request := &v1.SetApprovedRoutesRequest{
+			NodeId: identifier,
+			Routes: routes,
+		}
+		resp, err := client.SetApprovedRoutes(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Error while sending routes to headscale: %s", err),
+				output,
+			)
 		}
 
 		if resp != nil {

cmd/headscale/cli/policy.go

@@ -1,23 +1,42 @@
 package cli
 
 import (
+	"fmt"
 	"io"
 	"os"
 
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/db"
+	"github.com/juanfont/headscale/hscontrol/policy"
+	"github.com/juanfont/headscale/hscontrol/types"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"tailscale.com/types/views"
+)
+
+const (
+	bypassFlag = "bypass-grpc-and-access-database-directly"
 )
 
 func init() {
 	rootCmd.AddCommand(policyCmd)
+
+	getPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
 	policyCmd.AddCommand(getPolicy)
 
 	setPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
 	if err := setPolicy.MarkFlagRequired("file"); err != nil {
 		log.Fatal().Err(err).Msg("")
 	}
+	setPolicy.Flags().BoolP(bypassFlag, "", false, "Uses the headscale config to directly access the database, bypassing gRPC and does not require the server to be running")
 	policyCmd.AddCommand(setPolicy)
+
+	checkPolicy.Flags().StringP("file", "f", "", "Path to a policy file in HuJSON format")
+	if err := checkPolicy.MarkFlagRequired("file"); err != nil {
+		log.Fatal().Err(err).Msg("")
+	}
+	policyCmd.AddCommand(checkPolicy)
 }
 
 var policyCmd = &cobra.Command{
@@ -30,21 +49,59 @@ var getPolicy = &cobra.Command{
 	Short:   "Print the current ACL Policy",
 	Aliases: []string{"show", "view", "fetch"},
 	Run: func(cmd *cobra.Command, args []string) {
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
+		output, _ := cmd.Flags().GetString("output")
+		var policy string
+		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
+			confirm := false
+			force, _ := cmd.Flags().GetBool("force")
+			if !force {
+				confirm = util.YesNo("DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?")
+			}
 
-		request := &v1.GetPolicyRequest{}
+			if !confirm && !force {
+				ErrorOutput(nil, "Aborting command", output)
+				return
+			}
 
-		response, err := client.GetPolicy(ctx, request)
-		if err != nil {
-			log.Fatal().Err(err).Msg("Failed to get the policy")
+			cfg, err := types.LoadServerConfig()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading config: %s", err), output)
+			}
 
-			return
+			d, err := db.NewHeadscaleDatabase(
+				cfg.Database,
+				cfg.BaseDomain,
+				nil,
+			)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to open database: %s", err), output)
+			}
+
+			pol, err := d.GetPolicy()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading Policy from database: %s", err), output)
+			}
+
+			policy = pol.Data
+		} else {
+			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+			defer cancel()
+			defer conn.Close()
+
+			request := &v1.GetPolicyRequest{}
+
+			response, err := client.GetPolicy(ctx, request)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading ACL Policy: %s", err), output)
+			}
+
+			policy = response.GetPolicy()
 		}
 
 		// TODO(pallabpain): Maybe print this better?
-		SuccessOutput("", response.GetPolicy(), "hujson")
+		// This does not pass output as we dont support yaml, json or json-line
+		// output for this command. It is HuJSON already.
+		SuccessOutput("", policy, "")
 	},
 }
 
@@ -56,35 +113,100 @@ var setPolicy = &cobra.Command{
 	This command only works when the acl.policy_mode is set to "db", and the policy will be stored in the database.`,
 	Aliases: []string{"put", "update"},
 	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
 		policyPath, _ := cmd.Flags().GetString("file")
 
 		f, err := os.Open(policyPath)
 		if err != nil {
-			log.Fatal().Err(err).Msg("Error opening the policy file")
-
-			return
+			ErrorOutput(err, fmt.Sprintf("Error opening the policy file: %s", err), output)
 		}
 		defer f.Close()
 
 		policyBytes, err := io.ReadAll(f)
 		if err != nil {
-			log.Fatal().Err(err).Msg("Error reading the policy file")
-
-			return
+			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
 		}
 
-		request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
+		if bypass, _ := cmd.Flags().GetBool(bypassFlag); bypass {
+			confirm := false
+			force, _ := cmd.Flags().GetBool("force")
+			if !force {
+				confirm = util.YesNo("DO NOT run this command if an instance of headscale is running, are you sure headscale is not running?")
+			}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
+			if !confirm && !force {
+				ErrorOutput(nil, "Aborting command", output)
+				return
+			}
 
-		if _, err := client.SetPolicy(ctx, request); err != nil {
-			log.Fatal().Err(err).Msg("Failed to set ACL Policy")
+			cfg, err := types.LoadServerConfig()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed loading config: %s", err), output)
+			}
 
-			return
+			d, err := db.NewHeadscaleDatabase(
+				cfg.Database,
+				cfg.BaseDomain,
+				nil,
+			)
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to open database: %s", err), output)
+			}
+
+			users, err := d.ListUsers()
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to load users for policy validation: %s", err), output)
+			}
+
+			_, err = policy.NewPolicyManager(policyBytes, users, views.Slice[types.NodeView]{})
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Error parsing the policy file: %s", err), output)
+				return
+			}
+
+			_, err = d.SetPolicy(string(policyBytes))
+			if err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
+			}
+		} else {
+			request := &v1.SetPolicyRequest{Policy: string(policyBytes)}
+
+			ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+			defer cancel()
+			defer conn.Close()
+
+			if _, err := client.SetPolicy(ctx, request); err != nil {
+				ErrorOutput(err, fmt.Sprintf("Failed to set ACL Policy: %s", err), output)
+			}
 		}
 
 		SuccessOutput(nil, "Policy updated.", "")
 	},
 }
+
+var checkPolicy = &cobra.Command{
+	Use:   "check",
+	Short: "Check the Policy file for errors",
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		policyPath, _ := cmd.Flags().GetString("file")
+
+		f, err := os.Open(policyPath)
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error opening the policy file: %s", err), output)
+		}
+		defer f.Close()
+
+		policyBytes, err := io.ReadAll(f)
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error reading the policy file: %s", err), output)
+		}
+
+		_, err = policy.NewPolicyManager(policyBytes, nil, views.Slice[types.NodeView]{})
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error parsing the policy file: %s", err), output)
+		}
+
+		SuccessOutput(nil, "Policy is valid", "")
+	},
+}

cmd/headscale/cli/preauthkeys.go

@@ -20,7 +20,7 @@ const (
 
 func init() {
 	rootCmd.AddCommand(preauthkeysCmd)
-	preauthkeysCmd.PersistentFlags().StringP("user", "u", "", "User")
+	preauthkeysCmd.PersistentFlags().Uint64P("user", "u", 0, "User identifier (ID)")
 
 	preauthkeysCmd.PersistentFlags().StringP("namespace", "n", "", "User")
 	pakNamespaceFlag := preauthkeysCmd.PersistentFlags().Lookup("namespace")
@@ -34,6 +34,7 @@ func init() {
 	preauthkeysCmd.AddCommand(listPreAuthKeys)
 	preauthkeysCmd.AddCommand(createPreAuthKeyCmd)
 	preauthkeysCmd.AddCommand(expirePreAuthKeyCmd)
+	preauthkeysCmd.AddCommand(deletePreAuthKeyCmd)
 	createPreAuthKeyCmd.PersistentFlags().
 		Bool("reusable", false, "Make the preauthkey reusable")
 	createPreAuthKeyCmd.PersistentFlags().
@@ -57,14 +58,12 @@ var listPreAuthKeys = &cobra.Command{
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		user, err := cmd.Flags().GetString("user")
+		user, err := cmd.Flags().GetUint64("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -85,14 +84,12 @@ var listPreAuthKeys = &cobra.Command{
 
 		if output != "" {
 			SuccessOutput(response.GetPreAuthKeys(), "", output)
-
-			return
 		}
 
 		tableData := pterm.TableData{
 			{
 				"ID",
-				"Key",
+				"Key/Prefix",
 				"Reusable",
 				"Ephemeral",
 				"Used",
@@ -110,13 +107,13 @@ var listPreAuthKeys = &cobra.Command{
 			aclTags := ""
 
 			for _, tag := range key.GetAclTags() {
-				aclTags += "," + tag
+				aclTags += "\n" + tag
 			}
 
-			aclTags = strings.TrimLeft(aclTags, ",")
+			aclTags = strings.TrimLeft(aclTags, "\n")
 
 			tableData = append(tableData, []string{
-				key.GetId(),
+				strconv.FormatUint(key.GetId(), 10),
 				key.GetKey(),
 				strconv.FormatBool(key.GetReusable()),
 				strconv.FormatBool(key.GetEphemeral()),
@@ -134,8 +131,6 @@ var listPreAuthKeys = &cobra.Command{
 				fmt.Sprintf("Failed to render pterm table: %s", err),
 				output,
 			)
-
-			return
 		}
 	},
 }
@@ -147,23 +142,15 @@ var createPreAuthKeyCmd = &cobra.Command{
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		user, err := cmd.Flags().GetString("user")
+		user, err := cmd.Flags().GetUint64("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 
 		reusable, _ := cmd.Flags().GetBool("reusable")
 		ephemeral, _ := cmd.Flags().GetBool("ephemeral")
 		tags, _ := cmd.Flags().GetStringSlice("tags")
 
-		log.Trace().
-			Bool("reusable", reusable).
-			Bool("ephemeral", ephemeral).
-			Str("user", user).
-			Msg("Preparing to create preauthkey")
-
 		request := &v1.CreatePreAuthKeyRequest{
 			User:      user,
 			Reusable:  reusable,
@@ -180,8 +167,6 @@ var createPreAuthKeyCmd = &cobra.Command{
 				fmt.Sprintf("Could not parse duration: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		expiration := time.Now().UTC().Add(time.Duration(duration))
@@ -192,7 +177,7 @@ var createPreAuthKeyCmd = &cobra.Command{
 
 		request.Expiration = timestamppb.New(expiration)
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -203,8 +188,6 @@ var createPreAuthKeyCmd = &cobra.Command{
 				fmt.Sprintf("Cannot create Pre Auth Key: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetPreAuthKey(), response.GetPreAuthKey().GetKey(), output)
@@ -224,14 +207,12 @@ var expirePreAuthKeyCmd = &cobra.Command{
 	},
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
-		user, err := cmd.Flags().GetString("user")
+		user, err := cmd.Flags().GetUint64("user")
 		if err != nil {
 			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
-
-			return
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -247,10 +228,48 @@ var expirePreAuthKeyCmd = &cobra.Command{
 				fmt.Sprintf("Cannot expire Pre Auth Key: %s\n", err),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response, "Key expired", output)
 	},
 }
+
+var deletePreAuthKeyCmd = &cobra.Command{
+	Use:     "delete KEY",
+	Short:   "Delete a preauthkey",
+	Aliases: []string{"del", "rm", "d"},
+	Args: func(cmd *cobra.Command, args []string) error {
+		if len(args) < 1 {
+			return errMissingParameter
+		}
+
+		return nil
+	},
+	Run: func(cmd *cobra.Command, args []string) {
+		output, _ := cmd.Flags().GetString("output")
+		user, err := cmd.Flags().GetUint64("user")
+		if err != nil {
+			ErrorOutput(err, fmt.Sprintf("Error getting user: %s", err), output)
+		}
+
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
+		defer cancel()
+		defer conn.Close()
+
+		request := &v1.DeletePreAuthKeyRequest{
+			User: user,
+			Key:  args[0],
+		}
+
+		response, err := client.DeletePreAuthKey(ctx, request)
+		if err != nil {
+			ErrorOutput(
+				err,
+				fmt.Sprintf("Cannot delete Pre Auth Key: %s\n", err),
+				output,
+			)
+		}
+
+		SuccessOutput(response, "Key deleted", output)
+	},
+}

cmd/headscale/cli/root.go

@@ -4,11 +4,14 @@ import (
 	"fmt"
 	"os"
 	"runtime"
+	"slices"
+	"strings"
 
 	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/rs/zerolog"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"github.com/spf13/viper"
 	"github.com/tcnksm/go-latest"
 )
 
@@ -24,6 +27,11 @@ func init() {
 		return
 	}
 
+	if slices.Contains(os.Args, "policy") && slices.Contains(os.Args, "check") {
+		zerolog.SetGlobalLevel(zerolog.Disabled)
+		return
+	}
+
 	cobra.OnInitialize(initConfig)
 	rootCmd.PersistentFlags().
 		StringVarP(&cfgFile, "config", "c", "", "config file (default is /etc/headscale/config.yaml)")
@@ -49,11 +57,6 @@ func initConfig() {
 		}
 	}
 
-	cfg, err := types.GetHeadscaleConfig()
-	if err != nil {
-		log.Fatal().Err(err).Msg("Failed to read headscale configuration")
-	}
-
 	machineOutput := HasMachineOutputFlag()
 
 	// If the user has requested a "node" readable format,
@@ -62,30 +65,71 @@ func initConfig() {
 		zerolog.SetGlobalLevel(zerolog.Disabled)
 	}
 
-	if cfg.Log.Format == types.JSONLogFormat {
+	logFormat := viper.GetString("log.format")
+	if logFormat == types.JSONLogFormat {
 		log.Logger = log.Output(os.Stdout)
 	}
 
-	if !cfg.DisableUpdateCheck && !machineOutput {
+	disableUpdateCheck := viper.GetBool("disable_check_updates")
+	if !disableUpdateCheck && !machineOutput {
+		versionInfo := types.GetVersionInfo()
 		if (runtime.GOOS == "linux" || runtime.GOOS == "darwin") &&
-			Version != "dev" {
+			!versionInfo.Dirty {
 			githubTag := &latest.GithubTag{
-				Owner:      "juanfont",
-				Repository: "headscale",
+				Owner:         "juanfont",
+				Repository:    "headscale",
+				TagFilterFunc: filterPreReleasesIfStable(func() string { return versionInfo.Version }),
 			}
-			res, err := latest.Check(githubTag, Version)
+			res, err := latest.Check(githubTag, versionInfo.Version)
 			if err == nil && res.Outdated {
 				//nolint
 				log.Warn().Msgf(
 					"An updated version of Headscale has been found (%s vs. your current %s). Check it out https://github.com/juanfont/headscale/releases\n",
 					res.Current,
-					Version,
+					versionInfo.Version,
 				)
 			}
 		}
 	}
 }
 
+var prereleases = []string{"alpha", "beta", "rc", "dev"}
+
+func isPreReleaseVersion(version string) bool {
+	for _, unstable := range prereleases {
+		if strings.Contains(version, unstable) {
+			return true
+		}
+	}
+	return false
+}
+
+// filterPreReleasesIfStable returns a function that filters out
+// pre-release tags if the current version is stable.
+// If the current version is a pre-release, it does not filter anything.
+// versionFunc is a function that returns the current version string, it is
+// a func for testability.
+func filterPreReleasesIfStable(versionFunc func() string) func(string) bool {
+	return func(tag string) bool {
+		version := versionFunc()
+
+		// If we are on a pre-release version, then we do not filter anything
+		// as we want to recommend the user the latest pre-release.
+		if isPreReleaseVersion(version) {
+			return false
+		}
+
+		// If we are on a stable release, filter out pre-releases.
+		for _, ignore := range prereleases {
+			if strings.Contains(tag, ignore) {
+				return true
+			}
+		}
+
+		return false
+	}
+}
+
 var rootCmd = &cobra.Command{
 	Use:   "headscale",
 	Short: "headscale - a Tailscale control server",

cmd/headscale/cli/root_test.go

@@ -0,0 +1,293 @@
+package cli
+
+import (
+	"testing"
+)
+
+func TestFilterPreReleasesIfStable(t *testing.T) {
+	tests := []struct {
+		name           string
+		currentVersion string
+		tag            string
+		expectedFilter bool
+		description    string
+	}{
+		{
+			name:           "stable version filters alpha tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: true,
+			description:    "When on stable release, alpha tags should be filtered",
+		},
+		{
+			name:           "stable version filters beta tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-beta.2",
+			expectedFilter: true,
+			description:    "When on stable release, beta tags should be filtered",
+		},
+		{
+			name:           "stable version filters rc tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: true,
+			description:    "When on stable release, rc tags should be filtered",
+		},
+		{
+			name:           "stable version allows stable tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on stable release, stable tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows alpha tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-alpha.2",
+			expectedFilter: false,
+			description:    "When on alpha release, alpha tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows beta tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-beta.1",
+			expectedFilter: false,
+			description:    "When on alpha release, beta tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows rc tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: false,
+			description:    "When on alpha release, rc tags should not be filtered",
+		},
+		{
+			name:           "alpha version allows stable tag",
+			currentVersion: "0.23.0-alpha.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on alpha release, stable tags should not be filtered",
+		},
+		{
+			name:           "beta version allows alpha tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "When on beta release, alpha tags should not be filtered",
+		},
+		{
+			name:           "beta version allows beta tag",
+			currentVersion: "0.23.0-beta.2",
+			tag:            "v0.24.0-beta.3",
+			expectedFilter: false,
+			description:    "When on beta release, beta tags should not be filtered",
+		},
+		{
+			name:           "beta version allows rc tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0-rc.1",
+			expectedFilter: false,
+			description:    "When on beta release, rc tags should not be filtered",
+		},
+		{
+			name:           "beta version allows stable tag",
+			currentVersion: "0.23.0-beta.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on beta release, stable tags should not be filtered",
+		},
+		{
+			name:           "rc version allows alpha tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "When on rc release, alpha tags should not be filtered",
+		},
+		{
+			name:           "rc version allows beta tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0-beta.1",
+			expectedFilter: false,
+			description:    "When on rc release, beta tags should not be filtered",
+		},
+		{
+			name:           "rc version allows rc tag",
+			currentVersion: "0.23.0-rc.2",
+			tag:            "v0.24.0-rc.3",
+			expectedFilter: false,
+			description:    "When on rc release, rc tags should not be filtered",
+		},
+		{
+			name:           "rc version allows stable tag",
+			currentVersion: "0.23.0-rc.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on rc release, stable tags should not be filtered",
+		},
+		{
+			name:           "stable version with patch filters alpha",
+			currentVersion: "0.23.1",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: true,
+			description:    "Stable version with patch number should filter alpha tags",
+		},
+		{
+			name:           "stable version with patch allows stable",
+			currentVersion: "0.23.1",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "Stable version with patch number should allow stable tags",
+		},
+		{
+			name:           "tag with alpha substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-alpha.1",
+			expectedFilter: true,
+			description:    "Tags with alpha in version string should be filtered on stable",
+		},
+		{
+			name:           "tag with beta substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-beta.1",
+			expectedFilter: true,
+			description:    "Tags with beta in version string should be filtered on stable",
+		},
+		{
+			name:           "tag with rc substring in version number",
+			currentVersion: "0.23.0",
+			tag:            "v1.0.0-rc.1",
+			expectedFilter: true,
+			description:    "Tags with rc in version string should be filtered on stable",
+		},
+		{
+			name:           "empty tag on stable version",
+			currentVersion: "0.23.0",
+			tag:            "",
+			expectedFilter: false,
+			description:    "Empty tags should not be filtered",
+		},
+		{
+			name:           "dev version allows all tags",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0-alpha.1",
+			expectedFilter: false,
+			description:    "Dev versions should not filter any tags (pre-release allows all)",
+		},
+		{
+			name:           "stable version filters dev tag",
+			currentVersion: "0.23.0",
+			tag:            "v0.24.0-dev",
+			expectedFilter: true,
+			description:    "When on stable release, dev tags should be filtered",
+		},
+		{
+			name:           "dev version allows dev tag",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0-dev.1",
+			expectedFilter: false,
+			description:    "When on dev release, dev tags should not be filtered",
+		},
+		{
+			name:           "dev version allows stable tag",
+			currentVersion: "0.23.0-dev",
+			tag:            "v0.24.0",
+			expectedFilter: false,
+			description:    "When on dev release, stable tags should not be filtered",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := filterPreReleasesIfStable(func() string { return tt.currentVersion })(tt.tag)
+			if result != tt.expectedFilter {
+				t.Errorf("%s: got %v, want %v\nDescription: %s\nCurrent version: %s, Tag: %s",
+					tt.name,
+					result,
+					tt.expectedFilter,
+					tt.description,
+					tt.currentVersion,
+					tt.tag,
+				)
+			}
+		})
+	}
+}
+
+func TestIsPreReleaseVersion(t *testing.T) {
+	tests := []struct {
+		name        string
+		version     string
+		expected    bool
+		description string
+	}{
+		{
+			name:        "stable version",
+			version:     "0.23.0",
+			expected:    false,
+			description: "Stable version should not be pre-release",
+		},
+		{
+			name:        "alpha version",
+			version:     "0.23.0-alpha.1",
+			expected:    true,
+			description: "Alpha version should be pre-release",
+		},
+		{
+			name:        "beta version",
+			version:     "0.23.0-beta.1",
+			expected:    true,
+			description: "Beta version should be pre-release",
+		},
+		{
+			name:        "rc version",
+			version:     "0.23.0-rc.1",
+			expected:    true,
+			description: "RC version should be pre-release",
+		},
+		{
+			name:        "version with alpha substring",
+			version:     "0.23.0-alphabetical",
+			expected:    true,
+			description: "Version containing 'alpha' should be pre-release",
+		},
+		{
+			name:        "version with beta substring",
+			version:     "0.23.0-betamax",
+			expected:    true,
+			description: "Version containing 'beta' should be pre-release",
+		},
+		{
+			name:        "dev version",
+			version:     "0.23.0-dev",
+			expected:    true,
+			description: "Dev version should be pre-release",
+		},
+		{
+			name:        "empty version",
+			version:     "",
+			expected:    false,
+			description: "Empty version should not be pre-release",
+		},
+		{
+			name:        "version with patch number",
+			version:     "0.23.1",
+			expected:    false,
+			description: "Stable version with patch should not be pre-release",
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			result := isPreReleaseVersion(tt.version)
+			if result != tt.expected {
+				t.Errorf("%s: got %v, want %v\nDescription: %s\nVersion: %s",
+					tt.name,
+					result,
+					tt.expected,
+					tt.description,
+					tt.version,
+				)
+			}
+		})
+	}
+}

cmd/headscale/cli/routes.go

@@ -1,298 +0,0 @@
-package cli
-
-import (
-	"fmt"
-	"log"
-	"net/netip"
-	"strconv"
-
-	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
-	"github.com/juanfont/headscale/hscontrol/types"
-	"github.com/pterm/pterm"
-	"github.com/spf13/cobra"
-	"google.golang.org/grpc/status"
-)
-
-const (
-	Base10 = 10
-)
-
-func init() {
-	rootCmd.AddCommand(routesCmd)
-	listRoutesCmd.Flags().Uint64P("identifier", "i", 0, "Node identifier (ID)")
-	routesCmd.AddCommand(listRoutesCmd)
-
-	enableRouteCmd.Flags().Uint64P("route", "r", 0, "Route identifier (ID)")
-	err := enableRouteCmd.MarkFlagRequired("route")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-	routesCmd.AddCommand(enableRouteCmd)
-
-	disableRouteCmd.Flags().Uint64P("route", "r", 0, "Route identifier (ID)")
-	err = disableRouteCmd.MarkFlagRequired("route")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-	routesCmd.AddCommand(disableRouteCmd)
-
-	deleteRouteCmd.Flags().Uint64P("route", "r", 0, "Route identifier (ID)")
-	err = deleteRouteCmd.MarkFlagRequired("route")
-	if err != nil {
-		log.Fatalf(err.Error())
-	}
-	routesCmd.AddCommand(deleteRouteCmd)
-}
-
-var routesCmd = &cobra.Command{
-	Use:     "routes",
-	Short:   "Manage the routes of Headscale",
-	Aliases: []string{"r", "route"},
-}
-
-var listRoutesCmd = &cobra.Command{
-	Use:     "list",
-	Short:   "List all routes",
-	Aliases: []string{"ls", "show"},
-	Run: func(cmd *cobra.Command, args []string) {
-		output, _ := cmd.Flags().GetString("output")
-
-		machineID, err := cmd.Flags().GetUint64("identifier")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error getting machine id from flag: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
-
-		var routes []*v1.Route
-
-		if machineID == 0 {
-			response, err := client.GetRoutes(ctx, &v1.GetRoutesRequest{})
-			if err != nil {
-				ErrorOutput(
-					err,
-					fmt.Sprintf("Cannot get nodes: %s", status.Convert(err).Message()),
-					output,
-				)
-
-				return
-			}
-
-			if output != "" {
-				SuccessOutput(response.GetRoutes(), "", output)
-
-				return
-			}
-
-			routes = response.GetRoutes()
-		} else {
-			response, err := client.GetNodeRoutes(ctx, &v1.GetNodeRoutesRequest{
-				NodeId: machineID,
-			})
-			if err != nil {
-				ErrorOutput(
-					err,
-					fmt.Sprintf("Cannot get routes for node %d: %s", machineID, status.Convert(err).Message()),
-					output,
-				)
-
-				return
-			}
-
-			if output != "" {
-				SuccessOutput(response.GetRoutes(), "", output)
-
-				return
-			}
-
-			routes = response.GetRoutes()
-		}
-
-		tableData := routesToPtables(routes)
-		if err != nil {
-			ErrorOutput(err, fmt.Sprintf("Error converting to table: %s", err), output)
-
-			return
-		}
-
-		err = pterm.DefaultTable.WithHasHeader().WithData(tableData).Render()
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Failed to render pterm table: %s", err),
-				output,
-			)
-
-			return
-		}
-	},
-}
-
-var enableRouteCmd = &cobra.Command{
-	Use:   "enable",
-	Short: "Set a route as enabled",
-	Long:  `This command will make as enabled a given route.`,
-	Run: func(cmd *cobra.Command, args []string) {
-		output, _ := cmd.Flags().GetString("output")
-
-		routeID, err := cmd.Flags().GetUint64("route")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error getting machine id from flag: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
-
-		response, err := client.EnableRoute(ctx, &v1.EnableRouteRequest{
-			RouteId: routeID,
-		})
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Cannot enable route %d: %s", routeID, status.Convert(err).Message()),
-				output,
-			)
-
-			return
-		}
-
-		if output != "" {
-			SuccessOutput(response, "", output)
-
-			return
-		}
-	},
-}
-
-var disableRouteCmd = &cobra.Command{
-	Use:   "disable",
-	Short: "Set as disabled a given route",
-	Long:  `This command will make as disabled a given route.`,
-	Run: func(cmd *cobra.Command, args []string) {
-		output, _ := cmd.Flags().GetString("output")
-
-		routeID, err := cmd.Flags().GetUint64("route")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error getting machine id from flag: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
-
-		response, err := client.DisableRoute(ctx, &v1.DisableRouteRequest{
-			RouteId: routeID,
-		})
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Cannot disable route %d: %s", routeID, status.Convert(err).Message()),
-				output,
-			)
-
-			return
-		}
-
-		if output != "" {
-			SuccessOutput(response, "", output)
-
-			return
-		}
-	},
-}
-
-var deleteRouteCmd = &cobra.Command{
-	Use:   "delete",
-	Short: "Delete a given route",
-	Long:  `This command will delete a given route.`,
-	Run: func(cmd *cobra.Command, args []string) {
-		output, _ := cmd.Flags().GetString("output")
-
-		routeID, err := cmd.Flags().GetUint64("route")
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Error getting machine id from flag: %s", err),
-				output,
-			)
-
-			return
-		}
-
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
-		defer cancel()
-		defer conn.Close()
-
-		response, err := client.DeleteRoute(ctx, &v1.DeleteRouteRequest{
-			RouteId: routeID,
-		})
-		if err != nil {
-			ErrorOutput(
-				err,
-				fmt.Sprintf("Cannot delete route %d: %s", routeID, status.Convert(err).Message()),
-				output,
-			)
-
-			return
-		}
-
-		if output != "" {
-			SuccessOutput(response, "", output)
-
-			return
-		}
-	},
-}
-
-// routesToPtables converts the list of routes to a nice table.
-func routesToPtables(routes []*v1.Route) pterm.TableData {
-	tableData := pterm.TableData{{"ID", "Node", "Prefix", "Advertised", "Enabled", "Primary"}}
-
-	for _, route := range routes {
-		var isPrimaryStr string
-		prefix, err := netip.ParsePrefix(route.GetPrefix())
-		if err != nil {
-			log.Printf("Error parsing prefix %s: %s", route.GetPrefix(), err)
-
-			continue
-		}
-		if prefix == types.ExitRouteV4 || prefix == types.ExitRouteV6 {
-			isPrimaryStr = "-"
-		} else {
-			isPrimaryStr = strconv.FormatBool(route.GetIsPrimary())
-		}
-
-		tableData = append(tableData,
-			[]string{
-				strconv.FormatUint(route.GetId(), Base10),
-				route.GetNode().GetGivenName(),
-				route.GetPrefix(),
-				strconv.FormatBool(route.GetAdvertised()),
-				strconv.FormatBool(route.GetEnabled()),
-				isPrimaryStr,
-			})
-	}
-
-	return tableData
-}

cmd/headscale/cli/serve.go

@@ -1,8 +1,13 @@
 package cli
 
 import (
+	"errors"
+	"fmt"
+	"net/http"
+
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
+	"github.com/tailscale/squibble"
 )
 
 func init() {
@@ -16,14 +21,20 @@ var serveCmd = &cobra.Command{
 		return nil
 	},
 	Run: func(cmd *cobra.Command, args []string) {
-		app, err := getHeadscaleApp()
+		app, err := newHeadscaleServerWithConfig()
 		if err != nil {
+			var squibbleErr squibble.ValidationError
+			if errors.As(err, &squibbleErr) {
+				fmt.Printf("SQLite schema failed to validate:\n")
+				fmt.Println(squibbleErr.Diff)
+			}
+
 			log.Fatal().Caller().Err(err).Msg("Error initializing")
 		}
 
 		err = app.Serve()
-		if err != nil {
-			log.Fatal().Caller().Err(err).Msg("Error starting server")
+		if err != nil && !errors.Is(err, http.ErrServerClosed) {
+			log.Fatal().Caller().Err(err).Msg("Headscale ran into an error and had to shut down.")
 		}
 	},
 }

cmd/headscale/cli/users.go

@@ -3,21 +3,54 @@ package cli
 import (
 	"errors"
 	"fmt"
+	"net/url"
+	"strconv"
 
-	survey "github.com/AlecAivazis/survey/v2"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
+	"github.com/juanfont/headscale/hscontrol/util"
 	"github.com/pterm/pterm"
 	"github.com/rs/zerolog/log"
 	"github.com/spf13/cobra"
 	"google.golang.org/grpc/status"
 )
 
+func usernameAndIDFlag(cmd *cobra.Command) {
+	cmd.Flags().Int64P("identifier", "i", -1, "User identifier (ID)")
+	cmd.Flags().StringP("name", "n", "", "Username")
+}
+
+// usernameAndIDFromFlag returns the username and ID from the flags of the command.
+// If both are empty, it will exit the program with an error.
+func usernameAndIDFromFlag(cmd *cobra.Command) (uint64, string) {
+	username, _ := cmd.Flags().GetString("name")
+	identifier, _ := cmd.Flags().GetInt64("identifier")
+	if username == "" && identifier < 0 {
+		err := errors.New("--name or --identifier flag is required")
+		ErrorOutput(
+			err,
+			"Cannot rename user: "+status.Convert(err).Message(),
+			"",
+		)
+	}
+
+	return uint64(identifier), username
+}
+
 func init() {
 	rootCmd.AddCommand(userCmd)
 	userCmd.AddCommand(createUserCmd)
+	createUserCmd.Flags().StringP("display-name", "d", "", "Display name")
+	createUserCmd.Flags().StringP("email", "e", "", "Email")
+	createUserCmd.Flags().StringP("picture-url", "p", "", "Profile picture URL")
 	userCmd.AddCommand(listUsersCmd)
+	usernameAndIDFlag(listUsersCmd)
+	listUsersCmd.Flags().StringP("email", "e", "", "Email")
 	userCmd.AddCommand(destroyUserCmd)
+	usernameAndIDFlag(destroyUserCmd)
 	userCmd.AddCommand(renameUserCmd)
+	usernameAndIDFlag(renameUserCmd)
+	renameUserCmd.Flags().StringP("new-name", "r", "", "New username")
+	renameNodeCmd.MarkFlagRequired("new-name")
 }
 
 var errMissingParameter = errors.New("missing parameters")
@@ -44,7 +77,7 @@ var createUserCmd = &cobra.Command{
 
 		userName := args[0]
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
@@ -52,19 +85,36 @@ var createUserCmd = &cobra.Command{
 
 		request := &v1.CreateUserRequest{Name: userName}
 
+		if displayName, _ := cmd.Flags().GetString("display-name"); displayName != "" {
+			request.DisplayName = displayName
+		}
+
+		if email, _ := cmd.Flags().GetString("email"); email != "" {
+			request.Email = email
+		}
+
+		if pictureURL, _ := cmd.Flags().GetString("picture-url"); pictureURL != "" {
+			if _, err := url.Parse(pictureURL); err != nil {
+				ErrorOutput(
+					err,
+					fmt.Sprintf(
+						"Invalid Picture URL: %s",
+						err,
+					),
+					output,
+				)
+			}
+			request.PictureUrl = pictureURL
+		}
+
 		log.Trace().Interface("request", request).Msg("Sending CreateUser request")
 		response, err := client.CreateUser(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Cannot create user: %s",
-					status.Convert(err).Message(),
-				),
+				"Cannot create user: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		SuccessOutput(response.GetUser(), "User created", output)
@@ -72,70 +122,61 @@ var createUserCmd = &cobra.Command{
 }
 
 var destroyUserCmd = &cobra.Command{
-	Use:     "destroy NAME",
+	Use:     "destroy --identifier ID or --name NAME",
 	Short:   "Destroys a user",
 	Aliases: []string{"delete"},
-	Args: func(cmd *cobra.Command, args []string) error {
-		if len(args) < 1 {
-			return errMissingParameter
-		}
-
-		return nil
-	},
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		userName := args[0]
-
-		request := &v1.GetUserRequest{
-			Name: userName,
+		id, username := usernameAndIDFromFlag(cmd)
+		request := &v1.ListUsersRequest{
+			Name: username,
+			Id:   id,
 		}
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
-		_, err := client.GetUser(ctx, request)
+		users, err := client.ListUsers(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Error: %s", status.Convert(err).Message()),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
+		if len(users.GetUsers()) != 1 {
+			err := errors.New("Unable to determine user to delete, query returned multiple users, use ID")
+			ErrorOutput(
+				err,
+				"Error: "+status.Convert(err).Message(),
+				output,
+			)
+		}
+
+		user := users.GetUsers()[0]
+
 		confirm := false
 		force, _ := cmd.Flags().GetBool("force")
 		if !force {
-			prompt := &survey.Confirm{
-				Message: fmt.Sprintf(
-					"Do you want to remove the user '%s' and any associated preauthkeys?",
-					userName,
-				),
-			}
-			err := survey.AskOne(prompt, &confirm)
-			if err != nil {
-				return
-			}
+			confirm = util.YesNo(fmt.Sprintf(
+				"Do you want to remove the user %q (%d) and any associated preauthkeys?",
+				user.GetName(), user.GetId(),
+			))
 		}
 
 		if confirm || force {
-			request := &v1.DeleteUserRequest{Name: userName}
+			request := &v1.DeleteUserRequest{Id: user.GetId()}
 
 			response, err := client.DeleteUser(ctx, request)
 			if err != nil {
 				ErrorOutput(
 					err,
-					fmt.Sprintf(
-						"Cannot destroy user: %s",
-						status.Convert(err).Message(),
-					),
+					"Cannot destroy user: "+status.Convert(err).Message(),
 					output,
 				)
-
-				return
 			}
 			SuccessOutput(response, "User destroyed", output)
 		} else {
@@ -151,36 +192,48 @@ var listUsersCmd = &cobra.Command{
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
 		request := &v1.ListUsersRequest{}
 
+		id, _ := cmd.Flags().GetInt64("identifier")
+		username, _ := cmd.Flags().GetString("name")
+		email, _ := cmd.Flags().GetString("email")
+
+		// filter by one param at most
+		switch {
+		case id > 0:
+			request.Id = uint64(id)
+		case username != "":
+			request.Name = username
+		case email != "":
+			request.Email = email
+		}
+
 		response, err := client.ListUsers(ctx, request)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf("Cannot get users: %s", status.Convert(err).Message()),
+				"Cannot get users: "+status.Convert(err).Message(),
 				output,
 			)
-
-			return
 		}
 
 		if output != "" {
 			SuccessOutput(response.GetUsers(), "", output)
-
-			return
 		}
 
-		tableData := pterm.TableData{{"ID", "Name", "Created"}}
+		tableData := pterm.TableData{{"ID", "Name", "Username", "Email", "Created"}}
 		for _, user := range response.GetUsers() {
 			tableData = append(
 				tableData,
 				[]string{
-					user.GetId(),
+					strconv.FormatUint(user.GetId(), 10),
+					user.GetDisplayName(),
 					user.GetName(),
+					user.GetEmail(),
 					user.GetCreatedAt().AsTime().Format("2006-01-02 15:04:05"),
 				},
 			)
@@ -192,48 +245,59 @@ var listUsersCmd = &cobra.Command{
 				fmt.Sprintf("Failed to render pterm table: %s", err),
 				output,
 			)
-
-			return
 		}
 	},
 }
 
 var renameUserCmd = &cobra.Command{
-	Use:     "rename OLD_NAME NEW_NAME",
+	Use:     "rename",
 	Short:   "Renames a user",
 	Aliases: []string{"mv"},
-	Args: func(cmd *cobra.Command, args []string) error {
-		expectedArguments := 2
-		if len(args) < expectedArguments {
-			return errMissingParameter
-		}
-
-		return nil
-	},
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
 
-		ctx, client, conn, cancel := getHeadscaleCLIClient()
+		ctx, client, conn, cancel := newHeadscaleCLIWithConfig()
 		defer cancel()
 		defer conn.Close()
 
-		request := &v1.RenameUserRequest{
-			OldName: args[0],
-			NewName: args[1],
+		id, username := usernameAndIDFromFlag(cmd)
+		listReq := &v1.ListUsersRequest{
+			Name: username,
+			Id:   id,
 		}
 
-		response, err := client.RenameUser(ctx, request)
+		users, err := client.ListUsers(ctx, listReq)
 		if err != nil {
 			ErrorOutput(
 				err,
-				fmt.Sprintf(
-					"Cannot rename user: %s",
-					status.Convert(err).Message(),
-				),
+				"Error: "+status.Convert(err).Message(),
 				output,
 			)
+		}
 
-			return
+		if len(users.GetUsers()) != 1 {
+			err := errors.New("Unable to determine user to delete, query returned multiple users, use ID")
+			ErrorOutput(
+				err,
+				"Error: "+status.Convert(err).Message(),
+				output,
+			)
+		}
+
+		newName, _ := cmd.Flags().GetString("new-name")
+
+		renameReq := &v1.RenameUserRequest{
+			OldId:   id,
+			NewName: newName,
+		}
+
+		response, err := client.RenameUser(ctx, renameReq)
+		if err != nil {
+			ErrorOutput(
+				err,
+				"Cannot rename user: "+status.Convert(err).Message(),
+				output,
+			)
 		}
 
 		SuccessOutput(response.GetUser(), "User renamed", output)

cmd/headscale/cli/utils.go

@@ -23,25 +23,25 @@ const (
 	SocketWritePermissions  = 0o666
 )
 
-func getHeadscaleApp() (*hscontrol.Headscale, error) {
-	cfg, err := types.GetHeadscaleConfig()
+func newHeadscaleServerWithConfig() (*hscontrol.Headscale, error) {
+	cfg, err := types.LoadServerConfig()
 	if err != nil {
 		return nil, fmt.Errorf(
-			"failed to load configuration while creating headscale instance: %w",
+			"loading configuration: %w",
 			err,
 		)
 	}
 
 	app, err := hscontrol.NewHeadscale(cfg)
 	if err != nil {
-		return nil, err
+		return nil, fmt.Errorf("creating new headscale: %w", err)
 	}
 
 	return app, nil
 }
 
-func getHeadscaleCLIClient() (context.Context, v1.HeadscaleServiceClient, *grpc.ClientConn, context.CancelFunc) {
-	cfg, err := types.GetHeadscaleConfig()
+func newHeadscaleCLIWithConfig() (context.Context, v1.HeadscaleServiceClient, *grpc.ClientConn, context.CancelFunc) {
+	cfg, err := types.LoadCLIConfig()
 	if err != nil {
 		log.Fatal().
 			Err(err).
@@ -130,7 +130,7 @@ func getHeadscaleCLIClient() (context.Context, v1.HeadscaleServiceClient, *grpc.
 	return ctx, client, conn, cancel
 }
 
-func SuccessOutput(result interface{}, override string, outputFormat string) {
+func output(result any, override string, outputFormat string) string {
 	var jsonBytes []byte
 	var err error
 	switch outputFormat {
@@ -151,21 +151,33 @@ func SuccessOutput(result interface{}, override string, outputFormat string) {
 		}
 	default:
 		// nolint
-		fmt.Println(override)
-
-		return
+		return override
 	}
 
-	// nolint
-	fmt.Println(string(jsonBytes))
+	return string(jsonBytes)
 }
 
+// SuccessOutput prints the result to stdout and exits with status code 0.
+func SuccessOutput(result any, override string, outputFormat string) {
+	fmt.Println(output(result, override, outputFormat))
+	os.Exit(0)
+}
+
+// ErrorOutput prints an error message to stderr and exits with status code 1.
 func ErrorOutput(errResult error, override string, outputFormat string) {
 	type errOutput struct {
 		Error string `json:"error"`
 	}
 
-	SuccessOutput(errOutput{errResult.Error()}, override, outputFormat)
+	var errorMessage string
+	if errResult != nil {
+		errorMessage = errResult.Error()
+	} else {
+		errorMessage = override
+	}
+
+	fmt.Fprintf(os.Stderr, "%s\n", output(errOutput{errorMessage}, override, outputFormat))
+	os.Exit(1)
 }
 
 func HasMachineOutputFlag() bool {

cmd/headscale/cli/version.go

@@ -1,13 +1,13 @@
 package cli
 
 import (
+	"github.com/juanfont/headscale/hscontrol/types"
 	"github.com/spf13/cobra"
 )
 
-var Version = "dev"
-
 func init() {
 	rootCmd.AddCommand(versionCmd)
+	versionCmd.Flags().StringP("output", "o", "", "Output format. Empty for human-readable, 'json', 'json-line' or 'yaml'")
 }
 
 var versionCmd = &cobra.Command{
@@ -16,6 +16,9 @@ var versionCmd = &cobra.Command{
 	Long:  "The version of headscale.",
 	Run: func(cmd *cobra.Command, args []string) {
 		output, _ := cmd.Flags().GetString("output")
-		SuccessOutput(map[string]string{"version": Version}, Version, output)
+
+		info := types.GetVersionInfo()
+
+		SuccessOutput(info, info.String(), output)
 	},
 }

cmd/headscale/headscale_test.go

@@ -4,7 +4,6 @@ import (
 	"io/fs"
 	"os"
 	"path/filepath"
-	"strings"
 	"testing"
 
 	"github.com/juanfont/headscale/hscontrol/types"
@@ -113,60 +112,3 @@ func (*Suite) TestConfigLoading(c *check.C) {
 	c.Assert(viper.GetBool("logtail.enabled"), check.Equals, false)
 	c.Assert(viper.GetBool("randomize_client_port"), check.Equals, false)
 }
-
-func writeConfig(c *check.C, tmpDir string, configYaml []byte) {
-	// Populate a custom config file
-	configFile := filepath.Join(tmpDir, "config.yaml")
-	err := os.WriteFile(configFile, configYaml, 0o600)
-	if err != nil {
-		c.Fatalf("Couldn't write file %s", configFile)
-	}
-}
-
-func (*Suite) TestTLSConfigValidation(c *check.C) {
-	tmpDir, err := os.MkdirTemp("", "headscale")
-	if err != nil {
-		c.Fatal(err)
-	}
-	// defer os.RemoveAll(tmpDir)
-	configYaml := []byte(`---
-tls_letsencrypt_hostname: example.com
-tls_letsencrypt_challenge_type: ""
-tls_cert_path: abc.pem
-noise:
-  private_key_path: noise_private.key`)
-	writeConfig(c, tmpDir, configYaml)
-
-	// Check configuration validation errors (1)
-	err = types.LoadConfig(tmpDir, false)
-	c.Assert(err, check.NotNil)
-	// check.Matches can not handle multiline strings
-	tmp := strings.ReplaceAll(err.Error(), "\n", "***")
-	c.Assert(
-		tmp,
-		check.Matches,
-		".*Fatal config error: set either tls_letsencrypt_hostname or tls_cert_path/tls_key_path, not both.*",
-	)
-	c.Assert(
-		tmp,
-		check.Matches,
-		".*Fatal config error: the only supported values for tls_letsencrypt_challenge_type are.*",
-	)
-	c.Assert(
-		tmp,
-		check.Matches,
-		".*Fatal config error: server_url must start with https:// or http://.*",
-	)
-
-	// Check configuration validation errors (2)
-	configYaml = []byte(`---
-noise:
-  private_key_path: noise_private.key
-server_url: http://127.0.0.1:8080
-tls_letsencrypt_hostname: example.com
-tls_letsencrypt_challenge_type: TLS-ALPN-01
-`)
-	writeConfig(c, tmpDir, configYaml)
-	err = types.LoadConfig(tmpDir, false)
-	c.Assert(err, check.IsNil)
-}

cmd/hi/README.md

@@ -0,0 +1,6 @@
+# hi
+
+hi (headscale integration runner) is an entirely "vibe coded" wrapper around our
+[integration test suite](../integration). It essentially runs the docker
+commands for you with some added benefits of extracting resources like logs and
+databases.

cmd/hi/cleanup.go

@@ -0,0 +1,316 @@
+package main
+
+import (
+	"context"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/cenkalti/backoff/v5"
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/api/types/image"
+	"github.com/docker/docker/client"
+	"github.com/docker/docker/errdefs"
+)
+
+// cleanupBeforeTest performs cleanup operations before running tests.
+func cleanupBeforeTest(ctx context.Context) error {
+	if err := killTestContainers(ctx); err != nil {
+		return fmt.Errorf("failed to kill test containers: %w", err)
+	}
+
+	if err := pruneDockerNetworks(ctx); err != nil {
+		return fmt.Errorf("failed to prune networks: %w", err)
+	}
+
+	return nil
+}
+
+// cleanupAfterTest removes the test container after completion.
+func cleanupAfterTest(ctx context.Context, cli *client.Client, containerID string) error {
+	return cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
+		Force: true,
+	})
+}
+
+// killTestContainers terminates and removes all test containers.
+func killTestContainers(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	containers, err := cli.ContainerList(ctx, container.ListOptions{
+		All: true,
+	})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	removed := 0
+	for _, cont := range containers {
+		shouldRemove := false
+		for _, name := range cont.Names {
+			if strings.Contains(name, "headscale-test-suite") ||
+				strings.Contains(name, "hs-") ||
+				strings.Contains(name, "ts-") ||
+				strings.Contains(name, "derp-") {
+				shouldRemove = true
+				break
+			}
+		}
+
+		if shouldRemove {
+			// First kill the container if it's running
+			if cont.State == "running" {
+				_ = cli.ContainerKill(ctx, cont.ID, "KILL")
+			}
+
+			// Then remove the container with retry logic
+			if removeContainerWithRetry(ctx, cli, cont.ID) {
+				removed++
+			}
+		}
+	}
+
+	if removed > 0 {
+		fmt.Printf("Removed %d test containers\n", removed)
+	} else {
+		fmt.Println("No test containers found to remove")
+	}
+
+	return nil
+}
+
+const (
+	containerRemoveInitialInterval = 100 * time.Millisecond
+	containerRemoveMaxElapsedTime  = 2 * time.Second
+)
+
+// removeContainerWithRetry attempts to remove a container with exponential backoff retry logic.
+func removeContainerWithRetry(ctx context.Context, cli *client.Client, containerID string) bool {
+	expBackoff := backoff.NewExponentialBackOff()
+	expBackoff.InitialInterval = containerRemoveInitialInterval
+
+	_, err := backoff.Retry(ctx, func() (struct{}, error) {
+		err := cli.ContainerRemove(ctx, containerID, container.RemoveOptions{
+			Force: true,
+		})
+		if err != nil {
+			return struct{}{}, err
+		}
+
+		return struct{}{}, nil
+	}, backoff.WithBackOff(expBackoff), backoff.WithMaxElapsedTime(containerRemoveMaxElapsedTime))
+
+	return err == nil
+}
+
+// pruneDockerNetworks removes unused Docker networks.
+func pruneDockerNetworks(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	report, err := cli.NetworksPrune(ctx, filters.Args{})
+	if err != nil {
+		return fmt.Errorf("failed to prune networks: %w", err)
+	}
+
+	if len(report.NetworksDeleted) > 0 {
+		fmt.Printf("Removed %d unused networks\n", len(report.NetworksDeleted))
+	} else {
+		fmt.Println("No unused networks found to remove")
+	}
+
+	return nil
+}
+
+// cleanOldImages removes test-related and old dangling Docker images.
+func cleanOldImages(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	images, err := cli.ImageList(ctx, image.ListOptions{
+		All: true,
+	})
+	if err != nil {
+		return fmt.Errorf("failed to list images: %w", err)
+	}
+
+	removed := 0
+	for _, img := range images {
+		shouldRemove := false
+		for _, tag := range img.RepoTags {
+			if strings.Contains(tag, "hs-") ||
+				strings.Contains(tag, "headscale-integration") ||
+				strings.Contains(tag, "tailscale") {
+				shouldRemove = true
+				break
+			}
+		}
+
+		if len(img.RepoTags) == 0 && time.Unix(img.Created, 0).Before(time.Now().Add(-7*24*time.Hour)) {
+			shouldRemove = true
+		}
+
+		if shouldRemove {
+			_, err := cli.ImageRemove(ctx, img.ID, image.RemoveOptions{
+				Force: true,
+			})
+			if err == nil {
+				removed++
+			}
+		}
+	}
+
+	if removed > 0 {
+		fmt.Printf("Removed %d test images\n", removed)
+	} else {
+		fmt.Println("No test images found to remove")
+	}
+
+	return nil
+}
+
+// cleanCacheVolume removes the Docker volume used for Go module cache.
+func cleanCacheVolume(ctx context.Context) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	volumeName := "hs-integration-go-cache"
+	err = cli.VolumeRemove(ctx, volumeName, true)
+	if err != nil {
+		if errdefs.IsNotFound(err) {
+			fmt.Printf("Go module cache volume not found: %s\n", volumeName)
+		} else if errdefs.IsConflict(err) {
+			fmt.Printf("Go module cache volume is in use and cannot be removed: %s\n", volumeName)
+		} else {
+			fmt.Printf("Failed to remove Go module cache volume %s: %v\n", volumeName, err)
+		}
+	} else {
+		fmt.Printf("Removed Go module cache volume: %s\n", volumeName)
+	}
+
+	return nil
+}
+
+// cleanupSuccessfulTestArtifacts removes artifacts from successful test runs to save disk space.
+// This function removes large artifacts that are mainly useful for debugging failures:
+// - Database dumps (.db files)
+// - Profile data (pprof directories)
+// - MapResponse data (mapresponses directories)
+// - Prometheus metrics files
+//
+// It preserves:
+// - Log files (.log) which are small and useful for verification.
+func cleanupSuccessfulTestArtifacts(logsDir string, verbose bool) error {
+	entries, err := os.ReadDir(logsDir)
+	if err != nil {
+		return fmt.Errorf("failed to read logs directory: %w", err)
+	}
+
+	var (
+		removedFiles, removedDirs int
+		totalSize                 int64
+	)
+
+	for _, entry := range entries {
+		name := entry.Name()
+		fullPath := filepath.Join(logsDir, name)
+
+		if entry.IsDir() {
+			// Remove pprof and mapresponses directories (typically large)
+			// These directories contain artifacts from all containers in the test run
+			if name == "pprof" || name == "mapresponses" {
+				size, sizeErr := getDirSize(fullPath)
+				if sizeErr == nil {
+					totalSize += size
+				}
+
+				err := os.RemoveAll(fullPath)
+				if err != nil {
+					if verbose {
+						log.Printf("Warning: failed to remove directory %s: %v", name, err)
+					}
+				} else {
+					removedDirs++
+
+					if verbose {
+						log.Printf("Removed directory: %s/", name)
+					}
+				}
+			}
+		} else {
+			// Only process test-related files (headscale and tailscale)
+			if !strings.HasPrefix(name, "hs-") && !strings.HasPrefix(name, "ts-") {
+				continue
+			}
+
+			// Remove database, metrics, and status files, but keep logs
+			shouldRemove := strings.HasSuffix(name, ".db") ||
+				strings.HasSuffix(name, "_metrics.txt") ||
+				strings.HasSuffix(name, "_status.json")
+
+			if shouldRemove {
+				info, infoErr := entry.Info()
+				if infoErr == nil {
+					totalSize += info.Size()
+				}
+
+				err := os.Remove(fullPath)
+				if err != nil {
+					if verbose {
+						log.Printf("Warning: failed to remove file %s: %v", name, err)
+					}
+				} else {
+					removedFiles++
+
+					if verbose {
+						log.Printf("Removed file: %s", name)
+					}
+				}
+			}
+		}
+	}
+
+	if removedFiles > 0 || removedDirs > 0 {
+		const bytesPerMB = 1024 * 1024
+		log.Printf("Cleaned up %d files and %d directories (freed ~%.2f MB)",
+			removedFiles, removedDirs, float64(totalSize)/bytesPerMB)
+	}
+
+	return nil
+}
+
+// getDirSize calculates the total size of a directory.
+func getDirSize(path string) (int64, error) {
+	var size int64
+
+	err := filepath.Walk(path, func(_ string, info os.FileInfo, err error) error {
+		if err != nil {
+			return err
+		}
+
+		if !info.IsDir() {
+			size += info.Size()
+		}
+
+		return nil
+	})
+
+	return size, err
+}

cmd/hi/docker.go

@@ -0,0 +1,846 @@
+package main
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"io"
+	"log"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/image"
+	"github.com/docker/docker/api/types/mount"
+	"github.com/docker/docker/client"
+	"github.com/docker/docker/pkg/stdcopy"
+	"github.com/juanfont/headscale/integration/dockertestutil"
+)
+
+var (
+	ErrTestFailed              = errors.New("test failed")
+	ErrUnexpectedContainerWait = errors.New("unexpected end of container wait")
+	ErrNoDockerContext         = errors.New("no docker context found")
+	ErrAnotherRunInProgress    = errors.New("another integration test run is already in progress")
+)
+
+// RunningTestInfo contains information about a currently running integration test.
+type RunningTestInfo struct {
+	RunID         string
+	ContainerID   string
+	ContainerName string
+	StartTime     time.Time
+	Duration      time.Duration
+	TestPattern   string
+}
+
+// ErrNoRunningTests indicates that no integration test is currently running.
+var ErrNoRunningTests = errors.New("no running tests found")
+
+// checkForRunningTests checks if there's already an integration test running.
+// Returns ErrNoRunningTests if no test is running, or RunningTestInfo with details about the running test.
+func checkForRunningTests(ctx context.Context) (*RunningTestInfo, error) {
+	cli, err := createDockerClient()
+	if err != nil {
+		return nil, fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	// List all running containers
+	containers, err := cli.ContainerList(ctx, container.ListOptions{
+		All: false, // Only running containers
+	})
+	if err != nil {
+		return nil, fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	// Look for containers with hi.test-type=test-runner label
+	for _, cont := range containers {
+		if cont.Labels != nil && cont.Labels["hi.test-type"] == "test-runner" {
+			// Found a running test runner container
+			runID := cont.Labels["hi.run-id"]
+
+			containerName := ""
+			for _, name := range cont.Names {
+				containerName = strings.TrimPrefix(name, "/")
+
+				break
+			}
+
+			// Get more details via inspection
+			inspect, err := cli.ContainerInspect(ctx, cont.ID)
+			if err != nil {
+				// Return basic info if inspection fails
+				return &RunningTestInfo{
+					RunID:         runID,
+					ContainerID:   cont.ID,
+					ContainerName: containerName,
+				}, nil
+			}
+
+			startTime, _ := time.Parse(time.RFC3339Nano, inspect.State.StartedAt)
+			duration := time.Since(startTime)
+
+			// Try to extract test pattern from command
+			testPattern := ""
+
+			if len(inspect.Config.Cmd) > 0 {
+				for i, arg := range inspect.Config.Cmd {
+					if arg == "-run" && i+1 < len(inspect.Config.Cmd) {
+						testPattern = inspect.Config.Cmd[i+1]
+
+						break
+					}
+				}
+			}
+
+			return &RunningTestInfo{
+				RunID:         runID,
+				ContainerID:   cont.ID,
+				ContainerName: containerName,
+				StartTime:     startTime,
+				Duration:      duration,
+				TestPattern:   testPattern,
+			}, nil
+		}
+	}
+
+	return nil, ErrNoRunningTests
+}
+
+// runTestContainer executes integration tests in a Docker container.
+func runTestContainer(ctx context.Context, config *RunConfig) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	runID := dockertestutil.GenerateRunID()
+	containerName := "headscale-test-suite-" + runID
+	logsDir := filepath.Join(config.LogsDir, runID)
+
+	if config.Verbose {
+		log.Printf("Run ID: %s", runID)
+		log.Printf("Container name: %s", containerName)
+		log.Printf("Logs directory: %s", logsDir)
+	}
+
+	absLogsDir, err := filepath.Abs(logsDir)
+	if err != nil {
+		return fmt.Errorf("failed to get absolute path for logs directory: %w", err)
+	}
+
+	const dirPerm = 0o755
+	if err := os.MkdirAll(absLogsDir, dirPerm); err != nil {
+		return fmt.Errorf("failed to create logs directory: %w", err)
+	}
+
+	if config.CleanBefore {
+		if config.Verbose {
+			log.Printf("Running pre-test cleanup...")
+		}
+		if err := cleanupBeforeTest(ctx); err != nil && config.Verbose {
+			log.Printf("Warning: pre-test cleanup failed: %v", err)
+		}
+	}
+
+	goTestCmd := buildGoTestCommand(config)
+	if config.Verbose {
+		log.Printf("Command: %s", strings.Join(goTestCmd, " "))
+	}
+
+	imageName := "golang:" + config.GoVersion
+	if err := ensureImageAvailable(ctx, cli, imageName, config.Verbose); err != nil {
+		return fmt.Errorf("failed to ensure image availability: %w", err)
+	}
+
+	resp, err := createGoTestContainer(ctx, cli, config, containerName, absLogsDir, goTestCmd)
+	if err != nil {
+		return fmt.Errorf("failed to create container: %w", err)
+	}
+
+	if config.Verbose {
+		log.Printf("Created container: %s", resp.ID)
+	}
+
+	if err := cli.ContainerStart(ctx, resp.ID, container.StartOptions{}); err != nil {
+		return fmt.Errorf("failed to start container: %w", err)
+	}
+
+	log.Printf("Starting test: %s", config.TestPattern)
+
+	// Start stats collection for container resource monitoring (if enabled)
+	var statsCollector *StatsCollector
+	if config.Stats {
+		var err error
+		statsCollector, err = NewStatsCollector()
+		if err != nil {
+			if config.Verbose {
+				log.Printf("Warning: failed to create stats collector: %v", err)
+			}
+			statsCollector = nil
+		}
+
+		if statsCollector != nil {
+			defer statsCollector.Close()
+
+			// Start stats collection immediately - no need for complex retry logic
+			// The new implementation monitors Docker events and will catch containers as they start
+			if err := statsCollector.StartCollection(ctx, runID, config.Verbose); err != nil {
+				if config.Verbose {
+					log.Printf("Warning: failed to start stats collection: %v", err)
+				}
+			}
+			defer statsCollector.StopCollection()
+		}
+	}
+
+	exitCode, err := streamAndWait(ctx, cli, resp.ID)
+
+	// Ensure all containers have finished and logs are flushed before extracting artifacts
+	if waitErr := waitForContainerFinalization(ctx, cli, resp.ID, config.Verbose); waitErr != nil && config.Verbose {
+		log.Printf("Warning: failed to wait for container finalization: %v", waitErr)
+	}
+
+	// Extract artifacts from test containers before cleanup
+	if err := extractArtifactsFromContainers(ctx, resp.ID, logsDir, config.Verbose); err != nil && config.Verbose {
+		log.Printf("Warning: failed to extract artifacts from containers: %v", err)
+	}
+
+	// Always list control files regardless of test outcome
+	listControlFiles(logsDir)
+
+	// Print stats summary and check memory limits if enabled
+	if config.Stats && statsCollector != nil {
+		violations := statsCollector.PrintSummaryAndCheckLimits(config.HSMemoryLimit, config.TSMemoryLimit)
+		if len(violations) > 0 {
+			log.Printf("MEMORY LIMIT VIOLATIONS DETECTED:")
+			log.Printf("=================================")
+			for _, violation := range violations {
+				log.Printf("Container %s exceeded memory limit: %.1f MB > %.1f MB",
+					violation.ContainerName, violation.MaxMemoryMB, violation.LimitMB)
+			}
+
+			return fmt.Errorf("test failed: %d container(s) exceeded memory limits", len(violations))
+		}
+	}
+
+	shouldCleanup := config.CleanAfter && (!config.KeepOnFailure || exitCode == 0)
+	if shouldCleanup {
+		if config.Verbose {
+			log.Printf("Running post-test cleanup...")
+		}
+		if cleanErr := cleanupAfterTest(ctx, cli, resp.ID); cleanErr != nil && config.Verbose {
+			log.Printf("Warning: post-test cleanup failed: %v", cleanErr)
+		}
+
+		// Clean up artifacts from successful tests to save disk space in CI
+		if exitCode == 0 {
+			if config.Verbose {
+				log.Printf("Test succeeded, cleaning up artifacts to save disk space...")
+			}
+
+			cleanErr := cleanupSuccessfulTestArtifacts(logsDir, config.Verbose)
+
+			if cleanErr != nil && config.Verbose {
+				log.Printf("Warning: artifact cleanup failed: %v", cleanErr)
+			}
+		}
+	}
+
+	if err != nil {
+		return fmt.Errorf("test execution failed: %w", err)
+	}
+
+	if exitCode != 0 {
+		return fmt.Errorf("%w: exit code %d", ErrTestFailed, exitCode)
+	}
+
+	log.Printf("Test completed successfully!")
+
+	return nil
+}
+
+// buildGoTestCommand constructs the go test command arguments.
+func buildGoTestCommand(config *RunConfig) []string {
+	cmd := []string{"go", "test", "./..."}
+
+	if config.TestPattern != "" {
+		cmd = append(cmd, "-run", config.TestPattern)
+	}
+
+	if config.FailFast {
+		cmd = append(cmd, "-failfast")
+	}
+
+	cmd = append(cmd, "-timeout", config.Timeout.String())
+	cmd = append(cmd, "-v")
+
+	return cmd
+}
+
+// createGoTestContainer creates a Docker container configured for running integration tests.
+func createGoTestContainer(ctx context.Context, cli *client.Client, config *RunConfig, containerName, logsDir string, goTestCmd []string) (container.CreateResponse, error) {
+	pwd, err := os.Getwd()
+	if err != nil {
+		return container.CreateResponse{}, fmt.Errorf("failed to get working directory: %w", err)
+	}
+
+	projectRoot := findProjectRoot(pwd)
+
+	runID := dockertestutil.ExtractRunIDFromContainerName(containerName)
+
+	env := []string{
+		fmt.Sprintf("HEADSCALE_INTEGRATION_POSTGRES=%d", boolToInt(config.UsePostgres)),
+		"HEADSCALE_INTEGRATION_RUN_ID=" + runID,
+	}
+
+	// Pass through CI environment variable for CI detection
+	if ci := os.Getenv("CI"); ci != "" {
+		env = append(env, "CI="+ci)
+	}
+
+	// Pass through all HEADSCALE_INTEGRATION_* environment variables
+	for _, e := range os.Environ() {
+		if strings.HasPrefix(e, "HEADSCALE_INTEGRATION_") {
+			// Skip the ones we already set explicitly
+			if strings.HasPrefix(e, "HEADSCALE_INTEGRATION_POSTGRES=") ||
+				strings.HasPrefix(e, "HEADSCALE_INTEGRATION_RUN_ID=") {
+				continue
+			}
+
+			env = append(env, e)
+		}
+	}
+
+	// Set GOCACHE to a known location (used by both bind mount and volume cases)
+	env = append(env, "GOCACHE=/cache/go-build")
+
+	containerConfig := &container.Config{
+		Image:      "golang:" + config.GoVersion,
+		Cmd:        goTestCmd,
+		Env:        env,
+		WorkingDir: projectRoot + "/integration",
+		Tty:        true,
+		Labels: map[string]string{
+			"hi.run-id":    runID,
+			"hi.test-type": "test-runner",
+		},
+	}
+
+	// Get the correct Docker socket path from the current context
+	dockerSocketPath := getDockerSocketPath()
+
+	if config.Verbose {
+		log.Printf("Using Docker socket: %s", dockerSocketPath)
+	}
+
+	binds := []string{
+		fmt.Sprintf("%s:%s", projectRoot, projectRoot),
+		dockerSocketPath + ":/var/run/docker.sock",
+		logsDir + ":/tmp/control",
+	}
+
+	// Use bind mounts for Go cache if provided via environment variables,
+	// otherwise fall back to Docker volumes for local development
+	var mounts []mount.Mount
+
+	goCache := os.Getenv("HEADSCALE_INTEGRATION_GO_CACHE")
+	goBuildCache := os.Getenv("HEADSCALE_INTEGRATION_GO_BUILD_CACHE")
+
+	if goCache != "" {
+		binds = append(binds, goCache+":/go")
+	} else {
+		mounts = append(mounts, mount.Mount{
+			Type:   mount.TypeVolume,
+			Source: "hs-integration-go-cache",
+			Target: "/go",
+		})
+	}
+
+	if goBuildCache != "" {
+		binds = append(binds, goBuildCache+":/cache/go-build")
+	} else {
+		mounts = append(mounts, mount.Mount{
+			Type:   mount.TypeVolume,
+			Source: "hs-integration-go-build-cache",
+			Target: "/cache/go-build",
+		})
+	}
+
+	hostConfig := &container.HostConfig{
+		AutoRemove: false, // We'll remove manually for better control
+		Binds:      binds,
+		Mounts:     mounts,
+	}
+
+	return cli.ContainerCreate(ctx, containerConfig, hostConfig, nil, nil, containerName)
+}
+
+// streamAndWait streams container output and waits for completion.
+func streamAndWait(ctx context.Context, cli *client.Client, containerID string) (int, error) {
+	out, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
+		ShowStdout: true,
+		ShowStderr: true,
+		Follow:     true,
+	})
+	if err != nil {
+		return -1, fmt.Errorf("failed to get container logs: %w", err)
+	}
+	defer out.Close()
+
+	go func() {
+		_, _ = io.Copy(os.Stdout, out)
+	}()
+
+	statusCh, errCh := cli.ContainerWait(ctx, containerID, container.WaitConditionNotRunning)
+	select {
+	case err := <-errCh:
+		if err != nil {
+			return -1, fmt.Errorf("error waiting for container: %w", err)
+		}
+	case status := <-statusCh:
+		return int(status.StatusCode), nil
+	}
+
+	return -1, ErrUnexpectedContainerWait
+}
+
+// waitForContainerFinalization ensures all test containers have properly finished and flushed their output.
+func waitForContainerFinalization(ctx context.Context, cli *client.Client, testContainerID string, verbose bool) error {
+	// First, get all related test containers
+	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	testContainers := getCurrentTestContainers(containers, testContainerID, verbose)
+
+	// Wait for all test containers to reach a final state
+	maxWaitTime := 10 * time.Second
+	checkInterval := 500 * time.Millisecond
+	timeout := time.After(maxWaitTime)
+	ticker := time.NewTicker(checkInterval)
+	defer ticker.Stop()
+
+	for {
+		select {
+		case <-timeout:
+			if verbose {
+				log.Printf("Timeout waiting for container finalization, proceeding with artifact extraction")
+			}
+			return nil
+		case <-ticker.C:
+			allFinalized := true
+
+			for _, testCont := range testContainers {
+				inspect, err := cli.ContainerInspect(ctx, testCont.ID)
+				if err != nil {
+					if verbose {
+						log.Printf("Warning: failed to inspect container %s: %v", testCont.name, err)
+					}
+					continue
+				}
+
+				// Check if container is in a final state
+				if !isContainerFinalized(inspect.State) {
+					allFinalized = false
+					if verbose {
+						log.Printf("Container %s still finalizing (state: %s)", testCont.name, inspect.State.Status)
+					}
+
+					break
+				}
+			}
+
+			if allFinalized {
+				if verbose {
+					log.Printf("All test containers finalized, ready for artifact extraction")
+				}
+				return nil
+			}
+		}
+	}
+}
+
+// isContainerFinalized checks if a container has reached a final state where logs are flushed.
+func isContainerFinalized(state *container.State) bool {
+	// Container is finalized if it's not running and has a finish time
+	return !state.Running && state.FinishedAt != ""
+}
+
+// findProjectRoot locates the project root by finding the directory containing go.mod.
+func findProjectRoot(startPath string) string {
+	current := startPath
+	for {
+		if _, err := os.Stat(filepath.Join(current, "go.mod")); err == nil {
+			return current
+		}
+		parent := filepath.Dir(current)
+		if parent == current {
+			return startPath
+		}
+		current = parent
+	}
+}
+
+// boolToInt converts a boolean to an integer for environment variables.
+func boolToInt(b bool) int {
+	if b {
+		return 1
+	}
+	return 0
+}
+
+// DockerContext represents Docker context information.
+type DockerContext struct {
+	Name      string         `json:"Name"`
+	Metadata  map[string]any `json:"Metadata"`
+	Endpoints map[string]any `json:"Endpoints"`
+	Current   bool           `json:"Current"`
+}
+
+// createDockerClient creates a Docker client with context detection.
+func createDockerClient() (*client.Client, error) {
+	contextInfo, err := getCurrentDockerContext()
+	if err != nil {
+		return client.NewClientWithOpts(client.FromEnv, client.WithAPIVersionNegotiation())
+	}
+
+	var clientOpts []client.Opt
+	clientOpts = append(clientOpts, client.WithAPIVersionNegotiation())
+
+	if contextInfo != nil {
+		if endpoints, ok := contextInfo.Endpoints["docker"]; ok {
+			if endpointMap, ok := endpoints.(map[string]any); ok {
+				if host, ok := endpointMap["Host"].(string); ok {
+					if runConfig.Verbose {
+						log.Printf("Using Docker host from context '%s': %s", contextInfo.Name, host)
+					}
+					clientOpts = append(clientOpts, client.WithHost(host))
+				}
+			}
+		}
+	}
+
+	if len(clientOpts) == 1 {
+		clientOpts = append(clientOpts, client.FromEnv)
+	}
+
+	return client.NewClientWithOpts(clientOpts...)
+}
+
+// getCurrentDockerContext retrieves the current Docker context information.
+func getCurrentDockerContext() (*DockerContext, error) {
+	cmd := exec.Command("docker", "context", "inspect")
+	output, err := cmd.Output()
+	if err != nil {
+		return nil, fmt.Errorf("failed to get docker context: %w", err)
+	}
+
+	var contexts []DockerContext
+	if err := json.Unmarshal(output, &contexts); err != nil {
+		return nil, fmt.Errorf("failed to parse docker context: %w", err)
+	}
+
+	if len(contexts) > 0 {
+		return &contexts[0], nil
+	}
+
+	return nil, ErrNoDockerContext
+}
+
+// getDockerSocketPath returns the correct Docker socket path for the current context.
+func getDockerSocketPath() string {
+	// Always use the default socket path for mounting since Docker handles
+	// the translation to the actual socket (e.g., colima socket) internally
+	return "/var/run/docker.sock"
+}
+
+// checkImageAvailableLocally checks if the specified Docker image is available locally.
+func checkImageAvailableLocally(ctx context.Context, cli *client.Client, imageName string) (bool, error) {
+	_, _, err := cli.ImageInspectWithRaw(ctx, imageName)
+	if err != nil {
+		if client.IsErrNotFound(err) {
+			return false, nil
+		}
+		return false, fmt.Errorf("failed to inspect image %s: %w", imageName, err)
+	}
+
+	return true, nil
+}
+
+// ensureImageAvailable checks if the image is available locally first, then pulls if needed.
+func ensureImageAvailable(ctx context.Context, cli *client.Client, imageName string, verbose bool) error {
+	// First check if image is available locally
+	available, err := checkImageAvailableLocally(ctx, cli, imageName)
+	if err != nil {
+		return fmt.Errorf("failed to check local image availability: %w", err)
+	}
+
+	if available {
+		if verbose {
+			log.Printf("Image %s is available locally", imageName)
+		}
+		return nil
+	}
+
+	// Image not available locally, try to pull it
+	if verbose {
+		log.Printf("Image %s not found locally, pulling...", imageName)
+	}
+
+	reader, err := cli.ImagePull(ctx, imageName, image.PullOptions{})
+	if err != nil {
+		return fmt.Errorf("failed to pull image %s: %w", imageName, err)
+	}
+	defer reader.Close()
+
+	if verbose {
+		_, err = io.Copy(os.Stdout, reader)
+		if err != nil {
+			return fmt.Errorf("failed to read pull output: %w", err)
+		}
+	} else {
+		_, err = io.Copy(io.Discard, reader)
+		if err != nil {
+			return fmt.Errorf("failed to read pull output: %w", err)
+		}
+		log.Printf("Image %s pulled successfully", imageName)
+	}
+
+	return nil
+}
+
+// listControlFiles displays the headscale test artifacts created in the control logs directory.
+func listControlFiles(logsDir string) {
+	entries, err := os.ReadDir(logsDir)
+	if err != nil {
+		log.Printf("Logs directory: %s", logsDir)
+		return
+	}
+
+	var logFiles []string
+	var dataFiles []string
+	var dataDirs []string
+
+	for _, entry := range entries {
+		name := entry.Name()
+		// Only show headscale (hs-*) files and directories
+		if !strings.HasPrefix(name, "hs-") {
+			continue
+		}
+
+		if entry.IsDir() {
+			// Include directories (pprof, mapresponses)
+			if strings.Contains(name, "-pprof") || strings.Contains(name, "-mapresponses") {
+				dataDirs = append(dataDirs, name)
+			}
+		} else {
+			// Include files
+			switch {
+			case strings.HasSuffix(name, ".stderr.log") || strings.HasSuffix(name, ".stdout.log"):
+				logFiles = append(logFiles, name)
+			case strings.HasSuffix(name, ".db"):
+				dataFiles = append(dataFiles, name)
+			}
+		}
+	}
+
+	log.Printf("Test artifacts saved to: %s", logsDir)
+
+	if len(logFiles) > 0 {
+		log.Printf("Headscale logs:")
+		for _, file := range logFiles {
+			log.Printf("  %s", file)
+		}
+	}
+
+	if len(dataFiles) > 0 || len(dataDirs) > 0 {
+		log.Printf("Headscale data:")
+		for _, file := range dataFiles {
+			log.Printf("  %s", file)
+		}
+		for _, dir := range dataDirs {
+			log.Printf("  %s/", dir)
+		}
+	}
+}
+
+// extractArtifactsFromContainers collects container logs and files from the specific test run.
+func extractArtifactsFromContainers(ctx context.Context, testContainerID, logsDir string, verbose bool) error {
+	cli, err := createDockerClient()
+	if err != nil {
+		return fmt.Errorf("failed to create Docker client: %w", err)
+	}
+	defer cli.Close()
+
+	// List all containers
+	containers, err := cli.ContainerList(ctx, container.ListOptions{All: true})
+	if err != nil {
+		return fmt.Errorf("failed to list containers: %w", err)
+	}
+
+	// Get containers from the specific test run
+	currentTestContainers := getCurrentTestContainers(containers, testContainerID, verbose)
+
+	extractedCount := 0
+	for _, cont := range currentTestContainers {
+		// Extract container logs and tar files
+		if err := extractContainerArtifacts(ctx, cli, cont.ID, cont.name, logsDir, verbose); err != nil {
+			if verbose {
+				log.Printf("Warning: failed to extract artifacts from container %s (%s): %v", cont.name, cont.ID[:12], err)
+			}
+		} else {
+			if verbose {
+				log.Printf("Extracted artifacts from container %s (%s)", cont.name, cont.ID[:12])
+			}
+			extractedCount++
+		}
+	}
+
+	if verbose && extractedCount > 0 {
+		log.Printf("Extracted artifacts from %d containers", extractedCount)
+	}
+
+	return nil
+}
+
+// testContainer represents a container from the current test run.
+type testContainer struct {
+	ID   string
+	name string
+}
+
+// getCurrentTestContainers filters containers to only include those from the current test run.
+func getCurrentTestContainers(containers []container.Summary, testContainerID string, verbose bool) []testContainer {
+	var testRunContainers []testContainer
+
+	// Find the test container to get its run ID label
+	var runID string
+	for _, cont := range containers {
+		if cont.ID == testContainerID {
+			if cont.Labels != nil {
+				runID = cont.Labels["hi.run-id"]
+			}
+			break
+		}
+	}
+
+	if runID == "" {
+		log.Printf("Error: test container %s missing required hi.run-id label", testContainerID[:12])
+		return testRunContainers
+	}
+
+	if verbose {
+		log.Printf("Looking for containers with run ID: %s", runID)
+	}
+
+	// Find all containers with the same run ID
+	for _, cont := range containers {
+		for _, name := range cont.Names {
+			containerName := strings.TrimPrefix(name, "/")
+			if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
+				// Check if container has matching run ID label
+				if cont.Labels != nil && cont.Labels["hi.run-id"] == runID {
+					testRunContainers = append(testRunContainers, testContainer{
+						ID:   cont.ID,
+						name: containerName,
+					})
+					if verbose {
+						log.Printf("Including container %s (run ID: %s)", containerName, runID)
+					}
+				}
+
+				break
+			}
+		}
+	}
+
+	return testRunContainers
+}
+
+// extractContainerArtifacts saves logs and tar files from a container.
+func extractContainerArtifacts(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Ensure the logs directory exists
+	if err := os.MkdirAll(logsDir, 0o755); err != nil {
+		return fmt.Errorf("failed to create logs directory: %w", err)
+	}
+
+	// Extract container logs
+	if err := extractContainerLogs(ctx, cli, containerID, containerName, logsDir, verbose); err != nil {
+		return fmt.Errorf("failed to extract logs: %w", err)
+	}
+
+	// Extract tar files for headscale containers only
+	if strings.HasPrefix(containerName, "hs-") {
+		if err := extractContainerFiles(ctx, cli, containerID, containerName, logsDir, verbose); err != nil {
+			if verbose {
+				log.Printf("Warning: failed to extract files from %s: %v", containerName, err)
+			}
+			// Don't fail the whole extraction if files are missing
+		}
+	}
+
+	return nil
+}
+
+// extractContainerLogs saves the stdout and stderr logs from a container to files.
+func extractContainerLogs(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Get container logs
+	logReader, err := cli.ContainerLogs(ctx, containerID, container.LogsOptions{
+		ShowStdout: true,
+		ShowStderr: true,
+		Timestamps: false,
+		Follow:     false,
+		Tail:       "all",
+	})
+	if err != nil {
+		return fmt.Errorf("failed to get container logs: %w", err)
+	}
+	defer logReader.Close()
+
+	// Create log files following the headscale naming convention
+	stdoutPath := filepath.Join(logsDir, containerName+".stdout.log")
+	stderrPath := filepath.Join(logsDir, containerName+".stderr.log")
+
+	// Create buffers to capture stdout and stderr separately
+	var stdoutBuf, stderrBuf bytes.Buffer
+
+	// Demultiplex the Docker logs stream to separate stdout and stderr
+	_, err = stdcopy.StdCopy(&stdoutBuf, &stderrBuf, logReader)
+	if err != nil {
+		return fmt.Errorf("failed to demultiplex container logs: %w", err)
+	}
+
+	// Write stdout logs
+	if err := os.WriteFile(stdoutPath, stdoutBuf.Bytes(), 0o644); err != nil {
+		return fmt.Errorf("failed to write stdout log: %w", err)
+	}
+
+	// Write stderr logs
+	if err := os.WriteFile(stderrPath, stderrBuf.Bytes(), 0o644); err != nil {
+		return fmt.Errorf("failed to write stderr log: %w", err)
+	}
+
+	if verbose {
+		log.Printf("Saved logs for %s: %s, %s", containerName, stdoutPath, stderrPath)
+	}
+
+	return nil
+}
+
+// extractContainerFiles extracts database file and directories from headscale containers.
+// Note: The actual file extraction is now handled by the integration tests themselves
+// via SaveProfile, SaveMapResponses, and SaveDatabase functions in hsic.go.
+func extractContainerFiles(ctx context.Context, cli *client.Client, containerID, containerName, logsDir string, verbose bool) error {
+	// Files are now extracted directly by the integration tests
+	// This function is kept for potential future use or other file types
+	return nil
+}

cmd/hi/doctor.go

@@ -0,0 +1,374 @@
+package main
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"log"
+	"os/exec"
+	"strings"
+)
+
+var ErrSystemChecksFailed = errors.New("system checks failed")
+
+// DoctorResult represents the result of a single health check.
+type DoctorResult struct {
+	Name        string
+	Status      string // "PASS", "FAIL", "WARN"
+	Message     string
+	Suggestions []string
+}
+
+// runDoctorCheck performs comprehensive pre-flight checks for integration testing.
+func runDoctorCheck(ctx context.Context) error {
+	results := []DoctorResult{}
+
+	// Check 1: Docker binary availability
+	results = append(results, checkDockerBinary())
+
+	// Check 2: Docker daemon connectivity
+	dockerResult := checkDockerDaemon(ctx)
+	results = append(results, dockerResult)
+
+	// If Docker is available, run additional checks
+	if dockerResult.Status == "PASS" {
+		results = append(results, checkDockerContext(ctx))
+		results = append(results, checkDockerSocket(ctx))
+		results = append(results, checkGolangImage(ctx))
+	}
+
+	// Check 3: Go installation
+	results = append(results, checkGoInstallation())
+
+	// Check 4: Git repository
+	results = append(results, checkGitRepository())
+
+	// Check 5: Required files
+	results = append(results, checkRequiredFiles())
+
+	// Display results
+	displayDoctorResults(results)
+
+	// Return error if any critical checks failed
+	for _, result := range results {
+		if result.Status == "FAIL" {
+			return fmt.Errorf("%w - see details above", ErrSystemChecksFailed)
+		}
+	}
+
+	log.Printf("✅ All system checks passed - ready to run integration tests!")
+
+	return nil
+}
+
+// checkDockerBinary verifies Docker binary is available.
+func checkDockerBinary() DoctorResult {
+	_, err := exec.LookPath("docker")
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Binary",
+			Status:  "FAIL",
+			Message: "Docker binary not found in PATH",
+			Suggestions: []string{
+				"Install Docker: https://docs.docker.com/get-docker/",
+				"For macOS: consider using colima or Docker Desktop",
+				"Ensure docker is in your PATH",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Binary",
+		Status:  "PASS",
+		Message: "Docker binary found",
+	}
+}
+
+// checkDockerDaemon verifies Docker daemon is running and accessible.
+func checkDockerDaemon(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Daemon",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot create Docker client: %v", err),
+			Suggestions: []string{
+				"Start Docker daemon/service",
+				"Check Docker Desktop is running (if using Docker Desktop)",
+				"For colima: run 'colima start'",
+				"Verify DOCKER_HOST environment variable if set",
+			},
+		}
+	}
+	defer cli.Close()
+
+	_, err = cli.Ping(ctx)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Daemon",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot ping Docker daemon: %v", err),
+			Suggestions: []string{
+				"Ensure Docker daemon is running",
+				"Check Docker socket permissions",
+				"Try: docker info",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Daemon",
+		Status:  "PASS",
+		Message: "Docker daemon is running and accessible",
+	}
+}
+
+// checkDockerContext verifies Docker context configuration.
+func checkDockerContext(_ context.Context) DoctorResult {
+	contextInfo, err := getCurrentDockerContext()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Context",
+			Status:  "WARN",
+			Message: "Could not detect Docker context, using default settings",
+			Suggestions: []string{
+				"Check: docker context ls",
+				"Consider setting up a specific context if needed",
+			},
+		}
+	}
+
+	if contextInfo == nil {
+		return DoctorResult{
+			Name:    "Docker Context",
+			Status:  "PASS",
+			Message: "Using default Docker context",
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Context",
+		Status:  "PASS",
+		Message: "Using Docker context: " + contextInfo.Name,
+	}
+}
+
+// checkDockerSocket verifies Docker socket accessibility.
+func checkDockerSocket(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Socket",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot access Docker socket: %v", err),
+			Suggestions: []string{
+				"Check Docker socket permissions",
+				"Add user to docker group: sudo usermod -aG docker $USER",
+				"For colima: ensure socket is accessible",
+			},
+		}
+	}
+	defer cli.Close()
+
+	info, err := cli.Info(ctx)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Docker Socket",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot get Docker info: %v", err),
+			Suggestions: []string{
+				"Check Docker daemon status",
+				"Verify socket permissions",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Docker Socket",
+		Status:  "PASS",
+		Message: fmt.Sprintf("Docker socket accessible (Server: %s)", info.ServerVersion),
+	}
+}
+
+// checkGolangImage verifies the golang Docker image is available locally or can be pulled.
+func checkGolangImage(ctx context.Context) DoctorResult {
+	cli, err := createDockerClient()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: "Cannot create Docker client for image check",
+		}
+	}
+	defer cli.Close()
+
+	goVersion := detectGoVersion()
+	imageName := "golang:" + goVersion
+
+	// First check if image is available locally
+	available, err := checkImageAvailableLocally(ctx, cli, imageName)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot check golang image %s: %v", imageName, err),
+			Suggestions: []string{
+				"Check Docker daemon status",
+				"Try: docker images | grep golang",
+			},
+		}
+	}
+
+	if available {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "PASS",
+			Message: fmt.Sprintf("Golang image %s is available locally", imageName),
+		}
+	}
+
+	// Image not available locally, try to pull it
+	err = ensureImageAvailable(ctx, cli, imageName, false)
+	if err != nil {
+		return DoctorResult{
+			Name:    "Golang Image",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Golang image %s not available locally and cannot pull: %v", imageName, err),
+			Suggestions: []string{
+				"Check internet connectivity",
+				"Verify Docker Hub access",
+				"Try: docker pull " + imageName,
+				"Or run tests offline if image was pulled previously",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Golang Image",
+		Status:  "PASS",
+		Message: fmt.Sprintf("Golang image %s is now available", imageName),
+	}
+}
+
+// checkGoInstallation verifies Go is installed and working.
+func checkGoInstallation() DoctorResult {
+	_, err := exec.LookPath("go")
+	if err != nil {
+		return DoctorResult{
+			Name:    "Go Installation",
+			Status:  "FAIL",
+			Message: "Go binary not found in PATH",
+			Suggestions: []string{
+				"Install Go: https://golang.org/dl/",
+				"Ensure go is in your PATH",
+			},
+		}
+	}
+
+	cmd := exec.Command("go", "version")
+	output, err := cmd.Output()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Go Installation",
+			Status:  "FAIL",
+			Message: fmt.Sprintf("Cannot get Go version: %v", err),
+		}
+	}
+
+	version := strings.TrimSpace(string(output))
+
+	return DoctorResult{
+		Name:    "Go Installation",
+		Status:  "PASS",
+		Message: version,
+	}
+}
+
+// checkGitRepository verifies we're in a git repository.
+func checkGitRepository() DoctorResult {
+	cmd := exec.Command("git", "rev-parse", "--git-dir")
+	err := cmd.Run()
+	if err != nil {
+		return DoctorResult{
+			Name:    "Git Repository",
+			Status:  "FAIL",
+			Message: "Not in a Git repository",
+			Suggestions: []string{
+				"Run from within the headscale git repository",
+				"Clone the repository: git clone https://github.com/juanfont/headscale.git",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Git Repository",
+		Status:  "PASS",
+		Message: "Running in Git repository",
+	}
+}
+
+// checkRequiredFiles verifies required files exist.
+func checkRequiredFiles() DoctorResult {
+	requiredFiles := []string{
+		"go.mod",
+		"integration/",
+		"cmd/hi/",
+	}
+
+	var missingFiles []string
+	for _, file := range requiredFiles {
+		cmd := exec.Command("test", "-e", file)
+		if err := cmd.Run(); err != nil {
+			missingFiles = append(missingFiles, file)
+		}
+	}
+
+	if len(missingFiles) > 0 {
+		return DoctorResult{
+			Name:    "Required Files",
+			Status:  "FAIL",
+			Message: "Missing required files: " + strings.Join(missingFiles, ", "),
+			Suggestions: []string{
+				"Ensure you're in the headscale project root directory",
+				"Check that integration/ directory exists",
+				"Verify this is a complete headscale repository",
+			},
+		}
+	}
+
+	return DoctorResult{
+		Name:    "Required Files",
+		Status:  "PASS",
+		Message: "All required files found",
+	}
+}
+
+// displayDoctorResults shows the results in a formatted way.
+func displayDoctorResults(results []DoctorResult) {
+	log.Printf("🔍 System Health Check Results")
+	log.Printf("================================")
+
+	for _, result := range results {
+		var icon string
+		switch result.Status {
+		case "PASS":
+			icon = "✅"
+		case "WARN":
+			icon = "⚠️"
+		case "FAIL":
+			icon = "❌"
+		default:
+			icon = "❓"
+		}
+
+		log.Printf("%s %s: %s", icon, result.Name, result.Message)
+
+		if len(result.Suggestions) > 0 {
+			for _, suggestion := range result.Suggestions {
+				log.Printf("   💡 %s", suggestion)
+			}
+		}
+	}
+
+	log.Printf("================================")
+}

cmd/hi/main.go

@@ -0,0 +1,93 @@
+package main
+
+import (
+	"context"
+	"os"
+
+	"github.com/creachadair/command"
+	"github.com/creachadair/flax"
+)
+
+var runConfig RunConfig
+
+func main() {
+	root := command.C{
+		Name: "hi",
+		Help: "Headscale Integration test runner",
+		Commands: []*command.C{
+			{
+				Name:     "run",
+				Help:     "Run integration tests",
+				Usage:    "run [test-pattern] [flags]",
+				SetFlags: command.Flags(flax.MustBind, &runConfig),
+				Run:      runIntegrationTest,
+			},
+			{
+				Name: "doctor",
+				Help: "Check system requirements for running integration tests",
+				Run: func(env *command.Env) error {
+					return runDoctorCheck(env.Context())
+				},
+			},
+			{
+				Name: "clean",
+				Help: "Clean Docker resources",
+				Commands: []*command.C{
+					{
+						Name: "networks",
+						Help: "Prune unused Docker networks",
+						Run: func(env *command.Env) error {
+							return pruneDockerNetworks(env.Context())
+						},
+					},
+					{
+						Name: "images",
+						Help: "Clean old test images",
+						Run: func(env *command.Env) error {
+							return cleanOldImages(env.Context())
+						},
+					},
+					{
+						Name: "containers",
+						Help: "Kill all test containers",
+						Run: func(env *command.Env) error {
+							return killTestContainers(env.Context())
+						},
+					},
+					{
+						Name: "cache",
+						Help: "Clean Go module cache volume",
+						Run: func(env *command.Env) error {
+							return cleanCacheVolume(env.Context())
+						},
+					},
+					{
+						Name: "all",
+						Help: "Run all cleanup operations",
+						Run: func(env *command.Env) error {
+							return cleanAll(env.Context())
+						},
+					},
+				},
+			},
+			command.HelpCommand(nil),
+		},
+	}
+
+	env := root.NewEnv(nil).MergeFlags(true)
+	command.RunOrFail(env, os.Args[1:])
+}
+
+func cleanAll(ctx context.Context) error {
+	if err := killTestContainers(ctx); err != nil {
+		return err
+	}
+	if err := pruneDockerNetworks(ctx); err != nil {
+		return err
+	}
+	if err := cleanOldImages(ctx); err != nil {
+		return err
+	}
+
+	return cleanCacheVolume(ctx)
+}

cmd/hi/run.go

@@ -0,0 +1,196 @@
+package main
+
+import (
+	"errors"
+	"fmt"
+	"log"
+	"os"
+	"path/filepath"
+	"strings"
+	"time"
+
+	"github.com/creachadair/command"
+)
+
+var ErrTestPatternRequired = errors.New("test pattern is required as first argument or use --test flag")
+
+// formatRunningTestError creates a detailed error message about a running test.
+func formatRunningTestError(info *RunningTestInfo) error {
+	var msg strings.Builder
+	msg.WriteString("\n")
+	msg.WriteString("╔══════════════════════════════════════════════════════════════════╗\n")
+	msg.WriteString("║  Another integration test run is already in progress!            ║\n")
+	msg.WriteString("╚══════════════════════════════════════════════════════════════════╝\n")
+	msg.WriteString("\n")
+	msg.WriteString("Running test details:\n")
+	msg.WriteString(fmt.Sprintf("  Run ID:      %s\n", info.RunID))
+	msg.WriteString(fmt.Sprintf("  Container:   %s\n", info.ContainerName))
+
+	if info.TestPattern != "" {
+		msg.WriteString(fmt.Sprintf("  Test:        %s\n", info.TestPattern))
+	}
+
+	if !info.StartTime.IsZero() {
+		msg.WriteString(fmt.Sprintf("  Started:     %s\n", info.StartTime.Format("2006-01-02 15:04:05")))
+		msg.WriteString(fmt.Sprintf("  Running for: %s\n", formatDuration(info.Duration)))
+	}
+
+	msg.WriteString("\n")
+	msg.WriteString("Please wait for the current test to complete, or stop it with:\n")
+	msg.WriteString("  go run ./cmd/hi clean containers\n")
+	msg.WriteString("\n")
+	msg.WriteString("To monitor the running test:\n")
+	msg.WriteString(fmt.Sprintf("  docker logs -f %s\n", info.ContainerName))
+
+	return fmt.Errorf("%w\n%s", ErrAnotherRunInProgress, msg.String())
+}
+
+const secondsPerMinute = 60
+
+// formatDuration formats a duration in a human-readable way.
+func formatDuration(d time.Duration) string {
+	if d < time.Minute {
+		return fmt.Sprintf("%d seconds", int(d.Seconds()))
+	}
+
+	if d < time.Hour {
+		minutes := int(d.Minutes())
+		seconds := int(d.Seconds()) % secondsPerMinute
+
+		return fmt.Sprintf("%d minutes, %d seconds", minutes, seconds)
+	}
+
+	hours := int(d.Hours())
+	minutes := int(d.Minutes()) % secondsPerMinute
+
+	return fmt.Sprintf("%d hours, %d minutes", hours, minutes)
+}
+
+type RunConfig struct {
+	TestPattern   string        `flag:"test,Test pattern to run"`
+	Timeout       time.Duration `flag:"timeout,default=120m,Test timeout"`
+	FailFast      bool          `flag:"failfast,default=true,Stop on first test failure"`
+	UsePostgres   bool          `flag:"postgres,default=false,Use PostgreSQL instead of SQLite"`
+	GoVersion     string        `flag:"go-version,Go version to use (auto-detected from go.mod)"`
+	CleanBefore   bool          `flag:"clean-before,default=true,Clean resources before test"`
+	CleanAfter    bool          `flag:"clean-after,default=true,Clean resources after test"`
+	KeepOnFailure bool          `flag:"keep-on-failure,default=false,Keep containers on test failure"`
+	LogsDir       string        `flag:"logs-dir,default=control_logs,Control logs directory"`
+	Verbose       bool          `flag:"verbose,default=false,Verbose output"`
+	Stats         bool          `flag:"stats,default=false,Collect and display container resource usage statistics"`
+	HSMemoryLimit float64       `flag:"hs-memory-limit,default=0,Fail test if any Headscale container exceeds this memory limit in MB (0 = disabled)"`
+	TSMemoryLimit float64       `flag:"ts-memory-limit,default=0,Fail test if any Tailscale container exceeds this memory limit in MB (0 = disabled)"`
+	Force         bool          `flag:"force,default=false,Kill any running test and start a new one"`
+}
+
+// runIntegrationTest executes the integration test workflow.
+func runIntegrationTest(env *command.Env) error {
+	args := env.Args
+	if len(args) > 0 && runConfig.TestPattern == "" {
+		runConfig.TestPattern = args[0]
+	}
+
+	if runConfig.TestPattern == "" {
+		return ErrTestPatternRequired
+	}
+
+	if runConfig.GoVersion == "" {
+		runConfig.GoVersion = detectGoVersion()
+	}
+
+	// Check if another test run is already in progress
+	runningTest, err := checkForRunningTests(env.Context())
+	if err != nil && !errors.Is(err, ErrNoRunningTests) {
+		log.Printf("Warning: failed to check for running tests: %v", err)
+	} else if runningTest != nil {
+		if runConfig.Force {
+			log.Printf("Force flag set, killing existing test run: %s", runningTest.RunID)
+
+			err = killTestContainers(env.Context())
+			if err != nil {
+				return fmt.Errorf("failed to kill existing test containers: %w", err)
+			}
+		} else {
+			return formatRunningTestError(runningTest)
+		}
+	}
+
+	// Run pre-flight checks
+	if runConfig.Verbose {
+		log.Printf("Running pre-flight system checks...")
+	}
+	if err := runDoctorCheck(env.Context()); err != nil {
+		return fmt.Errorf("pre-flight checks failed: %w", err)
+	}
+
+	if runConfig.Verbose {
+		log.Printf("Running test: %s", runConfig.TestPattern)
+		log.Printf("Go version: %s", runConfig.GoVersion)
+		log.Printf("Timeout: %s", runConfig.Timeout)
+		log.Printf("Use PostgreSQL: %t", runConfig.UsePostgres)
+	}
+
+	return runTestContainer(env.Context(), &runConfig)
+}
+
+// detectGoVersion reads the Go version from go.mod file.
+func detectGoVersion() string {
+	goModPath := filepath.Join("..", "..", "go.mod")
+
+	if _, err := os.Stat("go.mod"); err == nil {
+		goModPath = "go.mod"
+	} else if _, err := os.Stat("../../go.mod"); err == nil {
+		goModPath = "../../go.mod"
+	}
+
+	content, err := os.ReadFile(goModPath)
+	if err != nil {
+		return "1.25"
+	}
+
+	lines := splitLines(string(content))
+	for _, line := range lines {
+		if len(line) > 3 && line[:3] == "go " {
+			version := line[3:]
+			if idx := indexOf(version, " "); idx != -1 {
+				version = version[:idx]
+			}
+
+			return version
+		}
+	}
+
+	return "1.25"
+}
+
+// splitLines splits a string into lines without using strings.Split.
+func splitLines(s string) []string {
+	var lines []string
+	var current string
+
+	for _, char := range s {
+		if char == '\n' {
+			lines = append(lines, current)
+			current = ""
+		} else {
+			current += string(char)
+		}
+	}
+
+	if current != "" {
+		lines = append(lines, current)
+	}
+
+	return lines
+}
+
+// indexOf finds the first occurrence of substr in s.
+func indexOf(s, substr string) int {
+	for i := 0; i <= len(s)-len(substr); i++ {
+		if s[i:i+len(substr)] == substr {
+			return i
+		}
+	}
+
+	return -1
+}

cmd/hi/stats.go

@@ -0,0 +1,471 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"log"
+	"sort"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/docker/docker/api/types"
+	"github.com/docker/docker/api/types/container"
+	"github.com/docker/docker/api/types/events"
+	"github.com/docker/docker/api/types/filters"
+	"github.com/docker/docker/client"
+)
+
+// ContainerStats represents statistics for a single container.
+type ContainerStats struct {
+	ContainerID   string
+	ContainerName string
+	Stats         []StatsSample
+	mutex         sync.RWMutex
+}
+
+// StatsSample represents a single stats measurement.
+type StatsSample struct {
+	Timestamp time.Time
+	CPUUsage  float64 // CPU usage percentage
+	MemoryMB  float64 // Memory usage in MB
+}
+
+// StatsCollector manages collection of container statistics.
+type StatsCollector struct {
+	client            *client.Client
+	containers        map[string]*ContainerStats
+	stopChan          chan struct{}
+	wg                sync.WaitGroup
+	mutex             sync.RWMutex
+	collectionStarted bool
+}
+
+// NewStatsCollector creates a new stats collector instance.
+func NewStatsCollector() (*StatsCollector, error) {
+	cli, err := createDockerClient()
+	if err != nil {
+		return nil, fmt.Errorf("failed to create Docker client: %w", err)
+	}
+
+	return &StatsCollector{
+		client:     cli,
+		containers: make(map[string]*ContainerStats),
+		stopChan:   make(chan struct{}),
+	}, nil
+}
+
+// StartCollection begins monitoring all containers and collecting stats for hs- and ts- containers with matching run ID.
+func (sc *StatsCollector) StartCollection(ctx context.Context, runID string, verbose bool) error {
+	sc.mutex.Lock()
+	defer sc.mutex.Unlock()
+
+	if sc.collectionStarted {
+		return errors.New("stats collection already started")
+	}
+
+	sc.collectionStarted = true
+
+	// Start monitoring existing containers
+	sc.wg.Add(1)
+	go sc.monitorExistingContainers(ctx, runID, verbose)
+
+	// Start Docker events monitoring for new containers
+	sc.wg.Add(1)
+	go sc.monitorDockerEvents(ctx, runID, verbose)
+
+	if verbose {
+		log.Printf("Started container monitoring for run ID %s", runID)
+	}
+
+	return nil
+}
+
+// StopCollection stops all stats collection.
+func (sc *StatsCollector) StopCollection() {
+	// Check if already stopped without holding lock
+	sc.mutex.RLock()
+	if !sc.collectionStarted {
+		sc.mutex.RUnlock()
+		return
+	}
+	sc.mutex.RUnlock()
+
+	// Signal stop to all goroutines
+	close(sc.stopChan)
+
+	// Wait for all goroutines to finish
+	sc.wg.Wait()
+
+	// Mark as stopped
+	sc.mutex.Lock()
+	sc.collectionStarted = false
+	sc.mutex.Unlock()
+}
+
+// monitorExistingContainers checks for existing containers that match our criteria.
+func (sc *StatsCollector) monitorExistingContainers(ctx context.Context, runID string, verbose bool) {
+	defer sc.wg.Done()
+
+	containers, err := sc.client.ContainerList(ctx, container.ListOptions{})
+	if err != nil {
+		if verbose {
+			log.Printf("Failed to list existing containers: %v", err)
+		}
+		return
+	}
+
+	for _, cont := range containers {
+		if sc.shouldMonitorContainer(cont, runID) {
+			sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
+		}
+	}
+}
+
+// monitorDockerEvents listens for container start events and begins monitoring relevant containers.
+func (sc *StatsCollector) monitorDockerEvents(ctx context.Context, runID string, verbose bool) {
+	defer sc.wg.Done()
+
+	filter := filters.NewArgs()
+	filter.Add("type", "container")
+	filter.Add("event", "start")
+
+	eventOptions := events.ListOptions{
+		Filters: filter,
+	}
+
+	events, errs := sc.client.Events(ctx, eventOptions)
+
+	for {
+		select {
+		case <-sc.stopChan:
+			return
+		case <-ctx.Done():
+			return
+		case event := <-events:
+			if event.Type == "container" && event.Action == "start" {
+				// Get container details
+				containerInfo, err := sc.client.ContainerInspect(ctx, event.ID)
+				if err != nil {
+					continue
+				}
+
+				// Convert to types.Container format for consistency
+				cont := types.Container{
+					ID:     containerInfo.ID,
+					Names:  []string{containerInfo.Name},
+					Labels: containerInfo.Config.Labels,
+				}
+
+				if sc.shouldMonitorContainer(cont, runID) {
+					sc.startStatsForContainer(ctx, cont.ID, cont.Names[0], verbose)
+				}
+			}
+		case err := <-errs:
+			if verbose {
+				log.Printf("Error in Docker events stream: %v", err)
+			}
+			return
+		}
+	}
+}
+
+// shouldMonitorContainer determines if a container should be monitored.
+func (sc *StatsCollector) shouldMonitorContainer(cont types.Container, runID string) bool {
+	// Check if it has the correct run ID label
+	if cont.Labels == nil || cont.Labels["hi.run-id"] != runID {
+		return false
+	}
+
+	// Check if it's an hs- or ts- container
+	for _, name := range cont.Names {
+		containerName := strings.TrimPrefix(name, "/")
+		if strings.HasPrefix(containerName, "hs-") || strings.HasPrefix(containerName, "ts-") {
+			return true
+		}
+	}
+
+	return false
+}
+
+// startStatsForContainer begins stats collection for a specific container.
+func (sc *StatsCollector) startStatsForContainer(ctx context.Context, containerID, containerName string, verbose bool) {
+	containerName = strings.TrimPrefix(containerName, "/")
+
+	sc.mutex.Lock()
+	// Check if we're already monitoring this container
+	if _, exists := sc.containers[containerID]; exists {
+		sc.mutex.Unlock()
+		return
+	}
+
+	sc.containers[containerID] = &ContainerStats{
+		ContainerID:   containerID,
+		ContainerName: containerName,
+		Stats:         make([]StatsSample, 0),
+	}
+	sc.mutex.Unlock()
+
+	if verbose {
+		log.Printf("Starting stats collection for container %s (%s)", containerName, containerID[:12])
+	}
+
+	sc.wg.Add(1)
+	go sc.collectStatsForContainer(ctx, containerID, verbose)
+}
+
+// collectStatsForContainer collects stats for a specific container using Docker API streaming.
+func (sc *StatsCollector) collectStatsForContainer(ctx context.Context, containerID string, verbose bool) {
+	defer sc.wg.Done()
+
+	// Use Docker API streaming stats - much more efficient than CLI
+	statsResponse, err := sc.client.ContainerStats(ctx, containerID, true)
+	if err != nil {
+		if verbose {
+			log.Printf("Failed to get stats stream for container %s: %v", containerID[:12], err)
+		}
+		return
+	}
+	defer statsResponse.Body.Close()
+
+	decoder := json.NewDecoder(statsResponse.Body)
+	var prevStats *container.Stats
+
+	for {
+		select {
+		case <-sc.stopChan:
+			return
+		case <-ctx.Done():
+			return
+		default:
+			var stats container.Stats
+			if err := decoder.Decode(&stats); err != nil {
+				// EOF is expected when container stops or stream ends
+				if err.Error() != "EOF" && verbose {
+					log.Printf("Failed to decode stats for container %s: %v", containerID[:12], err)
+				}
+				return
+			}
+
+			// Calculate CPU percentage (only if we have previous stats)
+			var cpuPercent float64
+			if prevStats != nil {
+				cpuPercent = calculateCPUPercent(prevStats, &stats)
+			}
+
+			// Calculate memory usage in MB
+			memoryMB := float64(stats.MemoryStats.Usage) / (1024 * 1024)
+
+			// Store the sample (skip first sample since CPU calculation needs previous stats)
+			if prevStats != nil {
+				// Get container stats reference without holding the main mutex
+				var containerStats *ContainerStats
+				var exists bool
+
+				sc.mutex.RLock()
+				containerStats, exists = sc.containers[containerID]
+				sc.mutex.RUnlock()
+
+				if exists && containerStats != nil {
+					containerStats.mutex.Lock()
+					containerStats.Stats = append(containerStats.Stats, StatsSample{
+						Timestamp: time.Now(),
+						CPUUsage:  cpuPercent,
+						MemoryMB:  memoryMB,
+					})
+					containerStats.mutex.Unlock()
+				}
+			}
+
+			// Save current stats for next iteration
+			prevStats = &stats
+		}
+	}
+}
+
+// calculateCPUPercent calculates CPU usage percentage from Docker stats.
+func calculateCPUPercent(prevStats, stats *container.Stats) float64 {
+	// CPU calculation based on Docker's implementation
+	cpuDelta := float64(stats.CPUStats.CPUUsage.TotalUsage) - float64(prevStats.CPUStats.CPUUsage.TotalUsage)
+	systemDelta := float64(stats.CPUStats.SystemUsage) - float64(prevStats.CPUStats.SystemUsage)
+
+	if systemDelta > 0 && cpuDelta >= 0 {
+		// Calculate CPU percentage: (container CPU delta / system CPU delta) * number of CPUs * 100
+		numCPUs := float64(len(stats.CPUStats.CPUUsage.PercpuUsage))
+		if numCPUs == 0 {
+			// Fallback: if PercpuUsage is not available, assume 1 CPU
+			numCPUs = 1.0
+		}
+
+		return (cpuDelta / systemDelta) * numCPUs * 100.0
+	}
+
+	return 0.0
+}
+
+// ContainerStatsSummary represents summary statistics for a container.
+type ContainerStatsSummary struct {
+	ContainerName string
+	SampleCount   int
+	CPU           StatsSummary
+	Memory        StatsSummary
+}
+
+// MemoryViolation represents a container that exceeded the memory limit.
+type MemoryViolation struct {
+	ContainerName string
+	MaxMemoryMB   float64
+	LimitMB       float64
+}
+
+// StatsSummary represents min, max, and average for a metric.
+type StatsSummary struct {
+	Min     float64
+	Max     float64
+	Average float64
+}
+
+// GetSummary returns a summary of collected statistics.
+func (sc *StatsCollector) GetSummary() []ContainerStatsSummary {
+	// Take snapshot of container references without holding main lock long
+	sc.mutex.RLock()
+	containerRefs := make([]*ContainerStats, 0, len(sc.containers))
+	for _, containerStats := range sc.containers {
+		containerRefs = append(containerRefs, containerStats)
+	}
+	sc.mutex.RUnlock()
+
+	summaries := make([]ContainerStatsSummary, 0, len(containerRefs))
+
+	for _, containerStats := range containerRefs {
+		containerStats.mutex.RLock()
+		stats := make([]StatsSample, len(containerStats.Stats))
+		copy(stats, containerStats.Stats)
+		containerName := containerStats.ContainerName
+		containerStats.mutex.RUnlock()
+
+		if len(stats) == 0 {
+			continue
+		}
+
+		summary := ContainerStatsSummary{
+			ContainerName: containerName,
+			SampleCount:   len(stats),
+		}
+
+		// Calculate CPU stats
+		cpuValues := make([]float64, len(stats))
+		memoryValues := make([]float64, len(stats))
+
+		for i, sample := range stats {
+			cpuValues[i] = sample.CPUUsage
+			memoryValues[i] = sample.MemoryMB
+		}
+
+		summary.CPU = calculateStatsSummary(cpuValues)
+		summary.Memory = calculateStatsSummary(memoryValues)
+
+		summaries = append(summaries, summary)
+	}
+
+	// Sort by container name for consistent output
+	sort.Slice(summaries, func(i, j int) bool {
+		return summaries[i].ContainerName < summaries[j].ContainerName
+	})
+
+	return summaries
+}
+
+// calculateStatsSummary calculates min, max, and average for a slice of values.
+func calculateStatsSummary(values []float64) StatsSummary {
+	if len(values) == 0 {
+		return StatsSummary{}
+	}
+
+	min := values[0]
+	max := values[0]
+	sum := 0.0
+
+	for _, value := range values {
+		if value < min {
+			min = value
+		}
+		if value > max {
+			max = value
+		}
+		sum += value
+	}
+
+	return StatsSummary{
+		Min:     min,
+		Max:     max,
+		Average: sum / float64(len(values)),
+	}
+}
+
+// PrintSummary prints the statistics summary to the console.
+func (sc *StatsCollector) PrintSummary() {
+	summaries := sc.GetSummary()
+
+	if len(summaries) == 0 {
+		log.Printf("No container statistics collected")
+		return
+	}
+
+	log.Printf("Container Resource Usage Summary:")
+	log.Printf("================================")
+
+	for _, summary := range summaries {
+		log.Printf("Container: %s (%d samples)", summary.ContainerName, summary.SampleCount)
+		log.Printf("  CPU Usage:    Min: %6.2f%%  Max: %6.2f%%  Avg: %6.2f%%",
+			summary.CPU.Min, summary.CPU.Max, summary.CPU.Average)
+		log.Printf("  Memory Usage: Min: %6.1f MB Max: %6.1f MB Avg: %6.1f MB",
+			summary.Memory.Min, summary.Memory.Max, summary.Memory.Average)
+		log.Printf("")
+	}
+}
+
+// CheckMemoryLimits checks if any containers exceeded their memory limits.
+func (sc *StatsCollector) CheckMemoryLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
+	if hsLimitMB <= 0 && tsLimitMB <= 0 {
+		return nil
+	}
+
+	summaries := sc.GetSummary()
+	var violations []MemoryViolation
+
+	for _, summary := range summaries {
+		var limitMB float64
+		if strings.HasPrefix(summary.ContainerName, "hs-") {
+			limitMB = hsLimitMB
+		} else if strings.HasPrefix(summary.ContainerName, "ts-") {
+			limitMB = tsLimitMB
+		} else {
+			continue // Skip containers that don't match our patterns
+		}
+
+		if limitMB > 0 && summary.Memory.Max > limitMB {
+			violations = append(violations, MemoryViolation{
+				ContainerName: summary.ContainerName,
+				MaxMemoryMB:   summary.Memory.Max,
+				LimitMB:       limitMB,
+			})
+		}
+	}
+
+	return violations
+}
+
+// PrintSummaryAndCheckLimits prints the statistics summary and returns memory violations if any.
+func (sc *StatsCollector) PrintSummaryAndCheckLimits(hsLimitMB, tsLimitMB float64) []MemoryViolation {
+	sc.PrintSummary()
+	return sc.CheckMemoryLimits(hsLimitMB, tsLimitMB)
+}
+
+// Close closes the stats collector and cleans up resources.
+func (sc *StatsCollector) Close() error {
+	sc.StopCollection()
+	return sc.client.Close()
+}

cmd/mapresponses/main.go

@@ -0,0 +1,61 @@
+package main
+
+import (
+	"encoding/json"
+	"fmt"
+	"os"
+
+	"github.com/creachadair/command"
+	"github.com/creachadair/flax"
+	"github.com/juanfont/headscale/hscontrol/mapper"
+	"github.com/juanfont/headscale/integration/integrationutil"
+)
+
+type MapConfig struct {
+	Directory string `flag:"directory,Directory to read map responses from"`
+}
+
+var mapConfig MapConfig
+
+func main() {
+	root := command.C{
+		Name: "mapresponses",
+		Help: "MapResponses is a tool to map and compare map responses from a directory",
+		Commands: []*command.C{
+			{
+				Name:     "online",
+				Help:     "",
+				Usage:    "run [test-pattern] [flags]",
+				SetFlags: command.Flags(flax.MustBind, &mapConfig),
+				Run:      runOnline,
+			},
+			command.HelpCommand(nil),
+		},
+	}
+
+	env := root.NewEnv(nil).MergeFlags(true)
+	command.RunOrFail(env, os.Args[1:])
+}
+
+// runIntegrationTest executes the integration test workflow.
+func runOnline(env *command.Env) error {
+	if mapConfig.Directory == "" {
+		return fmt.Errorf("directory is required")
+	}
+
+	resps, err := mapper.ReadMapResponsesFromDirectory(mapConfig.Directory)
+	if err != nil {
+		return fmt.Errorf("reading map responses from directory: %w", err)
+	}
+
+	expected := integrationutil.BuildExpectedOnlineMap(resps)
+
+	out, err := json.MarshalIndent(expected, "", "  ")
+	if err != nil {
+		return fmt.Errorf("marshaling expected online map: %w", err)
+	}
+
+	os.Stderr.Write(out)
+	os.Stderr.Write([]byte("\n"))
+	return nil
+}

config-example.yaml

@@ -18,10 +18,8 @@ server_url: http://127.0.0.1:8080
 # listen_addr: 0.0.0.0:8080
 listen_addr: 127.0.0.1:8080
 
-# Address to listen to /metrics, you may want
-# to keep this endpoint private to your internal
-# network
-#
+# Address to listen to /metrics and /debug, you may want
+# to keep this endpoint private to your internal network
 metrics_listen_addr: 127.0.0.1:9090
 
 # Address to listen for gRPC.
@@ -43,9 +41,9 @@ grpc_allow_insecure: false
 # The Noise section includes specific configuration for the
 # TS2021 Noise protocol
 noise:
-  # The Noise private key is used to encrypt the
-  # traffic between headscale and Tailscale clients when
-  # using the new Noise-based protocol.
+  # The Noise private key is used to encrypt the traffic between headscale and
+  # Tailscale clients when using the new Noise-based protocol. A missing key
+  # will be automatically generated.
   private_key_path: /var/lib/headscale/noise_private.key
 
 # List of IP prefixes to allocate tailaddresses from.
@@ -58,11 +56,13 @@ noise:
 # IPv4: https://github.com/tailscale/tailscale/blob/22ebb25e833264f58d7c3f534a8b166894a89536/net/tsaddr/tsaddr.go#L33
 # Any other range is NOT supported, and it will cause unexpected issues.
 prefixes:
-  v6: fd7a:115c:a1e0::/48
   v4: 100.64.0.0/10
+  v6: fd7a:115c:a1e0::/48
 
   # Strategy used for allocation of IPs to nodes, available options:
-  # - sequential (default): assigns the next free IP from the previous given IP.
+  # - sequential (default): assigns the next free IP from the previous given
+  #   IP. A best-effort approach is used and Headscale might leave holes in the
+  #   IP range or fill up existing holes in the IP range.
   # - random: assigns the next free IP from a pseudo-random IP generator (crypto/rand).
   allocation: sequential
 
@@ -87,16 +87,17 @@ derp:
     region_code: "headscale"
     region_name: "Headscale Embedded DERP"
 
+    # Only allow clients associated with this server access
+    verify_clients: true
+
     # Listens over UDP at the configured address for STUN connections - to help with NAT traversal.
     # When the embedded DERP server is enabled stun_listen_addr MUST be defined.
     #
     # For more details on how this works, check this great article: https://tailscale.com/blog/how-tailscale-works/
     stun_listen_addr: "0.0.0.0:3478"
 
-    # Private key used to encrypt the traffic between headscale DERP
-    # and Tailscale clients.
-    # The private key file will be autogenerated if it's missing.
-    #
+    # Private key used to encrypt the traffic between headscale DERP and
+    # Tailscale clients. A missing key will be automatically generated.
     private_key_path: /var/lib/headscale/derp_server_private.key
 
     # This flag can be used, so the DERP map entry for the embedded DERP server is not written automatically,
@@ -106,7 +107,7 @@ derp:
 
     # For better connection stability (especially when using an Exit-Node and DNS is not working),
     # it is possible to optionally add the public IPv4 and IPv6 address to the Derp-Map using:
-    ipv4: 1.2.3.4
+    ipv4: 198.51.100.1
     ipv6: 2001:db8::1
 
   # List of externally available DERP maps encoded in JSON
@@ -129,7 +130,7 @@ derp:
   auto_update_enabled: true
 
   # How often should we check for DERP updates?
-  update_frequency: 24h
+  update_frequency: 3h
 
 # Disables the automatic check for headscale updates on startup
 disable_check_updates: false
@@ -138,6 +139,9 @@ disable_check_updates: false
 ephemeral_node_inactivity_timeout: 30m
 
 database:
+  # Database type. Available options: sqlite, postgres
+  # Please note that using Postgres is highly discouraged as it is only supported for legacy reasons.
+  # All new development, testing and optimisations are done with SQLite in mind.
   type: sqlite
 
   # Enable debug mode. This setting requires the log.level to be set to "debug" or "trace".
@@ -165,7 +169,14 @@ database:
     # https://www.sqlite.org/wal.html
     write_ahead_log: true
 
+    # Maximum number of WAL file frames before the WAL file is automatically checkpointed.
+    # https://www.sqlite.org/c3ref/wal_autocheckpoint.html
+    # Set to 0 to disable automatic checkpointing.
+    wal_autocheckpoint: 1000
+
   # # Postgres config
+  # Please note that using Postgres is highly discouraged as it is only supported for legacy reasons.
+  # See database.type for more information.
   # postgres:
   #   # If using a Unix socket to connect to Postgres, set the socket path in the 'host' field and leave 'port' blank.
   #   host: localhost
@@ -204,7 +215,7 @@ tls_letsencrypt_cache_dir: /var/lib/headscale/cache
 
 # Type of ACME challenge to use, currently supported types:
 # HTTP-01 or TLS-ALPN-01
-# See [docs/tls.md](docs/tls.md) for more information
+# See: docs/ref/tls.md for more information
 tls_letsencrypt_challenge_type: HTTP-01
 # When HTTP-01 challenge is chosen, letsencrypt must set up a
 # verification endpoint, and it will be listening on:
@@ -216,9 +227,11 @@ tls_cert_path: ""
 tls_key_path: ""
 
 log:
+  # Valid log levels: panic, fatal, error, warn, info, debug, trace
+  level: info
+
   # Output formatting for logs: text or json
   format: text
-  level: info
 
 ## Policy
 # headscale supports Tailscale's ACL policies.
@@ -255,7 +268,6 @@ policy:
 # all the fields under `dns` should be set to empty values.
 dns:
   # Whether to use [MagicDNS](https://tailscale.com/kb/1081/magicdns/).
-  # Only works if there is at least a nameserver defined.
   magic_dns: true
 
   # Defines the base domain to create the hostnames for MagicDNS.
@@ -265,6 +277,10 @@ dns:
   # `hostname.base_domain` (e.g., _myhost.example.com_).
   base_domain: example.com
 
+  # Whether to use the local DNS settings of a node or override the local DNS
+  # settings (default) and force the use of Headscale's DNS configuration.
+  override_local_dns: true
+
   # List of DNS servers to expose to clients.
   nameservers:
     global:
@@ -279,8 +295,7 @@ dns:
 
     # Split DNS (see https://tailscale.com/kb/1054/dns/),
     # a map of domains and which DNS server to use for each.
-    split:
-      {}
+    split: {}
       # foo.bar.com:
       #   - 1.1.1.1
       # darp.headscale.net:
@@ -292,8 +307,8 @@ dns:
   search_domains: []
 
   # Extra DNS records
-  # so far only A-records are supported (on the tailscale side)
-  # See https://github.com/juanfont/headscale/blob/main/docs/dns-records.md#Limitations
+  # so far only A and AAAA records are supported (on the tailscale side)
+  # See: docs/ref/dns.md
   extra_records: []
   #   - name: "grafana.myvpn.example.com"
   #     type: "A"
@@ -301,78 +316,90 @@ dns:
   #
   #   # you can also put it in one line
   #   - { name: "prometheus.myvpn.example.com", type: "A", value: "100.64.0.3" }
-
-  # DEPRECATED
-  # Use the username as part of the DNS name for nodes, with this option enabled:
-  # node1.username.example.com
-  # while when this is disabled:
-  # node1.example.com
-  # This is a legacy option as Headscale has have this wrongly implemented
-  # while in upstream Tailscale, the username is not included.
-  use_username_in_magic_dns: false
+  #
+  # Alternatively, extra DNS records can be loaded from a JSON file.
+  # Headscale processes this file on each change.
+  # extra_records_path: /var/lib/headscale/extra-records.json
 
 # Unix socket used for the CLI to connect without authentication
 # Note: for production you will want to set this to something like:
 unix_socket: /var/run/headscale/headscale.sock
 unix_socket_permission: "0770"
-#
-# headscale supports experimental OpenID connect support,
-# it is still being tested and might have some bugs, please
-# help us test it.
+
 # OpenID Connect
 # oidc:
+#   # Block startup until the identity provider is available and healthy.
 #   only_start_if_oidc_is_available: true
+#
+#   # OpenID Connect Issuer URL from the identity provider
 #   issuer: "https://your-oidc.issuer.com/path"
+#
+#   # Client ID from the identity provider
 #   client_id: "your-oidc-client-id"
+#
+#   # Client secret generated by the identity provider
+#   # Note: client_secret and client_secret_path are mutually exclusive.
 #   client_secret: "your-oidc-client-secret"
 #   # Alternatively, set `client_secret_path` to read the secret from the file.
 #   # It resolves environment variables, making integration to systemd's
 #   # `LoadCredential` straightforward:
 #   client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
-#   # client_secret and client_secret_path are mutually exclusive.
 #
-#   # The amount of time from a node is authenticated with OpenID until it
-#   # expires and needs to reauthenticate.
+#   # The amount of time a node is authenticated with OpenID until it expires
+#   # and needs to reauthenticate.
 #   # Setting the value to "0" will mean no expiry.
 #   expiry: 180d
 #
 #   # Use the expiry from the token received from OpenID when the user logged
-#   # in, this will typically lead to frequent need to reauthenticate and should
-#   # only been enabled if you know what you are doing.
+#   # in. This will typically lead to frequent need to reauthenticate and should
+#   # only be enabled if you know what you are doing.
 #   # Note: enabling this will cause `oidc.expiry` to be ignored.
 #   use_expiry_from_token: false
 #
-#   # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
-#   # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
+#   # The OIDC scopes to use, defaults to "openid", "profile" and "email".
+#   # Custom scopes can be configured as needed, be sure to always include the
+#   # required "openid" scope.
+#   scope: ["openid", "profile", "email"]
 #
-#   scope: ["openid", "profile", "email", "custom"]
+#   # Provide custom key/value pairs which get sent to the identity provider's
+#   # authorization endpoint.
 #   extra_params:
 #     domain_hint: example.com
 #
-#   # List allowed principal domains and/or users. If an authenticated user's domain is not in this list, the
-#   # authentication request will be rejected.
-#
+#   # Only accept users whose email domain is part of the allowed_domains list.
 #   allowed_domains:
 #     - example.com
-#   # Note: Groups from keycloak have a leading '/'
-#   allowed_groups:
-#     - /headscale
+#
+#   # Only accept users whose email address is part of the allowed_users list.
 #   allowed_users:
 #     - alice@example.com
 #
-#   # If `strip_email_domain` is set to `true`, the domain part of the username email address will be removed.
-#   # This will transform `first-name.last-name@example.com` to the user `first-name.last-name`
-#   # If `strip_email_domain` is set to `false` the domain part will NOT be removed resulting to the following
-#   user: `first-name.last-name.example.com`
+#   # Only accept users which are members of at least one group in the
+#   # allowed_groups list.
+#   allowed_groups:
+#     - /headscale
 #
-#   strip_email_domain: true
+#   # Optional: PKCE (Proof Key for Code Exchange) configuration
+#   # PKCE adds an additional layer of security to the OAuth 2.0 authorization code flow
+#   # by preventing authorization code interception attacks
+#   # See https://datatracker.ietf.org/doc/html/rfc7636
+#   pkce:
+#     # Enable or disable PKCE support (default: false)
+#     enabled: false
+#
+#     # PKCE method to use:
+#     # - plain: Use plain code verifier
+#     # - S256: Use SHA256 hashed code verifier (default, recommended)
+#     method: S256
 
 # Logtail configuration
-# Logtail is Tailscales logging and auditing infrastructure, it allows the control panel
-# to instruct tailscale nodes to log their activity to a remote server.
+# Logtail is Tailscales logging and auditing infrastructure, it allows the
+# control panel to instruct tailscale nodes to log their activity to a remote
+# server. To disable logging on the client side, please refer to:
+# https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of-client-logging
 logtail:
-  # Enable logtail for this headscales clients.
-  # As there is currently no support for overriding the log server in headscale, this is
+  # Enable logtail for tailscale nodes of this Headscale instance.
+  # As there is currently no support for overriding the log server in Headscale, this is
   # disabled by default. Enabling this will make your clients send logs to Tailscale Inc.
   enabled: false
 
@@ -380,3 +407,23 @@ logtail:
 # default static port 41641. This option is intended as a workaround for some buggy
 # firewall devices. See https://tailscale.com/kb/1181/firewalls/ for more information.
 randomize_client_port: false
+
+# Taildrop configuration
+# Taildrop is the file sharing feature of Tailscale, allowing nodes to send files to each other.
+# https://tailscale.com/kb/1106/taildrop/
+taildrop:
+  # Enable or disable Taildrop for all nodes.
+  # When enabled, nodes can send files to other nodes owned by the same user.
+  # Tagged devices and cross-user transfers are not permitted by Tailscale clients.
+  enabled: true
+# Advanced performance tuning parameters.
+# The defaults are carefully chosen and should rarely need adjustment.
+# Only modify these if you have identified a specific performance issue.
+#
+# tuning:
+#   # NodeStore write batching configuration.
+#   # The NodeStore batches write operations before rebuilding peer relationships,
+#   # which is computationally expensive. Batching reduces rebuild frequency.
+#   #
+#   # node_store_batch_size: 100
+#   # node_store_batch_timeout: 500ms

derp-example.yaml

@@ -1,5 +1,6 @@
 # If you plan to somehow use headscale, please deploy your own DERP infra: https://tailscale.com/kb/1118/custom-derp-servers/
 regions:
+  1: null # Disable DERP region with ID 1
   900:
     regionid: 900
     regioncode: custom
@@ -7,9 +8,9 @@ regions:
     nodes:
       - name: 900a
         regionid: 900
-        hostname: myderp.mydomain.no
-        ipv4: 123.123.123.123
-        ipv6: "2604:a880:400:d1::828:b001"
+        hostname: myderp.example.com
+        ipv4: 198.51.100.1
+        ipv6: 2001:db8::1
         stunport: 0
         stunonly: false
         derpport: 0

docs/about/clients.md

@@ -0,0 +1,16 @@
+# Client and operating system support
+
+We aim to support the [**last 10 releases** of the Tailscale client](https://tailscale.com/changelog#client) on all
+provided operating systems and platforms. Some platforms might require additional configuration to connect with
+headscale.
+
+| OS      | Supports headscale                                                                                    |
+| ------- | ----------------------------------------------------------------------------------------------------- |
+| Linux   | Yes                                                                                                   |
+| OpenBSD | Yes                                                                                                   |
+| FreeBSD | Yes                                                                                                   |
+| Windows | Yes (see [docs](../usage/connect/windows.md) and `/windows` on your headscale for more information)   |
+| Android | Yes (see [docs](../usage/connect/android.md) for more information)                                    |
+| macOS   | Yes (see [docs](../usage/connect/apple.md#macos) and `/apple` on your headscale for more information) |
+| iOS     | Yes (see [docs](../usage/connect/apple.md#ios) and `/apple` on your headscale for more information)   |
+| tvOS    | Yes (see [docs](../usage/connect/apple.md#tvos) and `/apple` on your headscale for more information)  |

docs/about/contributing.md

@@ -0,0 +1,3 @@
+{%
+    include-markdown "../../CONTRIBUTING.md"
+%}

docs/about/faq.md

@@ -0,0 +1,177 @@
+# Frequently Asked Questions
+
+## What is the design goal of headscale?
+
+Headscale aims to implement a self-hosted, open source alternative to the
+[Tailscale](https://tailscale.com/) control server. Headscale's goal is to
+provide self-hosters and hobbyists with an open-source server they can use for
+their projects and labs. It implements a narrow scope, a _single_ Tailscale
+network (tailnet), suitable for a personal use, or a small open-source
+organisation.
+
+## How can I contribute?
+
+Headscale is "Open Source, acknowledged contribution", this means that any
+contribution will have to be discussed with the Maintainers before being submitted.
+
+Please see [Contributing](contributing.md) for more information.
+
+## Why is 'acknowledged contribution' the chosen model?
+
+Both maintainers have full-time jobs and families, and we want to avoid burnout. We also want to avoid frustration from contributors when their PRs are not accepted.
+
+We are more than happy to exchange emails, or to have dedicated calls before a PR is submitted.
+
+## When/Why is Feature X going to be implemented?
+
+We don't know. We might be working on it. If you're interested in contributing, please post a feature request about it.
+
+Please be aware that there are a number of reasons why we might not accept specific contributions:
+
+- It is not possible to implement the feature in a way that makes sense in a self-hosted environment.
+- Given that we are reverse-engineering Tailscale to satisfy our own curiosity, we might be interested in implementing the feature ourselves.
+- You are not sending unit and integration tests with it.
+
+## Do you support Y method of deploying headscale?
+
+We currently support deploying headscale using our binaries and the DEB packages. Visit our [installation guide using
+official releases](../setup/install/official.md) for more information.
+
+In addition to that, you may use packages provided by the community or from distributions. Learn more in the
+[installation guide using community packages](../setup/install/community.md).
+
+For convenience, we also [build container images with headscale](../setup/install/container.md). But **please be aware that
+we don't officially support deploying headscale using Docker**. On our [Discord server](https://discord.gg/c84AZQhmpx)
+we have a "docker-issues" channel where you can ask for Docker-specific help to the community.
+
+## What is the recommended update path? Can I skip multiple versions while updating?
+
+Please follow the steps outlined in the [upgrade guide](../setup/upgrade.md) to update your existing Headscale
+installation. Its best to update from one stable version to the next (e.g. 0.24.0 → 0.25.1 → 0.26.1) in case
+you are multiple releases behind. You should always pick the latest available patch release.
+
+Be sure to check the [changelog](https://github.com/juanfont/headscale/blob/main/CHANGELOG.md) for version specific
+upgrade instructions and breaking changes.
+
+## Scaling / How many clients does Headscale support?
+
+It depends. As often stated, Headscale is not enterprise software and our focus
+is homelabbers and self-hosters. Of course, we do not prevent people from using
+it in a commercial/professional setting and often get questions about scaling.
+
+Please note that when Headscale is developed, performance is not part of the
+consideration as the main audience is considered to be users with a modest
+amount of devices. We focus on correctness and feature parity with Tailscale
+SaaS over time.
+
+To understand if you might be able to use Headscale for your use case, I will
+describe two scenarios in an effort to explain what is the central bottleneck
+of Headscale:
+
+1. An environment with 1000 servers
+
+   - they rarely "move" (change their endpoints)
+   - new nodes are added rarely
+
+2. An environment with 80 laptops/phones (end user devices)
+
+   - nodes move often, e.g. switching from home to office
+
+Headscale calculates a map of all nodes that need to talk to each other,
+creating this "world map" requires a lot of CPU time. When an event that
+requires changes to this map happens, the whole "world" is recalculated, and a
+new "world map" is created for every node in the network.
+
+This means that under certain conditions, Headscale can likely handle 100s
+of devices (maybe more), if there is _little to no change_ happening in the
+network. For example, in Scenario 1, the process of computing the world map is
+extremely demanding due to the size of the network, but when the map has been
+created and the nodes are not changing, the Headscale instance will likely
+return to a very low resource usage until the next time there is an event
+requiring the new map.
+
+In the case of Scenario 2, the process of computing the world map is less
+demanding due to the smaller size of the network, however, the type of nodes
+will likely change frequently, which would lead to a constant resource usage.
+
+Headscale will start to struggle when the two scenarios overlap, e.g. many nodes
+with frequent changes will cause the resource usage to remain constantly high.
+In the worst case scenario, the queue of nodes waiting for their map will grow
+to a point where Headscale never will be able to catch up, and nodes will never
+learn about the current state of the world.
+
+We expect that the performance will improve over time as we improve the code
+base, but it is not a focus. In general, we will never make the tradeoff to make
+things faster on the cost of less maintainable or readable code. We are a small
+team and have to optimise for maintainability.
+
+## Which database should I use?
+
+We recommend the use of SQLite as database for headscale:
+
+- SQLite is simple to setup and easy to use
+- It scales well for all of headscale's use cases
+- Development and testing happens primarily on SQLite
+- PostgreSQL is still supported, but is considered to be in "maintenance mode"
+
+The headscale project itself does not provide a tool to migrate from PostgreSQL to SQLite. Please have a look at [the
+related tools documentation](../ref/integration/tools.md) for migration tooling provided by the community.
+
+The choice of database has little to no impact on the performance of the server,
+see [Scaling / How many clients does Headscale support?](#scaling-how-many-clients-does-headscale-support) for understanding how Headscale spends its resources.
+
+## Why is my reverse proxy not working with headscale?
+
+We don't know. We don't use reverse proxies with headscale ourselves, so we don't have any experience with them. We have
+[community documentation](../ref/integration/reverse-proxy.md) on how to configure various reverse proxies, and a
+dedicated "reverse-proxy-issues" channel on our [Discord server](https://discord.gg/c84AZQhmpx) where you can ask for
+help to the community.
+
+## Can I use headscale and tailscale on the same machine?
+
+Running headscale on a machine that is also in the tailnet can cause problems with subnet routers, traffic relay nodes, and MagicDNS. It might work, but it is not supported.
+
+## Why do two nodes see each other in their status, even if an ACL allows traffic only in one direction?
+
+A frequent use case is to allow traffic only from one node to another, but not the other way around. For example, the
+workstation of an administrator should be able to connect to all nodes but the nodes themselves shouldn't be able to
+connect back to the administrator's node. Why do all nodes see the administrator's workstation in the output of
+`tailscale status`?
+
+This is essentially how Tailscale works. If traffic is allowed to flow in one direction, then both nodes see each other
+in their output of `tailscale status`. Traffic is still filtered according to the ACL, with the exception of `tailscale
+ping` which is always allowed in either direction.
+
+See also <https://tailscale.com/kb/1087/device-visibility>.
+
+## My policy is stored in the database and Headscale refuses to start due to an invalid policy. How can I recover?
+
+Headscale checks if the policy is valid during startup and refuses to start if it detects an error. The error message
+indicates which part of the policy is invalid. Follow these steps to fix your policy:
+
+- Dump the policy to a file: `headscale policy get --bypass-grpc-and-access-database-directly > policy.json`
+- Edit and fixup `policy.json`. Use the command `headscale policy check --file policy.json` to validate the policy.
+- Load the modified policy: `headscale policy set --bypass-grpc-and-access-database-directly --file policy.json`
+- Start Headscale as usual.
+
+!!! warning "Full server configuration required"
+
+    The above commands to get/set the policy require a complete server configuration file including database settings. A
+    minimal config to [control Headscale via remote CLI](../ref/api.md#grpc) is not sufficient. You may use `headscale
+    -c /path/to/config.yaml` to specify the path to an alternative configuration file.
+
+## How can I avoid to send logs to Tailscale Inc?
+
+A Tailscale client [collects logs about its operation and connection attempts with other
+clients](https://tailscale.com/kb/1011/log-mesh-traffic#client-logs) and sends them to a central log service operated by
+Tailscale Inc.
+
+Headscale, by default, instructs clients to disable log submission to the central log service. This configuration is
+applied by a client once it successfully connected with Headscale. See the configuration option `logtail.enabled` in the
+[configuration file](../ref/configuration.md) for details.
+
+Alternatively, logging can also be disabled on the client side. This is independent of Headscale and opting out of
+client logging disables log submission early during client startup. The configuration is operating system specific and
+is usually achieved by setting the environment variable `TS_NO_LOGS_NO_SUPPORT=true` or by passing the flag
+`--no-logs-no-support` to `tailscaled`. See
+<https://tailscale.com/kb/1011/log-mesh-traffic#opting-out-of-client-logging> for details.

docs/about/features.md

@@ -0,0 +1,37 @@
+# Features
+
+Headscale aims to implement a self-hosted, open source alternative to the Tailscale control server. Headscale's goal is
+to provide self-hosters and hobbyists with an open-source server they can use for their projects and labs. This page
+provides on overview of Headscale's feature and compatibility with the Tailscale control server:
+
+- [x] Full "base" support of Tailscale's features
+- [x] Node registration
+    - [x] Interactive
+    - [x] Pre authenticated key
+- [x] [DNS](../ref/dns.md)
+    - [x] [MagicDNS](https://tailscale.com/kb/1081/magicdns)
+    - [x] [Global and restricted nameservers (split DNS)](https://tailscale.com/kb/1054/dns#nameservers)
+    - [x] [search domains](https://tailscale.com/kb/1054/dns#search-domains)
+    - [x] [Extra DNS records (Headscale only)](../ref/dns.md#setting-extra-dns-records)
+- [x] [Taildrop (File Sharing)](https://tailscale.com/kb/1106/taildrop)
+- [x] [Routes](../ref/routes.md)
+    - [x] [Subnet routers](../ref/routes.md#subnet-router)
+    - [x] [Exit nodes](../ref/routes.md#exit-node)
+- [x] Dual stack (IPv4 and IPv6)
+- [x] Ephemeral nodes
+- [x] Embedded [DERP server](../ref/derp.md)
+- [x] Access control lists ([GitHub label "policy"](https://github.com/juanfont/headscale/labels/policy%20%F0%9F%93%9D))
+    - [x] ACL management via API
+    - [x] Some [Autogroups](https://tailscale.com/kb/1396/targets#autogroups), currently: `autogroup:internet`,
+      `autogroup:nonroot`, `autogroup:member`, `autogroup:tagged`, `autogroup:self`
+    - [x] [Auto approvers](https://tailscale.com/kb/1337/acl-syntax#auto-approvers) for [subnet
+      routers](../ref/routes.md#automatically-approve-routes-of-a-subnet-router) and [exit
+      nodes](../ref/routes.md#automatically-approve-an-exit-node-with-auto-approvers)
+    - [x] [Tailscale SSH](https://tailscale.com/kb/1193/tailscale-ssh)
+* [x] [Node registration using Single-Sign-On (OpenID Connect)](../ref/oidc.md) ([GitHub label "OIDC"](https://github.com/juanfont/headscale/labels/OIDC))
+    - [x] Basic registration
+    - [x] Update user profile from identity provider
+    - [ ] OIDC groups cannot be used in ACLs
+- [ ] [Funnel](https://tailscale.com/kb/1223/funnel) ([#1040](https://github.com/juanfont/headscale/issues/1040))
+- [ ] [Serve](https://tailscale.com/kb/1312/serve) ([#1234](https://github.com/juanfont/headscale/issues/1921))
+- [ ] [Network flow logs](https://tailscale.com/kb/1219/network-flow-logs) ([#1687](https://github.com/juanfont/headscale/issues/1687))

docs/about/help.md

@@ -0,0 +1,5 @@
+# Getting help
+
+Join our [Discord server](https://discord.gg/c84AZQhmpx) for announcements and community support.
+
+Please report bugs via [GitHub issues](https://github.com/juanfont/headscale/issues)

docs/about/releases.md

@@ -0,0 +1,10 @@
+# Releases
+
+All headscale releases are available on the [GitHub release page](https://github.com/juanfont/headscale/releases). Those
+releases are available as binaries for various platforms and architectures, packages for Debian based systems and source
+code archives. Container images are available on [Docker Hub](https://hub.docker.com/r/headscale/headscale) and
+[GitHub Container Registry](https://github.com/juanfont/headscale/pkgs/container/headscale).
+
+An Atom/RSS feed of headscale releases is available [here](https://github.com/juanfont/headscale/releases.atom).
+
+See the "announcements" channel on our [Discord server](https://discord.gg/c84AZQhmpx) for news about headscale.

docs/about/sponsor.md

@@ -0,0 +1,4 @@
+# Sponsor
+
+If you like to support the development of headscale, please consider a donation via
+[ko-fi.com/headscale](https://ko-fi.com/headscale). Thank you!

docs/android-client.md

@@ -1,19 +0,0 @@
-# Connecting an Android client
-
-## Goal
-
-This documentation has the goal of showing how a user can use the official Android [Tailscale](https://tailscale.com) client with `headscale`.
-
-## Installation
-
-Install the official Tailscale Android client from the [Google Play Store](https://play.google.com/store/apps/details?id=com.tailscale.ipn) or [F-Droid](https://f-droid.org/packages/com.tailscale.ipn/).
-
-Ensure that the installed version is at least 1.30.0, as that is the first release to support custom URLs.
-
-## Configuring the headscale URL
-
-After opening the app:
-
-- Open setting and go into account settings
-- In the kebab menu icon (three dots) on the top bar on the right select “Use an alternate server”
-- Enter your server URL and follow the instructions

docs/assets/logo/headscale3-dots.svg

@@ -1 +1 @@
-<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2" viewBox="0 0 1280 640"><circle cx="141.023" cy="338.36" r="117.472" style="fill:#f8b5cb" transform="matrix(.997276 0 0 1.00556 10.0024 -14.823)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 0)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.43 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.851 0)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 3.36978 -10.2458)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 255.633 -10.2458)"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030" transform="matrix(-1 0 0 1 1857.19 0)"/></svg>
+<svg xmlns="http://www.w3.org/2000/svg" xml:space="preserve" style="fill-rule:evenodd;clip-rule:evenodd;stroke-linejoin:round;stroke-miterlimit:2" viewBox="0 0 1280 640"><circle cx="141.023" cy="338.36" r="117.472" style="fill:#f8b5cb" transform="matrix(.997276 0 0 1.00556 10.0024 -14.823)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 0)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 -3.15847 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.43 115.914)"/><circle cx="352.014" cy="268.302" r="33.095" style="fill:#a2a2a2" transform="matrix(1.01749 0 0 1 148.851 0)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 3.36978 -10.2458)"/><circle cx="805.557" cy="336.915" r="118.199" style="fill:#8d8d8d" transform="matrix(.99196 0 0 1 255.633 -10.2458)"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030"/><path d="M680.282 124.808h-68.093v390.325h68.081v-28.23H640V153.228h40.282v-28.42Z" style="fill:#303030" transform="matrix(-1 0 0 1 1857.19 0)"/></svg>

docs/dns-records.md

@@ -1,92 +0,0 @@
-# Setting custom DNS records
-
-!!! warning "Community documentation"
-
-    This page is not actively maintained by the headscale authors and is
-    written by community members. It is _not_ verified by `headscale` developers.
-
-    **It might be outdated and it might miss necessary steps**.
-
-## Goal
-
-This documentation has the goal of showing how a user can set custom DNS records with `headscale`s magic dns.
-An example use case is to serve apps on the same host via a reverse proxy like NGINX, in this case a Prometheus monitoring stack. This allows to nicely access the service with "http://grafana.myvpn.example.com" instead of the hostname and portnum combination "http://hostname-in-magic-dns.myvpn.example.com:3000".
-
-## Setup
-
-### 1. Change the configuration
-
-1. Change the `config.yaml` to contain the desired records like so:
-
-    ```yaml
-    dns_config:
-      ...
-      extra_records:
-        - name: "prometheus.myvpn.example.com"
-          type: "A"
-          value: "100.64.0.3"
-
-        - name: "grafana.myvpn.example.com"
-          type: "A"
-          value: "100.64.0.3"
-      ...
-    ```
-
-1. Restart your headscale instance.
-
-    !!! warning
-
-        Beware of the limitations listed later on!
-
-### 2. Verify that the records are set
-
-You can use a DNS querying tool of your choice on one of your hosts to verify that your newly set records are actually available in MagicDNS, here we used [`dig`](https://man.archlinux.org/man/dig.1.en):
-
-```
-$ dig grafana.myvpn.example.com
-
-; <<>> DiG 9.18.10 <<>> grafana.myvpn.example.com
-;; global options: +cmd
-;; Got answer:
-;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 44054
-;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
-
-;; OPT PSEUDOSECTION:
-; EDNS: version: 0, flags:; udp: 65494
-;; QUESTION SECTION:
-;grafana.myvpn.example.com.         IN      A
-
-;; ANSWER SECTION:
-grafana.myvpn.example.com.  593     IN      A       100.64.0.3
-
-;; Query time: 0 msec
-;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
-;; WHEN: Sat Dec 31 11:46:55 CET 2022
-;; MSG SIZE  rcvd: 66
-```
-
-### 3. Optional: Setup the reverse proxy
-
-The motivating example here was to be able to access internal monitoring services on the same host without specifying a port:
-
-```
-server {
-    listen 80;
-    listen [::]:80;
-
-    server_name grafana.myvpn.example.com;
-
-    location / {
-        proxy_pass http://localhost:3000;
-        proxy_set_header Host $http_host;
-        proxy_set_header X-Real-IP $remote_addr;
-        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
-        proxy_set_header X-Forwarded-Proto $scheme;
-    }
-
-}
-```
-
-## Limitations
-
-[Not all types of records are supported](https://github.com/tailscale/tailscale/blob/6edf357b96b28ee1be659a70232c0135b2ffedfd/ipn/ipnlocal/local.go#L2989-L3007), especially no CNAME records.

docs/exit-node.md

@@ -1,49 +0,0 @@
-# Exit Nodes
-
-## On the node
-
-Register the node and make it advertise itself as an exit node:
-
-```console
-$ sudo tailscale up --login-server https://my-server.com --advertise-exit-node
-```
-
-If the node is already registered, it can advertise exit capabilities like this:
-
-```console
-$ sudo tailscale set --advertise-exit-node
-```
-
-To use a node as an exit node, IP forwarding must be enabled on the node. Check the official [Tailscale documentation](https://tailscale.com/kb/1019/subnets/?tab=linux#enable-ip-forwarding) for how to enable IP forwarding.
-
-## On the control server
-
-```console
-$ # list nodes
-$ headscale routes list
-ID | Machine | Prefix    | Advertised | Enabled | Primary
-1  |         | 0.0.0.0/0 | false      | false   | -
-2  |         | ::/0      | false      | false   | -
-3  | phobos  | 0.0.0.0/0 | true       | false   | -
-4  | phobos  | ::/0      | true       | false   | -
-$ # enable routes for phobos
-$ headscale routes enable -r 3
-$ headscale routes enable -r 4
-$ # Check node list again. The routes are now enabled.
-$ headscale routes list
-ID | Machine | Prefix    | Advertised | Enabled | Primary
-1  |         | 0.0.0.0/0 | false      | false   | -
-2  |         | ::/0      | false      | false   | -
-3  | phobos  | 0.0.0.0/0 | true       | true    | -
-4  | phobos  | ::/0      | true       | true    | -
-```
-
-## On the client
-
-The exit node can now be used with:
-
-```console
-$ sudo tailscale set --exit-node phobos
-```
-
-Check the official [Tailscale documentation](https://tailscale.com/kb/1103/exit-nodes/?q=exit#step-3-use-the-exit-node) for how to do it on your device.

docs/faq.md

@@ -1,57 +0,0 @@
----
-hide:
-  - navigation
----
-
-# Frequently Asked Questions
-
-## What is the design goal of headscale?
-
-`headscale` aims to implement a self-hosted, open source alternative to the [Tailscale](https://tailscale.com/)
-control server.
-`headscale`'s goal is to provide self-hosters and hobbyists with an open-source
-server they can use for their projects and labs.
-It implements a narrow scope, a _single_ Tailnet, suitable for a personal use, or a small
-open-source organisation.
-
-## How can I contribute?
-
-Headscale is "Open Source, acknowledged contribution", this means that any
-contribution will have to be discussed with the Maintainers before being submitted.
-
-Headscale is open to code contributions for bug fixes without discussion.
-
-If you find mistakes in the documentation, please also submit a fix to the documentation.
-
-## Why is 'acknowledged contribution' the chosen model?
-
-Both maintainers have full-time jobs and families, and we want to avoid burnout. We also want to avoid frustration from contributors when their PRs are not accepted.
-
-We are more than happy to exchange emails, or to have dedicated calls before a PR is submitted.
-
-## When/Why is Feature X going to be implemented?
-
-We don't know. We might be working on it. If you want to help, please send us a PR.
-
-Please be aware that there are a number of reasons why we might not accept specific contributions:
-
-- It is not possible to implement the feature in a way that makes sense in a self-hosted environment.
-- Given that we are reverse-engineering Tailscale to satisfy our own curiosity, we might be interested in implementing the feature ourselves.
-- You are not sending unit and integration tests with it.
-
-## Do you support Y method of deploying Headscale?
-
-We currently support deploying `headscale` using our binaries and the DEB packages. Both can be found in the
-[GitHub releases page](https://github.com/juanfont/headscale/releases).
-
-In addition to that, there are semi-official RPM packages by the Fedora infra team https://copr.fedorainfracloud.org/coprs/jonathanspw/headscale/
-
-For convenience, we also build Docker images with `headscale`. But **please be aware that we don't officially support deploying `headscale` using Docker**. We have a [Discord channel](https://discord.com/channels/896711691637780480/1070619770942148618) where you can ask for Docker-specific help to the community.
-
-## Why is my reverse proxy not working with Headscale?
-
-We don't know. We don't use reverse proxies with `headscale` ourselves, so we don't have any experience with them. We have [community documentation](https://headscale.net/reverse-proxy/) on how to configure various reverse proxies, and a dedicated [Discord channel](https://discord.com/channels/896711691637780480/1070619818346164324) where you can ask for help to the community.
-
-## Can I use headscale and tailscale on the same machine?
-
-Running headscale on a machine that is also in the tailnet can cause problems with subnet routers, traffic relay nodes, and MagicDNS. It might work, but it is not supported.

docs/glossary.md

@@ -1,6 +0,0 @@
-# Glossary
-
-| Term      | Description                                                                                                                                 |
-| --------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
-| Machine   | A machine is a single entity connected to `headscale`, typically an installation of Tailscale. Also known as **Node**                       |
-| Namespace | A namespace was a logical grouping of machines "owned" by the same entity, in Tailscale, this is typically a User (This is now called user) |

docs/iOS-client.md

@@ -1,30 +0,0 @@
-# Connecting an iOS client
-
-## Goal
-
-This documentation has the goal of showing how a user can use the official iOS [Tailscale](https://tailscale.com) client with `headscale`.
-
-## Installation
-
-Install the official Tailscale iOS client from the [App Store](https://apps.apple.com/app/tailscale/id1470499037).
-
-Ensure that the installed version is at least 1.38.1, as that is the first release to support alternate control servers.
-
-## Configuring the headscale URL
-
-!!! info "Apple devices"
-
-    An endpoint with information on how to connect your Apple devices
-    (currently macOS only) is available at `/apple` on your running instance.
-
-Ensure that the tailscale app is logged out before proceeding.
-
-Go to iOS settings, scroll down past game center and tv provider to the tailscale app and select it. The headscale URL can be entered into the _"ALTERNATE COORDINATION SERVER URL"_ box.
-
-> **Note**
->
-> If the app was previously logged into tailscale, toggle on the _Reset Keychain_ switch.
-
-Restart the app by closing it from the iOS app switcher, open the app and select the regular _Sign in_ option (non-SSO), and it should open up to the headscale authentication page.
-
-Enter your credentials and log in. Headscale should now be working on your iOS device.

docs/index.md

@@ -4,40 +4,34 @@ hide:
   - toc
 ---
 
-# headscale
+# Welcome to headscale
 
-`headscale` is an open source, self-hosted implementation of the Tailscale control server.
+Headscale is an open source, self-hosted implementation of the Tailscale control server.
 
-This page contains the documentation for the latest version of headscale. Please also check our [FAQ](faq.md).
+This page contains the documentation for the latest version of headscale. Please also check our [FAQ](./about/faq.md).
 
-Join our [Discord](https://discord.gg/c84AZQhmpx) server for a chat and community support.
+Join our [Discord server](https://discord.gg/c84AZQhmpx) for a chat and community support.
 
 ## Design goal
 
-Headscale aims to implement a self-hosted, open source alternative to the Tailscale
-control server.
-Headscale's goal is to provide self-hosters and hobbyists with an open-source
-server they can use for their projects and labs.
-It implements a narrower scope, a single Tailnet, suitable for a personal use, or a small
-open-source organisation.
+Headscale aims to implement a self-hosted, open source alternative to the
+[Tailscale](https://tailscale.com/) control server. Headscale's goal is to
+provide self-hosters and hobbyists with an open-source server they can use for
+their projects and labs. It implements a narrow scope, a _single_ Tailscale
+network (tailnet), suitable for a personal use, or a small open-source
+organisation.
 
 ## Supporting headscale
 
-If you like `headscale` and find it useful, there is a sponsorship and donation
-buttons available in the repo.
+Please see [Sponsor](about/sponsor.md) for more information.
 
 ## Contributing
 
 Headscale is "Open Source, acknowledged contribution", this means that any
 contribution will have to be discussed with the Maintainers before being submitted.
 
-This model has been chosen to reduce the risk of burnout by limiting the
-maintenance overhead of reviewing and validating third-party code.
-
-Headscale is open to code contributions for bug fixes without discussion.
-
-If you find mistakes in the documentation, please submit a fix to the documentation.
+Please see [Contributing](about/contributing.md) for more information.
 
 ## About
 
-`headscale` is maintained by [Kristoffer Dalby](https://kradalby.no/) and [Juan Font](https://font.eu).
+Headscale is maintained by [Kristoffer Dalby](https://kradalby.no/) and [Juan Font](https://font.eu).

docs/oidc.md

@@ -1,174 +0,0 @@
-# Configuring Headscale to use OIDC authentication
-
-In order to authenticate users through a centralized solution one must enable the OIDC integration.
-
-Known limitations:
-
-- No dynamic ACL support
-- OIDC groups cannot be used in ACLs
-
-## Basic configuration
-
-In your `config.yaml`, customize this to your liking:
-
-```yaml
-oidc:
-  # Block further startup until the OIDC provider is healthy and available
-  only_start_if_oidc_is_available: true
-  # Specified by your OIDC provider
-  issuer: "https://your-oidc.issuer.com/path"
-  # Specified/generated by your OIDC provider
-  client_id: "your-oidc-client-id"
-  client_secret: "your-oidc-client-secret"
-  # alternatively, set `client_secret_path` to read the secret from the file.
-  # It resolves environment variables, making integration to systemd's
-  # `LoadCredential` straightforward:
-  #client_secret_path: "${CREDENTIALS_DIRECTORY}/oidc_client_secret"
-  # as third option, it's also possible to load the oidc secret from environment variables
-  # set HEADSCALE_OIDC_CLIENT_SECRET to the required value
-
-  # Customize the scopes used in the OIDC flow, defaults to "openid", "profile" and "email" and add custom query
-  # parameters to the Authorize Endpoint request. Scopes default to "openid", "profile" and "email".
-  scope: ["openid", "profile", "email", "custom"]
-  # Optional: Passed on to the browser login request – used to tweak behaviour for the OIDC provider
-  extra_params:
-    domain_hint: example.com
-
-  # Optional: List allowed principal domains and/or users. If an authenticated user's domain is not in this list,
-  # the authentication request will be rejected.
-  allowed_domains:
-    - example.com
-  # Optional. Note that groups from Keycloak have a leading '/'.
-  allowed_groups:
-    - /headscale
-  # Optional.
-  allowed_users:
-    - alice@example.com
-
-  # If `strip_email_domain` is set to `true`, the domain part of the username email address will be removed.
-  # This will transform `first-name.last-name@example.com` to the user `first-name.last-name`
-  # If `strip_email_domain` is set to `false` the domain part will NOT be removed resulting to the following
-  # user: `first-name.last-name.example.com`
-  strip_email_domain: true
-```
-
-## Azure AD example
-
-In order to integrate Headscale with Azure Active Directory, we'll need to provision an App Registration with the correct scopes and redirect URI. Here with Terraform:
-
-```hcl
-resource "azuread_application" "headscale" {
-  display_name = "Headscale"
-
-  sign_in_audience = "AzureADMyOrg"
-  fallback_public_client_enabled = false
-
-  required_resource_access {
-    // Microsoft Graph
-    resource_app_id = "00000003-0000-0000-c000-000000000000"
-
-    resource_access {
-      // scope: profile
-      id   = "14dad69e-099b-42c9-810b-d002981feec1"
-      type = "Scope"
-    }
-    resource_access {
-      // scope: openid
-      id   = "37f7f235-527c-4136-accd-4a02d197296e"
-      type = "Scope"
-    }
-    resource_access {
-      // scope: email
-      id   = "64a6cdd6-aab1-4aaf-94b8-3cc8405e90d0"
-      type = "Scope"
-    }
-  }
-  web {
-    # Points at your running Headscale instance
-    redirect_uris = ["https://headscale.example.com/oidc/callback"]
-
-    implicit_grant {
-      access_token_issuance_enabled = false
-      id_token_issuance_enabled = true
-    }
-  }
-
-  group_membership_claims = ["SecurityGroup"]
-  optional_claims {
-    # Expose group memberships
-    id_token {
-      name = "groups"
-    }
-  }
-}
-
-resource "azuread_application_password" "headscale-application-secret" {
-  display_name          = "Headscale Server"
-  application_object_id = azuread_application.headscale.object_id
-}
-
-resource "azuread_service_principal" "headscale" {
-  application_id = azuread_application.headscale.application_id
-}
-
-resource "azuread_service_principal_password" "headscale" {
-  service_principal_id = azuread_service_principal.headscale.id
-  end_date_relative    = "44640h"
-}
-
-output "headscale_client_id" {
-  value = azuread_application.headscale.application_id
-}
-
-output "headscale_client_secret" {
-  value = azuread_application_password.headscale-application-secret.value
-}
-```
-
-And in your Headscale `config.yaml`:
-
-```yaml
-oidc:
-  issuer: "https://login.microsoftonline.com/<tenant-UUID>/v2.0"
-  client_id: "<client-id-from-terraform>"
-  client_secret: "<client-secret-from-terraform>"
-
-  # Optional: add "groups"
-  scope: ["openid", "profile", "email"]
-  extra_params:
-    # Use your own domain, associated with Azure AD
-    domain_hint: example.com
-    # Optional: Force the Azure AD account picker
-    prompt: select_account
-```
-
-## Google OAuth Example
-
-In order to integrate Headscale with Google, you'll need to have a [Google Cloud Console](https://console.cloud.google.com) account.
-
-Google OAuth has a [verification process](https://support.google.com/cloud/answer/9110914?hl=en) if you need to have users authenticate who are outside of your domain. If you only need to authenticate users from your domain name (ie `@example.com`), you don't need to go through the verification process.
-
-However if you don't have a domain, or need to add users outside of your domain, you can manually add emails via Google Console.
-
-### Steps
-
-1. Go to [Google Console](https://console.cloud.google.com) and login or create an account if you don't have one.
-2. Create a project (if you don't already have one).
-3. On the left hand menu, go to `APIs and services` -> `Credentials`
-4. Click `Create Credentials` -> `OAuth client ID`
-5. Under `Application Type`, choose `Web Application`
-6. For `Name`, enter whatever you like
-7. Under `Authorised redirect URIs`, use `https://example.com/oidc/callback`, replacing example.com with your Headscale URL.
-8. Click `Save` at the bottom of the form
-9. Take note of the `Client ID` and `Client secret`, you can also download it for reference if you need it.
-10. Edit your headscale config, under `oidc`, filling in your `client_id` and `client_secret`:
-
-```yaml
-oidc:
-  issuer: "https://accounts.google.com"
-  client_id: ""
-  client_secret: ""
-  scope: ["openid", "profile", "email"]
-```
-
-You can also use `allowed_domains` and `allowed_users` to restrict the users who can authenticate.

docs/packaging/README.md

@@ -1,5 +0,0 @@
-# Packaging
-
-We use [nFPM](https://nfpm.goreleaser.com/) for making `.deb`, `.rpm` and `.apk`.
-
-This folder contains files we need to package with these releases.

docs/packaging/postinstall.sh

@@ -1,86 +0,0 @@
-#!/bin/sh
-# Determine OS platform
-# shellcheck source=/dev/null
-. /etc/os-release
-
-HEADSCALE_EXE="/usr/bin/headscale"
-BSD_HIER=""
-HEADSCALE_RUN_DIR="/var/run/headscale"
-HEADSCALE_USER="headscale"
-HEADSCALE_GROUP="headscale"
-
-ensure_sudo() {
-	if [ "$(id -u)" = "0" ]; then
-		echo "Sudo permissions detected"
-	else
-		echo "No sudo permission detected, please run as sudo"
-		exit 1
-	fi
-}
-
-ensure_headscale_path() {
-	if [ ! -f "$HEADSCALE_EXE" ]; then
-		echo "headscale not in default path, exiting..."
-		exit 1
-	fi
-
-	printf "Found headscale %s\n" "$HEADSCALE_EXE"
-}
-
-create_headscale_user() {
-	printf "PostInstall: Adding headscale user %s\n" "$HEADSCALE_USER"
-	useradd -s /bin/sh -c "headscale default user" headscale
-}
-
-create_headscale_group() {
-	if command -V systemctl >/dev/null 2>&1; then
-		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
-		groupadd "$HEADSCALE_GROUP"
-
-		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-		usermod -a -G "$HEADSCALE_GROUP" "$HEADSCALE_USER"
-	fi
-
-	if [ "$ID" = "alpine" ]; then
-		printf "PostInstall: Adding headscale group %s\n" "$HEADSCALE_GROUP"
-		addgroup "$HEADSCALE_GROUP"
-
-		printf "PostInstall: Adding headscale user %s to group %s\n" "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-		addgroup "$HEADSCALE_USER" "$HEADSCALE_GROUP"
-	fi
-}
-
-create_run_dir() {
-	printf "PostInstall: Creating headscale run directory \n"
-	mkdir -p "$HEADSCALE_RUN_DIR"
-
-	printf "PostInstall: Modifying group ownership of headscale run directory \n"
-	chown "$HEADSCALE_USER":"$HEADSCALE_GROUP" "$HEADSCALE_RUN_DIR"
-}
-
-summary() {
-	echo "----------------------------------------------------------------------"
-	echo " headscale package has been successfully installed."
-	echo ""
-	echo " Please follow the next steps to start the software:"
-	echo ""
-	echo "    sudo systemctl enable headscale"
-	echo "    sudo systemctl start headscale"
-	echo ""
-	echo " Configuration settings can be adjusted here:"
-	echo "    ${BSD_HIER}/etc/headscale/config.yaml"
-	echo ""
-	echo "----------------------------------------------------------------------"
-}
-
-#
-# Main body of the script
-#
-{
-	ensure_sudo
-	ensure_headscale_path
-	create_headscale_user
-	create_headscale_group
-	create_run_dir
-	summary
-}

docs/packaging/postremove.sh

@@ -1,15 +0,0 @@
-#!/bin/sh
-# Determine OS platform
-# shellcheck source=/dev/null
-. /etc/os-release
-
-if command -V systemctl >/dev/null 2>&1; then
-	echo "Stop and disable headscale service"
-	systemctl stop headscale >/dev/null 2>&1 || true
-	systemctl disable headscale >/dev/null 2>&1 || true
-	echo "Running daemon-reload"
-	systemctl daemon-reload || true
-fi
-
-echo "Removing run directory"
-rm -rf "/var/run/headscale.sock"

docs/proposals/001-acls.md

@@ -1,362 +0,0 @@
-# ACLs
-
-A key component of tailscale is the notion of Tailnet. This notion is hidden
-but the implications that it have on how to use tailscale are not.
-
-For tailscale an [tailnet](https://tailscale.com/kb/1136/tailnet/) is the
-following:
-
-> For personal users, you are a tailnet of many devices and one person. Each
-> device gets a private Tailscale IP address in the CGNAT range and every
-> device can talk directly to every other device, wherever they are on the
-> internet.
->
-> For businesses and organizations, a tailnet is many devices and many users.
-> It can be based on your Microsoft Active Directory, your Google Workspace, a
-> GitHub organization, Okta tenancy, or other identity provider namespace. All
-> of the devices and users in your tailnet can be seen by the tailnet
-> administrators in the Tailscale admin console. There you can apply
-> tailnet-wide configuration, such as ACLs that affect visibility of devices
-> inside your tailnet, DNS settings, and more.
-
-## Current implementation and issues
-
-Currently in headscale, the namespaces are used both as tailnet and users. The
-issue is that if we want to use the ACL's we can't use both at the same time.
-
-Tailnet's cannot communicate with each others. So we can't have an ACL that
-authorize tailnet (namespace) A to talk to tailnet (namespace) B.
-
-We also can't write ACLs based on the users (namespaces in headscale) since all
-devices belong to the same user.
-
-With the current implementation the only ACL that we can user is to associate
-each headscale IP to a host manually then write the ACLs according to this
-manual mapping.
-
-```json
-{
-  "hosts": {
-    "host1": "100.64.0.1",
-    "server": "100.64.0.2"
-  },
-  "acls": [
-    { "action": "accept", "users": ["host1"], "ports": ["host2:80,443"] }
-  ]
-}
-```
-
-While this works, it requires a lot of manual editing on the configuration and
-to keep track of all devices IP address.
-
-## Proposition for a next implementation
-
-In order to ease the use of ACL's we need to split the tailnet and users
-notion.
-
-A solution could be to consider a headscale server (in it's entirety) as a
-tailnet.
-
-For personal users the default behavior could either allow all communications
-between all namespaces (like tailscale) or disallow all communications between
-namespaces (current behavior).
-
-For businesses and organisations, viewing a headscale instance a single tailnet
-would allow users (namespace) to talk to each other with the ACLs. As described
-in tailscale's documentation [[1]], a server should be tagged and personal
-devices should be tied to a user. Translated in headscale's terms each user can
-have multiple devices and all those devices should be in the same namespace.
-The servers should be tagged and used as such.
-
-This implementation would render useless the sharing feature that is currently
-implemented since an ACL could do the same. Simplifying to only one user
-interface to do one thing is easier and less confusing for the users.
-
-To better suit the ACLs in this proposition, it's advised to consider that each
-namespaces belong to one person. This person can have multiple devices, they
-will all be considered as the same user in the ACLs. OIDC feature wouldn't need
-to map people to namespace, just create a namespace if the person isn't
-registered yet.
-
-As a sidenote, users would like to write ACLs as YAML. We should offer users
-the ability to rules in either format (HuJSON or YAML).
-
-[1]: https://tailscale.com/kb/1068/acl-tags/
-
-## Example
-
-Let's build an example use case for a small business (It may be the place where
-ACL's are the most useful).
-
-We have a small company with a boss, an admin, two developer and an intern.
-
-The boss should have access to all servers but not to the users hosts. Admin
-should also have access to all hosts except that their permissions should be
-limited to maintaining the hosts (for example purposes). The developers can do
-anything they want on dev hosts, but only watch on productions hosts. Intern
-can only interact with the development servers.
-
-Each user have at least a device connected to the network and we have some
-servers.
-
-- database.prod
-- database.dev
-- app-server1.prod
-- app-server1.dev
-- billing.internal
-
-### Current headscale implementation
-
-Let's create some namespaces
-
-```bash
-headscale namespaces create prod
-headscale namespaces create dev
-headscale namespaces create internal
-headscale namespaces create users
-
-headscale nodes register -n users boss-computer
-headscale nodes register -n users admin1-computer
-headscale nodes register -n users dev1-computer
-headscale nodes register -n users dev1-phone
-headscale nodes register -n users dev2-computer
-headscale nodes register -n users intern1-computer
-
-headscale nodes register -n prod database
-headscale nodes register -n prod app-server1
-
-headscale nodes register -n dev database
-headscale nodes register -n dev app-server1
-
-headscale nodes register -n internal billing
-
-headscale nodes list
-ID  | Name              | Namespace | IP address
-1   | boss-computer     | users     | 100.64.0.1
-2   | admin1-computer   | users     | 100.64.0.2
-3   | dev1-computer     | users     | 100.64.0.3
-4   | dev1-phone        | users     | 100.64.0.4
-5   | dev2-computer     | users     | 100.64.0.5
-6   | intern1-computer  | users     | 100.64.0.6
-7   | database          | prod      | 100.64.0.7
-8   | app-server1       | prod      | 100.64.0.8
-9   | database          | dev       | 100.64.0.9
-10  | app-server1       | dev       | 100.64.0.10
-11  | internal          | internal  | 100.64.0.11
-```
-
-In order to only allow the communications related to our description above we
-need to add the following ACLs
-
-```json
-{
-  "hosts": {
-    "boss-computer": "100.64.0.1",
-    "admin1-computer": "100.64.0.2",
-    "dev1-computer": "100.64.0.3",
-    "dev1-phone": "100.64.0.4",
-    "dev2-computer": "100.64.0.5",
-    "intern1-computer": "100.64.0.6",
-    "prod-app-server1": "100.64.0.8"
-  },
-  "groups": {
-    "group:dev": ["dev1-computer", "dev1-phone", "dev2-computer"],
-    "group:admin": ["admin1-computer"],
-    "group:boss": ["boss-computer"],
-    "group:intern": ["intern1-computer"]
-  },
-  "acls": [
-    // boss have access to all servers but no users hosts
-    {
-      "action": "accept",
-      "users": ["group:boss"],
-      "ports": ["prod:*", "dev:*", "internal:*"]
-    },
-
-    // admin have access to administration port (lets only consider port 22 here)
-    {
-      "action": "accept",
-      "users": ["group:admin"],
-      "ports": ["prod:22", "dev:22", "internal:22"]
-    },
-
-    // dev can do anything on dev servers and check access on prod servers
-    {
-      "action": "accept",
-      "users": ["group:dev"],
-      "ports": ["dev:*", "prod-app-server1:80,443"]
-    },
-
-    // interns only have access to port 80 and 443 on dev servers (lame internship)
-    { "action": "accept", "users": ["group:intern"], "ports": ["dev:80,443"] },
-
-    // users can access their own devices
-    {
-      "action": "accept",
-      "users": ["dev1-computer"],
-      "ports": ["dev1-phone:*"]
-    },
-    {
-      "action": "accept",
-      "users": ["dev1-phone"],
-      "ports": ["dev1-computer:*"]
-    },
-
-    // internal namespace communications should still be allowed within the namespace
-    { "action": "accept", "users": ["dev"], "ports": ["dev:*"] },
-    { "action": "accept", "users": ["prod"], "ports": ["prod:*"] },
-    { "action": "accept", "users": ["internal"], "ports": ["internal:*"] }
-  ]
-}
-```
-
-Since communications between namespace isn't possible we also have to share the
-devices between the namespaces.
-
-```bash
-
-// add boss host to prod, dev and internal network
-headscale nodes share -i 1 -n prod
-headscale nodes share -i 1 -n dev
-headscale nodes share -i 1 -n internal
-
-// add admin computer to prod, dev and internal network
-headscale nodes share -i 2 -n prod
-headscale nodes share -i 2 -n dev
-headscale nodes share -i 2 -n internal
-
-// add all dev to prod and dev network
-headscale nodes share -i 3 -n dev
-headscale nodes share -i 4 -n dev
-headscale nodes share -i 3 -n prod
-headscale nodes share -i 4 -n prod
-headscale nodes share -i 5 -n dev
-headscale nodes share -i 5 -n prod
-
-headscale nodes share -i 6 -n dev
-```
-
-This fake network have not been tested but it should work. Operating it could
-be quite tedious if the company grows. Each time a new user join we have to add
-it to a group, and share it to the correct namespaces. If the user want
-multiple devices we have to allow communication to each of them one by one. If
-business conduct a change in the organisations we may have to rewrite all acls
-and reorganise all namespaces.
-
-If we add servers in production we should also update the ACLs to allow dev
-access to certain category of them (only app servers for example).
-
-### example based on the proposition in this document
-
-Let's create the namespaces
-
-```bash
-headscale namespaces create boss
-headscale namespaces create admin1
-headscale namespaces create dev1
-headscale namespaces create dev2
-headscale namespaces create intern1
-```
-
-We don't need to create namespaces for the servers because the servers will be
-tagged. When registering the servers we will need to add the flag
-`--advertised-tags=tag:<tag1>,tag:<tag2>`, and the user (namespace) that is
-registering the server should be allowed to do it. Since anyone can add tags to
-a server they can register, the check of the tags is done on headscale server
-and only valid tags are applied. A tag is valid if the namespace that is
-registering it is allowed to do it.
-
-Here are the ACL's to implement the same permissions as above:
-
-```json
-{
-  // groups are simpler and only list the namespaces name
-  "groups": {
-    "group:boss": ["boss"],
-    "group:dev": ["dev1", "dev2"],
-    "group:admin": ["admin1"],
-    "group:intern": ["intern1"]
-  },
-  "tagOwners": {
-    // the administrators can add servers in production
-    "tag:prod-databases": ["group:admin"],
-    "tag:prod-app-servers": ["group:admin"],
-
-    // the boss can tag any server as internal
-    "tag:internal": ["group:boss"],
-
-    // dev can add servers for dev purposes as well as admins
-    "tag:dev-databases": ["group:admin", "group:dev"],
-    "tag:dev-app-servers": ["group:admin", "group:dev"]
-
-    // interns cannot add servers
-  },
-  "acls": [
-    // boss have access to all servers
-    {
-      "action": "accept",
-      "users": ["group:boss"],
-      "ports": [
-        "tag:prod-databases:*",
-        "tag:prod-app-servers:*",
-        "tag:internal:*",
-        "tag:dev-databases:*",
-        "tag:dev-app-servers:*"
-      ]
-    },
-
-    // admin have only access to administrative ports of the servers
-    {
-      "action": "accept",
-      "users": ["group:admin"],
-      "ports": [
-        "tag:prod-databases:22",
-        "tag:prod-app-servers:22",
-        "tag:internal:22",
-        "tag:dev-databases:22",
-        "tag:dev-app-servers:22"
-      ]
-    },
-
-    {
-      "action": "accept",
-      "users": ["group:dev"],
-      "ports": [
-        "tag:dev-databases:*",
-        "tag:dev-app-servers:*",
-        "tag:prod-app-servers:80,443"
-      ]
-    },
-
-    // servers should be able to talk to database. Database should not be able to initiate connections to server
-    {
-      "action": "accept",
-      "users": ["tag:dev-app-servers"],
-      "ports": ["tag:dev-databases:5432"]
-    },
-    {
-      "action": "accept",
-      "users": ["tag:prod-app-servers"],
-      "ports": ["tag:prod-databases:5432"]
-    },
-
-    // interns have access to dev-app-servers only in reading mode
-    {
-      "action": "accept",
-      "users": ["group:intern"],
-      "ports": ["tag:dev-app-servers:80,443"]
-    },
-
-    // we still have to allow internal namespaces communications since nothing guarantees that each user have their own namespaces. This could be talked over.
-    { "action": "accept", "users": ["boss"], "ports": ["boss:*"] },
-    { "action": "accept", "users": ["dev1"], "ports": ["dev1:*"] },
-    { "action": "accept", "users": ["dev2"], "ports": ["dev2:*"] },
-    { "action": "accept", "users": ["admin1"], "ports": ["admin1:*"] },
-    { "action": "accept", "users": ["intern1"], "ports": ["intern1:*"] }
-  ]
-}
-```
-
-With this implementation, the sharing step is not necessary. Maintenance cost
-of the ACL file is lower and less tedious (no need to map hostname and IP's
-into it).

docs/proposals/002-better-routing.md

@@ -1,48 +0,0 @@
-# Better route management
-
-As of today, route management in Headscale is very basic and does not allow for much flexibility, including implementing subnet HA, 4via6 or more advanced features. We also have a number of bugs (e.g., routes exposed by ephemeral nodes)
-
-This proposal aims to improve the route management.
-
-## Current situation
-
-Routes advertised by the nodes are read from the Hostinfo struct. If approved from the the CLI or via autoApprovers, the route is added to the EnabledRoutes field in `Machine`.
-
-This means that the advertised routes are not persisted in the database, as Hostinfo is always replaced. In the same way, EnabledRoutes can get out of sync with the actual routes in the node.
-
-In case of colliding routes (i.e., subnets that are exposed from multiple nodes), we are currently just sending all of them in `PrimaryRoutes`... and hope for the best. (`PrimaryRoutes` is the field in `Node` used for subnet failover).
-
-## Proposal
-
-The core part is to create a new `Route` struct (and DB table), with the following fields:
-
-```go
-type Route struct {
-	ID          uint64 `gorm:"primary_key"`
-
-    Machine     *Machine
-    Prefix      IPPrefix
-
-    Advertised  bool
-    Enabled     bool
-    IsPrimary   bool
-
-
-    CreatedAt   *time.Time
-    UpdatedAt   *time.Time
-    DeletedAt   *time.Time
-}
-```
-
-- The `Advertised` field is set to true if the route is being advertised by the node. It is set to false if the route is removed. This way we can indicate if a later enabled route has stopped being advertised. A similar behaviour happens in the Tailscale.com control panel.
-
-- The `Enabled` field is set to true if the route is enabled - via CLI or autoApprovers.
-
-- `IsPrimary` indicates if Headscale has selected this route as the primary route for that particular subnet. This allows us to implement subnet failover. This would be fully automatic if there is more than subnet routers advertising the same network - which is the behaviour of Tailscale.com.
-
-## Stuff to bear in mind
-
-- We need to make sure to migrate the current `EnabledRoutes` of `Machine` into the new table.
-- When a node stops sharing a subnet, I reckon we should mark it both as not `Advertised` and not `Enabled`. Users should re-enable it if the node advertises it again.
-- If only one subnet router is advertising a subnet, we should mark it as primary.
-- Regarding subnet failover, the current behaviour of Tailscale.com is to perform the failover after 15 seconds from the node disconnecting from their control panel. I reckon we cannot do the same currently. Our maximum granularity is the keep alive period.

docs/ref/acls.md

@@ -3,15 +3,44 @@ Headscale implements the same policy ACLs as Tailscale.com, adapted to the self-
 For instance, instead of referring to users when defining groups you must
 use users (which are the equivalent to user/logins in Tailscale.com).
 
-Please check https://tailscale.com/kb/1018/acls/, and `./tests/acls/` in this repo for working examples.
+Please check https://tailscale.com/kb/1018/acls/ for further information.
 
 When using ACL's the User borders are no longer applied. All machines
 whichever the User have the ability to communicate with other hosts as
 long as the ACL's permits this exchange.
 
-## ACLs use case example
+## ACL Setup
 
-Let's build an example use case for a small business (It may be the place where
+To enable and configure ACLs in Headscale, you need to specify the path to your ACL policy file in the `policy.path` key in `config.yaml`.
+
+Your ACL policy file must be formatted using [huJSON](https://github.com/tailscale/hujson).
+
+Info on how these policies are written can be found
+[here](https://tailscale.com/kb/1018/acls/).
+
+Please reload or restart Headscale after updating the ACL file. Headscale may be reloaded either via its systemd service
+(`sudo systemctl reload headscale`) or by sending a SIGHUP signal (`sudo kill -HUP $(pidof headscale)`) to the main
+process. Headscale logs the result of ACL policy processing after each reload.
+
+## Simple Examples
+
+- [**Allow All**](https://tailscale.com/kb/1192/acl-samples#allow-all-default-acl): If you define an ACL file but completely omit the `"acls"` field from its content, Headscale will default to an "allow all" policy. This means all devices connected to your tailnet will be able to communicate freely with each other.
+
+    ```json
+    {}
+    ```
+
+- [**Deny All**](https://tailscale.com/kb/1192/acl-samples#deny-all): To prevent all communication within your tailnet, you can include an empty array for the `"acls"` field in your policy file.
+
+    ```json
+    {
+      "acls": []
+    }
+    ```
+
+## Complex Example
+
+Let's build a more complex example use case for a small business (It may be the place where
 ACL's are the most useful).
 
 We have a small company with a boss, an admin, two developers and an intern.
@@ -36,36 +65,26 @@ servers.
 - billing.internal
 - router.internal
 
-![ACL implementation example](images/headscale-acl-network.png)
+![ACL implementation example](../assets/images/headscale-acl-network.png)
 
-## ACL setup
-
-Note: Users will be created automatically when users authenticate with the
-Headscale server.
-
-ACLs could be written either on [huJSON](https://github.com/tailscale/hujson)
-or YAML. Check the [test ACLs](../tests/acls) for further information.
-
-When registering the servers we will need to add the flag
-`--advertise-tags=tag:<tag1>,tag:<tag2>`, and the user that is
-registering the server should be allowed to do it. Since anyone can add tags to
-a server they can register, the check of the tags is done on headscale server
-and only valid tags are applied. A tag is valid if the user that is
+When [registering the servers](../usage/getting-started.md#register-a-node) we
+will need to add the flag `--advertise-tags=tag:<tag1>,tag:<tag2>`, and the user
+that is registering the server should be allowed to do it. Since anyone can add
+tags to a server they can register, the check of the tags is done on headscale
+server and only valid tags are applied. A tag is valid if the user that is
 registering it is allowed to do it.
 
-To use ACLs in headscale, you must edit your config.yaml file. In there you will find a `acl_policy_path: ""` parameter. This will need to point to your ACL file. More info on how these policies are written can be found [here](https://tailscale.com/kb/1018/acls/).
-
 Here are the ACL's to implement the same permissions as above:
 
-```json
+```json title="acl.json"
 {
   // groups are collections of users having a common scope. A user can be in multiple groups
   // groups cannot be composed of groups
   "groups": {
-    "group:boss": ["boss"],
-    "group:dev": ["dev1", "dev2"],
-    "group:admin": ["admin1"],
-    "group:intern": ["intern1"]
+    "group:boss": ["boss@"],
+    "group:dev": ["dev1@", "dev2@"],
+    "group:admin": ["admin1@"],
+    "group:intern": ["intern1@"]
   },
   // tagOwners in tailscale is an association between a TAG and the people allowed to set this TAG on a server.
   // This is documented [here](https://tailscale.com/kb/1068/acl-tags#defining-a-tag)
@@ -88,7 +107,7 @@ Here are the ACL's to implement the same permissions as above:
   // to define a single host, use a /32 mask. You cannot use DNS entries here,
   // as they're prone to be hijacked by replacing their IP addresses.
   // see https://github.com/tailscale/tailscale/issues/3800 for more information.
-  "Hosts": {
+  "hosts": {
     "postgresql.internal": "10.20.0.2/32",
     "webservers.internal": "10.20.10.1/29"
   },
@@ -147,13 +166,11 @@ Here are the ACL's to implement the same permissions as above:
     },
     // developers have access to the internal network through the router.
     // the internal network is composed of HTTPS endpoints and Postgresql
-    // database servers. There's an additional rule to allow traffic to be
-    // forwarded to the internal subnet, 10.20.0.0/16. See this issue
-    // https://github.com/juanfont/headscale/issues/502
+    // database servers.
     {
       "action": "accept",
       "src": ["group:dev"],
-      "dst": ["10.20.0.0/16:443,5432", "router.internal:0"]
+      "dst": ["10.20.0.0/16:443,5432"]
     },
 
     // servers should be able to talk to database in tcp/5432. Database should not be able to initiate connections to
@@ -177,13 +194,94 @@ Here are the ACL's to implement the same permissions as above:
       "dst": ["tag:dev-app-servers:80,443"]
     },
 
-    // We still have to allow internal users communications since nothing guarantees that each user have
-    // their own users.
-    { "action": "accept", "src": ["boss"], "dst": ["boss:*"] },
-    { "action": "accept", "src": ["dev1"], "dst": ["dev1:*"] },
-    { "action": "accept", "src": ["dev2"], "dst": ["dev2:*"] },
-    { "action": "accept", "src": ["admin1"], "dst": ["admin1:*"] },
-    { "action": "accept", "src": ["intern1"], "dst": ["intern1:*"] }
+    // Allow users to access their own devices using autogroup:self (see below for more details about performance impact)
+    {
+      "action": "accept",
+      "src": ["autogroup:member"],
+      "dst": ["autogroup:self:*"]
+    }
   ]
 }
 ```
+
+## Autogroups
+
+Headscale supports several autogroups that automatically include users, destinations, or devices with specific properties. Autogroups provide a convenient way to write ACL rules without manually listing individual users or devices.
+
+### `autogroup:internet`
+
+Allows access to the internet through [exit nodes](routes.md#exit-node). Can only be used in ACL destinations.
+
+```json
+{
+  "action": "accept",
+  "src": ["group:users"],
+  "dst": ["autogroup:internet:*"]
+}
+```
+
+### `autogroup:member`
+
+Includes all users who are direct members of the tailnet. Does not include users from shared devices.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["tag:prod-app-servers:80,443"]
+}
+```
+
+### `autogroup:tagged`
+
+Includes all devices that have at least one tag.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:tagged"],
+  "dst": ["tag:monitoring:9090"]
+}
+```
+
+### `autogroup:self`
+**(EXPERIMENTAL)**
+
+!!! warning "The current implementation of `autogroup:self` is inefficient"
+
+Includes devices where the same user is authenticated on both the source and destination. Does not include tagged devices. Can only be used in ACL destinations.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["autogroup:self:*"]
+}
+```
+*Using `autogroup:self` may cause performance degradation on the Headscale coordinator server in large deployments, as filter rules must be compiled per-node rather than globally and the current implementation is not very efficient.*
+
+If you experience performance issues, consider using more specific ACL rules or limiting the use of `autogroup:self`.
+```json
+{
+  // The following rules allow internal users to communicate with their
+  // own nodes in case autogroup:self is causing performance issues.
+  { "action": "accept", "src": ["boss@"], "dst": ["boss@:*"] },
+  { "action": "accept", "src": ["dev1@"], "dst": ["dev1@:*"] },
+  { "action": "accept", "src": ["dev2@"], "dst": ["dev2@:*"] },
+  { "action": "accept", "src": ["admin1@"], "dst": ["admin1@:*"] },
+  { "action": "accept", "src": ["intern1@"], "dst": ["intern1@:*"] }
+}
+```
+
+### `autogroup:nonroot`
+
+Used in Tailscale SSH rules to allow access to any user except root. Can only be used in the `users` field of SSH rules.
+
+```json
+{
+  "action": "accept",
+  "src": ["autogroup:member"],
+  "dst": ["autogroup:self"],
+  "users": ["autogroup:nonroot"]
+}
+```

docs/ref/api.md

@@ -0,0 +1,128 @@
+# API
+Headscale provides a [HTTP REST API](#rest-api) and a [gRPC interface](#grpc) which may be used to integrate a [web
+interface](integration/web-ui.md), [remote control Headscale](#setup-remote-control) or provide a base for custom
+integration and tooling.
+
+Both interfaces require a valid API key before use. To create an API key, log into your Headscale server and generate
+one with the default expiration of 90 days:
+
+```shell
+headscale apikeys create
+```
+
+Copy the output of the command and save it for later. Please note that you can not retrieve an API key again. If the API
+key is lost, expire the old one, and create a new one.
+
+To list the API keys currently associated with the server:
+
+```shell
+headscale apikeys list
+```
+
+and to expire an API key:
+
+```shell
+headscale apikeys expire --prefix <PREFIX>
+```
+
+## REST API
+
+- API endpoint: `/api/v1`, e.g. `https://headscale.example.com/api/v1`
+- Documentation: `/swagger`, e.g. `https://headscale.example.com/swagger`
+- Authenticate using HTTP Bearer authentication by sending the [API key](#api) with the HTTP `Authorization: Bearer
+  <API_KEY>` header.
+
+Start by [creating an API key](#api) and test it with the examples below. Read the API documentation provided by your
+Headscale server at `/swagger` for details.
+
+=== "Get details for all users"
+
+    ```console
+    curl -H "Authorization: Bearer <API_KEY>" \
+        https://headscale.example.com/api/v1/user
+    ```
+
+=== "Get details for user 'bob'"
+
+    ```console
+    curl -H "Authorization: Bearer <API_KEY>" \
+        https://headscale.example.com/api/v1/user?name=bob
+    ```
+
+=== "Register a node"
+
+    ```console
+    curl -H "Authorization: Bearer <API_KEY>" \
+        -d user=<USER> -d key=<KEY> \
+        https://headscale.example.com/api/v1/node/register
+    ```
+
+## gRPC
+
+The gRPC interface can be used to control a Headscale instance from a remote machine with the `headscale` binary.
+
+### Prerequisite
+
+- A workstation to run `headscale` (any supported platform, e.g. Linux).
+- A Headscale server with gRPC enabled.
+- Connections to the gRPC port (default: `50443`) are allowed.
+- Remote access requires an encrypted connection via TLS.
+- An [API key](#api) to authenticate with the Headscale server.
+
+### Setup remote control
+
+1.  Download the [`headscale` binary from GitHub's release page](https://github.com/juanfont/headscale/releases). Make
+    sure to use the same version as on the server.
+
+1.  Put the binary somewhere in your `PATH`, e.g. `/usr/local/bin/headscale`
+
+1.  Make `headscale` executable: `chmod +x /usr/local/bin/headscale`
+
+1.  [Create an API key](#api) on the Headscale server.
+
+1.  Provide the connection parameters for the remote Headscale server either via a minimal YAML configuration file or
+    via environment variables:
+
+    === "Minimal YAML configuration file"
+
+        ```yaml title="config.yaml"
+        cli:
+            address: <HEADSCALE_ADDRESS>:<PORT>
+            api_key: <API_KEY>
+        ```
+
+    === "Environment variables"
+
+        ```shell
+        export HEADSCALE_CLI_ADDRESS="<HEADSCALE_ADDRESS>:<PORT>"
+        export HEADSCALE_CLI_API_KEY="<API_KEY>"
+        ```
+
+    This instructs the `headscale` binary to connect to a remote instance at `<HEADSCALE_ADDRESS>:<PORT>`, instead of
+    connecting to the local instance.
+
+1.  Test the connection by listing all nodes:
+
+    ```shell
+    headscale nodes list
+    ```
+
+    You should now be able to see a list of your nodes from your workstation, and you can
+    now control the Headscale server from your workstation.
+
+### Behind a proxy
+
+It's possible to run the gRPC remote endpoint behind a reverse proxy, like Nginx, and have it run on the _same_ port as Headscale.
+
+While this is _not a supported_ feature, an example on how this can be set up on
+[NixOS is shown here](https://github.com/kradalby/dotfiles/blob/4489cdbb19cddfbfae82cd70448a38fde5a76711/machines/headscale.oracldn/headscale.nix#L61-L91).
+
+### Troubleshooting
+
+- Make sure you have the _same_ Headscale version on your server and workstation.
+- Ensure that connections to the gRPC port are allowed.
+- Verify that your TLS certificate is valid and trusted.
+- If you don't have access to a trusted certificate (e.g. from Let's Encrypt), either:
+    - Add your self-signed certificate to the trust store of your OS _or_
+    - Disable certificate verification by either setting `cli.insecure: true` in the configuration file or by setting
+      `HEADSCALE_CLI_INSECURE=1` via an environment variable. We do **not** recommend to disable certificate validation.

docs/ref/configuration.md

@@ -0,0 +1,41 @@
+# Configuration
+
+- Headscale loads its configuration from a YAML file
+- It searches for `config.yaml` in the following paths:
+    - `/etc/headscale`
+    - `$HOME/.headscale`
+    - the current working directory
+- To load the configuration from a different path, use:
+    - the command line flag `-c`, `--config`
+    - the environment variable `HEADSCALE_CONFIG`
+- Validate the configuration file with: `headscale configtest`
+
+!!! example "Get the [example configuration from the GitHub repository](https://github.com/juanfont/headscale/blob/main/config-example.yaml)"
+
+    Always select the [same GitHub tag](https://github.com/juanfont/headscale/tags) as the released version you use to
+    ensure you have the correct example configuration. The `main` branch might contain unreleased changes.
+
+    === "View on GitHub"
+
+        * Development version: <https://github.com/juanfont/headscale/blob/main/config-example.yaml>
+        * Version {{ headscale.version }}: <https://github.com/juanfont/headscale/blob/v{{ headscale.version }}/config-example.yaml>
+
+    === "Download with `wget`"
+
+        ```shell
+        # Development version
+        wget -O config.yaml https://raw.githubusercontent.com/juanfont/headscale/main/config-example.yaml
+
+        # Version {{ headscale.version }}
+        wget -O config.yaml https://raw.githubusercontent.com/juanfont/headscale/v{{ headscale.version }}/config-example.yaml
+        ```
+
+    === "Download with `curl`"
+
+        ```shell
+        # Development version
+        curl -o config.yaml https://raw.githubusercontent.com/juanfont/headscale/main/config-example.yaml
+
+        # Version {{ headscale.version }}
+        curl -o config.yaml https://raw.githubusercontent.com/juanfont/headscale/v{{ headscale.version }}/config-example.yaml
+        ```