Merge pull request #687 from huskyii/node_ls

more intuitive output of node ls
update CHANGELOG
2025-08-21 04:17:29 +00:00 · 2022-07-24 12:06:41 +02:00 · 2022-07-23 04:47:37 +08:00 · 2022-07-23 01:33:11 +08:00 · 2022-07-22 18:56:49 +02:00 · 2022-07-22 07:36:16 +00:00
20 changed files with 263 additions and 148 deletions
--- a/.github/workflows/test-integration.yml
+++ b/.github/workflows/test-integration.yml
@@ -27,4 +27,9 @@ jobs:

      - name: Run Integration tests
        if: steps.changed-files.outputs.any_changed == 'true'
-        run: nix develop --command -- make test_integration
+        uses: nick-fields/retry@v2
+        with:
+          timeout_minutes: 240
+          max_attempts: 5
+          retry_on: error
+          command: nix develop --command -- make test_integration
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -30,11 +30,12 @@
 - Add -c option to specify config file from command line [#285](https://github.com/juanfont/headscale/issues/285) [#612](https://github.com/juanfont/headscale/pull/601)
 - Add configuration option to allow Tailscale clients to use a random WireGuard port. [kb/1181/firewalls](https://tailscale.com/kb/1181/firewalls) [#624](https://github.com/juanfont/headscale/pull/624)
 - Improve obtuse UX regarding missing configuration (`ephemeral_node_inactivity_timeout` not set) [#639](https://github.com/juanfont/headscale/pull/639)
- Fix nodes being shown as 'offline' in `tailscale status` [648](https://github.com/juanfont/headscale/pull/648)
 - Fix nodes being shown as 'offline' in `tailscale status` [#648](https://github.com/juanfont/headscale/pull/648)
 - Improve shutdown behaviour [#651](https://github.com/juanfont/headscale/pull/651)
- Drop Gin as web framework in Headscale [648](https://github.com/juanfont/headscale/pull/648)
-
+- Drop Gin as web framework in Headscale [648](https://github.com/juanfont/headscale/pull/648) [677](https://github.com/juanfont/headscale/pull/677)
+- Make tailnet node updates check interval configurable [#675](https://github.com/juanfont/headscale/pull/675)
+- Fix regression with HTTP API [#684](https://github.com/juanfont/headscale/pull/684)
+- nodes ls now print both Hostname and Name(Issue [#647](https://github.com/juanfont/headscale/issues/647) PR [#687](https://github.com/juanfont/headscale/pull/687))

 ## 0.15.0 (2022-03-20)

--- a/README.md
+++ b/README.md
@@ -188,13 +188,6 @@ make build
            <sub style="font-size:14px"><b>Ward Vandewege</b></sub>
        </a>
    </td>
-    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
-        <a href=https://github.com/reynico>
-            <img src=https://avatars.githubusercontent.com/u/715768?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Nico/>
-            <br />
-            <sub style="font-size:14px"><b>Nico</b></sub>
-        </a>
-    </td>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/huskyii>
            <img src=https://avatars.githubusercontent.com/u/5499746?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Jiang Zhu/>
@@ -202,6 +195,13 @@ make build
            <sub style="font-size:14px"><b>Jiang Zhu</b></sub>
        </a>
    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/reynico>
+            <img src=https://avatars.githubusercontent.com/u/715768?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Nico/>
+            <br />
+            <sub style="font-size:14px"><b>Nico</b></sub>
+        </a>
+    </td>
 </tr>
 <tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -283,6 +283,15 @@ make build
            <sub style="font-size:14px"><b>Fernando De Lucchi</b></sub>
        </a>
    </td>
+    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
+        <a href=https://github.com/GrigoriyMikhalkin>
+            <img src=https://avatars.githubusercontent.com/u/3637857?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=GrigoriyMikhalkin/>
+            <br />
+            <sub style="font-size:14px"><b>GrigoriyMikhalkin</b></sub>
+        </a>
+    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/hdhoang>
            <img src=https://avatars.githubusercontent.com/u/12537?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Hoàng Đức Hiếu/>
@@ -290,8 +299,6 @@ make build
            <sub style="font-size:14px"><b>Hoàng Đức Hiếu</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/bravechamp>
            <img src=https://avatars.githubusercontent.com/u/48980452?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=bravechamp/>
@@ -327,6 +334,8 @@ make build
            <sub style="font-size:14px"><b>Michael G.</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/ptman>
            <img src=https://avatars.githubusercontent.com/u/24669?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Paul Tötterman/>
@@ -334,8 +343,6 @@ make build
            <sub style="font-size:14px"><b>Paul Tötterman</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/samson4649>
            <img src=https://avatars.githubusercontent.com/u/12725953?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Samuel Lock/>
@@ -371,6 +378,8 @@ make build
            <sub style="font-size:14px"><b>Pavlos Vinieratos</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/SilverBut>
            <img src=https://avatars.githubusercontent.com/u/6560655?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Silver Bullet/>
@@ -378,8 +387,6 @@ make build
            <sub style="font-size:14px"><b>Silver Bullet</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/lachy2849>
            <img src=https://avatars.githubusercontent.com/u/98844035?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=lachy2849/>
@@ -415,6 +422,8 @@ make build
            <sub style="font-size:14px"><b>Aofei Sheng</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/awoimbee>
            <img src=https://avatars.githubusercontent.com/u/22431493?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Arthur Woimbée/>
@@ -422,8 +431,6 @@ make build
            <sub style="font-size:14px"><b>Arthur Woimbée</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/stensonb>
            <img src=https://avatars.githubusercontent.com/u/933389?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Bryan Stenson/>
@@ -459,6 +466,8 @@ make build
            <sub style="font-size:14px"><b>Felix Yan</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/JJGadgets>
            <img src=https://avatars.githubusercontent.com/u/5709019?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=JJGadgets/>
@@ -466,8 +475,6 @@ make build
            <sub style="font-size:14px"><b>JJGadgets</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/madjam002>
            <img src=https://avatars.githubusercontent.com/u/679137?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Jamie Greeff/>
@@ -503,6 +510,8 @@ make build
            <sub style="font-size:14px"><b>WhiteSource Renovate</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/ryanfowler>
            <img src=https://avatars.githubusercontent.com/u/2668821?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Ryan Fowler/>
@@ -510,8 +519,6 @@ make build
            <sub style="font-size:14px"><b>Ryan Fowler</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/shaananc>
            <img src=https://avatars.githubusercontent.com/u/2287839?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Shaanan Cohney/>
@@ -547,6 +554,8 @@ make build
            <sub style="font-size:14px"><b>Tianon Gravi</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/woudsma>
            <img src=https://avatars.githubusercontent.com/u/6162978?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Tjerk Woudsma/>
@@ -554,8 +563,6 @@ make build
            <sub style="font-size:14px"><b>Tjerk Woudsma</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/y0ngb1n>
            <img src=https://avatars.githubusercontent.com/u/25719408?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Yang Bin/>
@@ -572,9 +579,9 @@ make build
    </td>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/Bpazy>
-            <img src=https://avatars.githubusercontent.com/u/9838749?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=ZiYuan/>
+            <img src=https://avatars.githubusercontent.com/u/9838749?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=Ziyuan Han/>
            <br />
-            <sub style="font-size:14px"><b>ZiYuan</b></sub>
+            <sub style="font-size:14px"><b>Ziyuan Han</b></sub>
        </a>
    </td>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
@@ -591,6 +598,8 @@ make build
            <sub style="font-size:14px"><b>henning mueller</b></sub>
        </a>
    </td>
+</tr>
+<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/ignoramous>
            <img src=https://avatars.githubusercontent.com/u/852289?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=ignoramous/>
@@ -598,8 +607,6 @@ make build
            <sub style="font-size:14px"><b>ignoramous</b></sub>
        </a>
    </td>
-</tr>
-<tr>
    <td align="center" style="word-wrap: break-word; width: 150.0; height: 150.0">
        <a href=https://github.com/lion24>
            <img src=https://avatars.githubusercontent.com/u/1382102?v=4 width="100;"  style="border-radius:50%;align-items:center;justify-content:center;overflow:hidden;padding-top:10px" alt=lion24/>
--- a/app.go
+++ b/app.go
@@ -17,17 +17,16 @@ import (
 	"time"

 	"github.com/coreos/go-oidc/v3/oidc"
-	"github.com/gin-gonic/gin"
 	"github.com/gorilla/mux"
 	grpc_middleware "github.com/grpc-ecosystem/go-grpc-middleware"
 	"github.com/grpc-ecosystem/grpc-gateway/v2/runtime"
 	v1 "github.com/juanfont/headscale/gen/go/headscale/v1"
 	"github.com/patrickmn/go-cache"
 	zerolog "github.com/philip-bui/grpc-zerolog"
+	"github.com/prometheus/client_golang/prometheus/promhttp"
 	"github.com/puzpuzpuz/xsync"
 	zl "github.com/rs/zerolog"
 	"github.com/rs/zerolog/log"
-	ginprometheus "github.com/zsais/go-gin-prometheus"
 	"golang.org/x/crypto/acme"
 	"golang.org/x/crypto/acme/autocert"
 	"golang.org/x/oauth2"
@@ -95,7 +94,8 @@ type Headscale struct {

 	ipAllocationMutex sync.Mutex

-	shutdownChan chan struct{}
+	shutdownChan       chan struct{}
+	pollNetMapStreamWG sync.WaitGroup
 }

 // Look up the TLS constant relative to user-supplied TLS client
@@ -148,12 +148,13 @@ func NewHeadscale(cfg *Config) (*Headscale, error) {
 	)

 	app := Headscale{
-		cfg:               cfg,
-		dbType:            cfg.DBtype,
-		dbString:          dbString,
-		privateKey:        privKey,
-		aclRules:          tailcfg.FilterAllowAll, // default allowall
-		registrationCache: registrationCache,
+		cfg:                cfg,
+		dbType:             cfg.DBtype,
+		dbString:           dbString,
+		privateKey:         privKey,
+		aclRules:           tailcfg.FilterAllowAll, // default allowall
+		registrationCache:  registrationCache,
+		pollNetMapStreamWG: sync.WaitGroup{},
 	}

 	err = app.initDB()
@@ -411,15 +412,6 @@ func (h *Headscale) ensureUnixSocketIsAbsent() error {
 	return os.Remove(h.cfg.UnixSocket)
 }

-func (h *Headscale) createPrometheusRouter() *gin.Engine {
-	promRouter := gin.Default()
-
-	prometheus := ginprometheus.NewPrometheus("gin")
-	prometheus.Use(promRouter)
-
-	return promRouter
-}
-
 func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *mux.Router {
 	router := mux.NewRouter()

@@ -455,11 +447,9 @@ func (h *Headscale) createRouter(grpcMux *runtime.ServeMux) *mux.Router {
 		router.HandleFunc("/bootstrap-dns", h.DERPBootstrapDNSHandler)
 	}

-	api := router.PathPrefix("/api").Subrouter()
-	api.Use(h.httpAuthenticationMiddleware)
-	{
-		api.HandleFunc("/v1/*any", grpcMux.ServeHTTP)
-	}
+	apiRouter := router.PathPrefix("/api").Subrouter()
+	apiRouter.Use(h.httpAuthenticationMiddleware)
+	apiRouter.PathPrefix("/v1/").HandlerFunc(grpcMux.ServeHTTP)

 	router.PathPrefix("/").HandlerFunc(stdoutHandler)

@@ -577,6 +567,8 @@ func (h *Headscale) Serve() error {
 	// https://github.com/soheilhy/cmux/issues/68
 	// https://github.com/soheilhy/cmux/issues/91

+	var grpcServer *grpc.Server
+	var grpcListener net.Listener
 	if tlsConfig != nil || h.cfg.GRPCAllowInsecure {
 		log.Info().Msgf("Enabling remote gRPC at %s", h.cfg.GRPCAddr)

@@ -597,12 +589,12 @@ func (h *Headscale) Serve() error {
 			log.Warn().Msg("gRPC is running without security")
 		}

-		grpcServer := grpc.NewServer(grpcOptions...)
+		grpcServer = grpc.NewServer(grpcOptions...)

 		v1.RegisterHeadscaleServiceServer(grpcServer, newHeadscaleV1APIServer(h))
 		reflection.Register(grpcServer)

-		grpcListener, err := net.Listen("tcp", h.cfg.GRPCAddr)
+		grpcListener, err = net.Listen("tcp", h.cfg.GRPCAddr)
 		if err != nil {
 			return fmt.Errorf("failed to bind to TCP address: %w", err)
 		}
@@ -647,11 +639,12 @@ func (h *Headscale) Serve() error {
 	log.Info().
 		Msgf("listening and serving HTTP on: %s", h.cfg.Addr)

-	promRouter := h.createPrometheusRouter()
+	promMux := http.NewServeMux()
+	promMux.Handle("/metrics", promhttp.Handler())

 	promHTTPServer := &http.Server{
 		Addr:         h.cfg.MetricsAddr,
-		Handler:      promRouter,
+		Handler:      promMux,
 		ReadTimeout:  HTTPReadTimeout,
 		WriteTimeout: 0,
 	}
@@ -677,7 +670,7 @@ func (h *Headscale) Serve() error {
 		syscall.SIGTERM,
 		syscall.SIGQUIT,
 		syscall.SIGHUP)
-	go func(c chan os.Signal) {
+	sigFunc := func(c chan os.Signal) {
 		// Wait for a SIGINT or SIGKILL:
 		for {
 			sig := <-c
@@ -687,7 +680,7 @@ func (h *Headscale) Serve() error {
 					Str("signal", sig.String()).
 					Msg("Received SIGHUP, reloading ACL and Config")

-					// TODO(kradalby): Reload config on SIGHUP
+				// TODO(kradalby): Reload config on SIGHUP

 				if h.cfg.ACL.PolicyPath != "" {
 					aclPath := AbsolutePathFromConfigPath(h.cfg.ACL.PolicyPath)
@@ -707,7 +700,8 @@ func (h *Headscale) Serve() error {
 					Str("signal", sig.String()).
 					Msg("Received signal to stop, shutting down gracefully")

-				h.shutdownChan <- struct{}{}
+				close(h.shutdownChan)
+				h.pollNetMapStreamWG.Wait()

 				// Gracefully shut down servers
 				ctx, cancel := context.WithTimeout(context.Background(), HTTPShutdownTimeout)
@@ -719,6 +713,11 @@ func (h *Headscale) Serve() error {
 				}
 				grpcSocket.GracefulStop()

+				if grpcServer != nil {
+					grpcServer.GracefulStop()
+					grpcListener.Close()
+				}
+
 				// Close network listeners
 				promHTTPListener.Close()
 				httpListener.Close()
@@ -745,7 +744,12 @@ func (h *Headscale) Serve() error {
 				os.Exit(0)
 			}
 		}
-	}(sigc)
+	}
+	errorGroup.Go(func() error {
+		sigFunc(sigc)
+
+		return nil
+	})

 	return errorGroup.Wait()
 }
@@ -769,13 +773,13 @@ func (h *Headscale) getTLSSettings() (*tls.Config, error) {
 		}

 		switch h.cfg.TLS.LetsEncrypt.ChallengeType {
-		case "TLS-ALPN-01":
+		case tlsALPN01ChallengeType:
 			// Configuration via autocert with TLS-ALPN-01 (https://tools.ietf.org/html/rfc8737)
 			// The RFC requires that the validation is done on port 443; in other words, headscale
 			// must be reachable on port 443.
 			return certManager.TLSConfig(), nil

-		case "HTTP-01":
+		case http01ChallengeType:
 			// Configuration via autocert with HTTP-01. This requires listening on
 			// port 80 for the certificate validation in addition to the headscale
 			// service, which can be configured to run on any other port.
--- a/cmd/headscale/cli/nodes.go
+++ b/cmd/headscale/cli/nodes.go
@@ -465,6 +465,7 @@ func nodesToPtables(
 ) (pterm.TableData, error) {
 	tableHeader := []string{
 		"ID",
+		"Hostname",
 		"Name",
 		"NodeKey",
 		"Namespace",
@@ -566,6 +567,7 @@ func nodesToPtables(
 		nodeData := []string{
 			strconv.FormatUint(machine.Id, headscale.Base10),
 			machine.Name,
+			machine.GetGivenName(),
 			nodeKey.ShortString(),
 			namespace,
 			strings.Join([]string{IPV4Address, IPV6Address}, ", "),
--- a/config-example.yaml
+++ b/config-example.yaml
@@ -103,6 +103,12 @@ disable_check_updates: false
 # Time before an inactive ephemeral node is deleted?
 ephemeral_node_inactivity_timeout: 30m

+# Period to check for node updates in the tailnet. A value too low will severily affect
+# CPU consumption of Headscale. A value too high (over 60s) will cause problems
+# to the nodes, as they won't get updates or keep alive messages in time.
+# In case of doubts, do not touch the default 10s.
+node_update_check_interval: 10s
+
 # SQLite config
 db_type: sqlite3
 db_path: /var/lib/headscale/db.sqlite
--- a/config.go
+++ b/config.go
@@ -18,6 +18,11 @@ import (
 	"tailscale.com/types/dnstype"
 )

+const (
+	tlsALPN01ChallengeType = "TLS-ALPN-01"
+	http01ChallengeType    = "HTTP-01"
+)
+
 // Config contains the initial Headscale configuration.
 type Config struct {
 	ServerURL                      string
@@ -26,6 +31,7 @@ type Config struct {
 	GRPCAddr                       string
 	GRPCAllowInsecure              bool
 	EphemeralNodeInactivityTimeout time.Duration
+	NodeUpdateCheckInterval        time.Duration
 	IPPrefixes                     []netaddr.IPPrefix
 	PrivateKeyPath                 string
 	BaseDomain                     string
@@ -135,7 +141,7 @@ func LoadConfig(path string, isFile bool) error {
 	viper.AutomaticEnv()

 	viper.SetDefault("tls_letsencrypt_cache_dir", "/var/www/.cache")
-	viper.SetDefault("tls_letsencrypt_challenge_type", "HTTP-01")
+	viper.SetDefault("tls_letsencrypt_challenge_type", http01ChallengeType)
 	viper.SetDefault("tls_client_auth_mode", "relaxed")

 	viper.SetDefault("log_level", "info")
@@ -162,6 +168,8 @@ func LoadConfig(path string, isFile bool) error {

 	viper.SetDefault("ephemeral_node_inactivity_timeout", "120s")

+	viper.SetDefault("node_update_check_interval", "10s")
+
 	if err := viper.ReadInConfig(); err != nil {
 		log.Warn().Err(err).Msg("Failed to read configuration from disk")

@@ -176,15 +184,15 @@ func LoadConfig(path string, isFile bool) error {
 	}

 	if (viper.GetString("tls_letsencrypt_hostname") != "") &&
-		(viper.GetString("tls_letsencrypt_challenge_type") == "TLS-ALPN-01") &&
+		(viper.GetString("tls_letsencrypt_challenge_type") == tlsALPN01ChallengeType) &&
 		(!strings.HasSuffix(viper.GetString("listen_addr"), ":443")) {
 		// this is only a warning because there could be something sitting in front of headscale that redirects the traffic (e.g. an iptables rule)
 		log.Warn().
 			Msg("Warning: when using tls_letsencrypt_hostname with TLS-ALPN-01 as challenge type, headscale must be reachable on port 443, i.e. listen_addr should probably end in :443")
 	}

-	if (viper.GetString("tls_letsencrypt_challenge_type") != "HTTP-01") &&
-		(viper.GetString("tls_letsencrypt_challenge_type") != "TLS-ALPN-01") {
+	if (viper.GetString("tls_letsencrypt_challenge_type") != http01ChallengeType) &&
+		(viper.GetString("tls_letsencrypt_challenge_type") != tlsALPN01ChallengeType) {
 		errorText += "Fatal config error: the only supported values for tls_letsencrypt_challenge_type are HTTP-01 and TLS-ALPN-01\n"
 	}

@@ -217,6 +225,15 @@ func LoadConfig(path string, isFile bool) error {
 		)
 	}

+	maxNodeUpdateCheckInterval, _ := time.ParseDuration("60s")
+	if viper.GetDuration("node_update_check_interval") > maxNodeUpdateCheckInterval {
+		errorText += fmt.Sprintf(
+			"Fatal config error: node_update_check_interval (%s) is set too high, must be less than %s",
+			viper.GetString("node_update_check_interval"),
+			maxNodeUpdateCheckInterval,
+		)
+	}
+
 	if errorText != "" {
 		//nolint
 		return errors.New(strings.TrimSuffix(errorText, "\n"))
@@ -478,6 +495,10 @@ func GetHeadscaleConfig() (*Config, error) {
 			"ephemeral_node_inactivity_timeout",
 		),

+		NodeUpdateCheckInterval: viper.GetDuration(
+			"node_update_check_interval",
+		),
+
 		DBtype: viper.GetString("db_type"),
 		DBpath: AbsolutePathFromConfigPath(viper.GetString("db_path")),
 		DBhost: viper.GetString("db_host"),
--- a/flake.nix
+++ b/flake.nix
@@ -24,7 +24,7 @@

              # When updating go.mod or go.sum, a new sha will need to be calculated,
              # update this if you have a mismatch after doing a change to thos files.
-              vendorSha256 = "sha256-T6rH+aqofFmCPxDfoA5xd3kNUJeZkT4GRyuFEnenps8=";
+              vendorSha256 = "sha256-b9C6F+7N0ecW0HiTx+rztZnxb+n6U6YTSOJvp3GqnWQ=";

              ldflags = [ "-s" "-w" "-X github.com/juanfont/headscale/cmd/headscale/cli.Version=v${version}" ];
            };
--- a/go.mod
+++ b/go.mod
@@ -8,7 +8,6 @@ require (
 	github.com/coreos/go-oidc/v3 v3.1.0
 	github.com/deckarep/golang-set/v2 v2.1.0
 	github.com/efekarakus/termcolor v1.0.1
-	github.com/gin-gonic/gin v1.7.7
 	github.com/glebarez/sqlite v1.4.3
 	github.com/gofrs/uuid v4.2.0+incompatible
 	github.com/gorilla/mux v1.8.0
@@ -28,7 +27,6 @@ require (
 	github.com/stretchr/testify v1.7.1
 	github.com/tailscale/hujson v0.0.0-20220506202205-92b4b88a9e17
 	github.com/tcnksm/go-latest v0.0.0-20170313132115-e3007ae9052e
-	github.com/zsais/go-gin-prometheus v0.1.0
 	golang.org/x/crypto v0.0.0-20220427172511-eb4f295cb31f
 	golang.org/x/oauth2 v0.0.0-20220411215720-9780585627b5
 	golang.org/x/sync v0.0.0-20220513210516-0976fa681c29
@@ -61,11 +59,7 @@ require (
 	github.com/docker/go-connections v0.4.0 // indirect
 	github.com/docker/go-units v0.4.0 // indirect
 	github.com/fsnotify/fsnotify v1.5.1 // indirect
-	github.com/gin-contrib/sse v0.1.0 // indirect
 	github.com/glebarez/go-sqlite v1.16.0 // indirect
-	github.com/go-playground/locales v0.13.0 // indirect
-	github.com/go-playground/universal-translator v0.17.0 // indirect
-	github.com/go-playground/validator/v10 v10.4.1 // indirect
 	github.com/gogo/protobuf v1.3.2 // indirect
 	github.com/golang/protobuf v1.5.2 // indirect
 	github.com/google/go-cmp v0.5.8 // indirect
@@ -90,11 +84,9 @@ require (
 	github.com/jinzhu/now v1.1.4 // indirect
 	github.com/josharian/native v1.0.0 // indirect
 	github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b // indirect
-	github.com/json-iterator/go v1.1.12 // indirect
 	github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
 	github.com/kr/pretty v0.3.0 // indirect
 	github.com/kr/text v0.2.0 // indirect
-	github.com/leodido/go-urn v1.2.0 // indirect
 	github.com/magiconair/properties v1.8.6 // indirect
 	github.com/mattn/go-colorable v0.1.12 // indirect
 	github.com/mattn/go-isatty v0.0.14 // indirect
@@ -106,8 +98,6 @@ require (
 	github.com/mitchellh/go-ps v1.0.0 // indirect
 	github.com/mitchellh/mapstructure v1.4.3 // indirect
 	github.com/moby/term v0.0.0-20201216013528-df9cb8a40635 // indirect
-	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
-	github.com/modern-go/reflect2 v1.0.2 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/opencontainers/image-spec v1.0.3-0.20220114050600-8b9d41f48198 // indirect
 	github.com/opencontainers/runc v1.0.2 // indirect
@@ -126,7 +116,6 @@ require (
 	github.com/spf13/jwalterweatherman v1.1.0 // indirect
 	github.com/spf13/pflag v1.0.5 // indirect
 	github.com/subosito/gotenv v1.2.0 // indirect
-	github.com/ugorji/go/codec v1.1.7 // indirect
 	github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f // indirect
 	github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 // indirect
 	github.com/xeipuuv/gojsonschema v1.2.0 // indirect
--- a/go.sum
+++ b/go.sum
@@ -236,10 +236,6 @@ github.com/fsnotify/fsnotify v1.5.1/go.mod h1:T3375wBYaZdLLcVNkcVbzGHY7f1l/uK5T5
 github.com/fullstorydev/grpcurl v1.6.0/go.mod h1:ZQ+ayqbKMJNhzLmbpCiurTVlaK2M/3nqZCxaQ2Ze/sM=
 github.com/fzipp/gocyclo v0.3.1/go.mod h1:DJHO6AUmbdqj2ET4Z9iArSuwWgYDRryYt2wASxc7x3E=
 github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
-github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
-github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
-github.com/gin-gonic/gin v1.7.7 h1:3DoBmSbJbZAWqXJC3SLjAPfutPJJRN1U5pALB7EeTTs=
-github.com/gin-gonic/gin v1.7.7/go.mod h1:axIBovoeJpVj8S3BwE0uPMTeReE4+AfFtqpqaZ1qq1U=
 github.com/glebarez/go-sqlite v1.16.0 h1:h28rHued+hGof3fNLksBcLwz/a71fiGZ/eIJHK0SsLI=
 github.com/glebarez/go-sqlite v1.16.0/go.mod h1:i8/JtqoqzBAFkrUTxbQFkQ05odCOds3j7NlDaXjqiPY=
 github.com/glebarez/sqlite v1.4.3 h1:ZABNo+2YIau8F8sZ7Qh/1h/ZnlSUMHFGD4zJKPval7A=
@@ -256,14 +252,6 @@ github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V
 github.com/go-logfmt/logfmt v0.5.0/go.mod h1:wCYkCAKZfumFQihp8CzCvQ3paCTfi41vtzG1KdI/P7A=
 github.com/go-ole/go-ole v1.2.5/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
 github.com/go-ole/go-ole v1.2.6/go.mod h1:pprOEPIfldk/42T2oK7lQ4v4JSDwmV0As9GaiUsvbm0=
-github.com/go-playground/assert/v2 v2.0.1 h1:MsBgLAaY856+nPRTKrp3/OZK38U/wa0CcBYNjji3q3A=
-github.com/go-playground/assert/v2 v2.0.1/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
-github.com/go-playground/locales v0.13.0 h1:HyWk6mgj5qFqCT5fjGBuRArbVDfE4hi8+e8ceBS/t7Q=
-github.com/go-playground/locales v0.13.0/go.mod h1:taPMhCMXrRLJO55olJkUXHZBHCxTMfnGwq/HNwmWNS8=
-github.com/go-playground/universal-translator v0.17.0 h1:icxd5fm+REJzpZx7ZfpaD876Lmtgy7VtROAbHHXk8no=
-github.com/go-playground/universal-translator v0.17.0/go.mod h1:UkSxE5sNxxRwHyU+Scu5vgOQjsIJAF8j9muTVoKLVtA=
-github.com/go-playground/validator/v10 v10.4.1 h1:pH2c5ADXtd66mxoE0Zm9SUhxE20r7aM3F26W0hOn+GE=
-github.com/go-playground/validator/v10 v10.4.1/go.mod h1:nlOn6nFhuKACm19sB/8EGNn9GlaMV7XkbRSipzJ0Ii4=
 github.com/go-redis/redis v6.15.8+incompatible/go.mod h1:NAIEuMOZ/fxfXJIrKDQDz8wamY7mA7PouImQ2Jvg6kA=
 github.com/go-sql-driver/mysql v1.4.0/go.mod h1:zAC/RDZ24gD3HViQzih4MyKcchzm+sOG5ZlKdlhCg5w=
 github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg=
@@ -546,10 +534,8 @@ github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b h1:Yws7RV6k
 github.com/jsimonetti/rtnetlink v1.1.2-0.20220408201609-d380b505068b/go.mod h1:TzDCVOZKUa79z6iXbbXqhtAflVgUKaFkZ21M5tK5tzY=
 github.com/json-iterator/go v1.1.6/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
 github.com/json-iterator/go v1.1.7/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
-github.com/json-iterator/go v1.1.9/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/json-iterator/go v1.1.10/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
 github.com/json-iterator/go v1.1.11/go.mod h1:KdQUCv79m/52Kvf8AW2vK1V8akMuk1QjK/uOdHXbAo4=
-github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
 github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
 github.com/jstemmer/go-junit-report v0.0.0-20190106144839-af01ea7f8024/go.mod h1:6v2b51hI/fHJwM22ozAgKL4VKDeJcHhJFhtBdhmNjmU=
 github.com/jstemmer/go-junit-report v0.9.1/go.mod h1:Brl9GWCQeLvo8nXZwPNNblvFj/XSXhF0NWZEnDohbsk=
@@ -595,8 +581,6 @@ github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+
 github.com/kyoh86/exportloopref v0.1.8/go.mod h1:1tUcJeiioIs7VWe5gcOObrux3lb66+sBqGZrRkMwPgg=
 github.com/ldez/gomoddirectives v0.2.2/go.mod h1:cpgBogWITnCfRq2qGoDkKMEVSaarhdBr6g8G04uz6d0=
 github.com/ldez/tagliatelle v0.2.0/go.mod h1:8s6WJQwEYHbKZDsp/LjArytKOG8qaMrKQQ3mFukHs88=
-github.com/leodido/go-urn v1.2.0 h1:hpXL4XnriNwQ/ABnpepYM/1vCLWNDfUNts8dX3xTG6Y=
-github.com/leodido/go-urn v1.2.0/go.mod h1:+8+nEpDfqqsY+g338gtMEUOtuK+4dEMhiQEgxpxOKII=
 github.com/letsencrypt/pkcs11key/v4 v4.0.0/go.mod h1:EFUvBDay26dErnNb70Nd0/VW3tJiIbETBPTl9ATXQag=
 github.com/lib/pq v0.0.0-20180327071824-d34b9ff171c2/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
 github.com/lib/pq v1.0.0/go.mod h1:5WUZQaWbwv1U+lTReE5YruASi9Al49XbQIvNi/34Woo=
@@ -686,11 +670,9 @@ github.com/moby/sys/mountinfo v0.4.1/go.mod h1:rEr8tzG/lsIZHBtN/JjGG+LMYx9eXgW2J
 github.com/moby/term v0.0.0-20201216013528-df9cb8a40635 h1:rzf0wL0CHVc8CEsgyygG0Mn9CNCCPZqOPaz8RiiHYQk=
 github.com/moby/term v0.0.0-20201216013528-df9cb8a40635/go.mod h1:FBS0z0QWA44HXygs7VXDUOGoN/1TV3RuWkLO04am3wc=
 github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
-github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
 github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
 github.com/modern-go/reflect2 v0.0.0-20180701023420-4b7aa43c6742/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
 github.com/modern-go/reflect2 v1.0.1/go.mod h1:bx2lNnkwVCuqBIxFjflWJWanXIb3RllmbCylyMrvgv0=
-github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
 github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
 github.com/mohae/deepcopy v0.0.0-20170929034955-c48cc78d4826/go.mod h1:TaXosZuwdSHYgviHp1DAtfrULt5eUgsSMsZf+YrPgl8=
 github.com/moricho/tparallel v0.2.1/go.mod h1:fXEIZxG2vdfl0ZF8b42f5a78EhjjD5mX8qUplsoSU4k=
@@ -925,10 +907,7 @@ github.com/tmc/grpc-websocket-proxy v0.0.0-20200427203606-3cfed13b9966/go.mod h1
 github.com/tomarrell/wrapcheck/v2 v2.4.0/go.mod h1:68bQ/eJg55BROaRTbMjC7vuhL2OgfoG8bLp9ZyoBfyY=
 github.com/tomasen/realip v0.0.0-20180522021738-f0c99a92ddce/go.mod h1:o8v6yHRoik09Xen7gje4m9ERNah1d1PPsVq1VEx9vE4=
 github.com/tommy-muehle/go-mnd/v2 v2.4.0/go.mod h1:WsUAkMJMYww6l/ufffCD3m+P7LEvr8TnZn9lwVDlgzw=
-github.com/ugorji/go v1.1.7/go.mod h1:kZn38zHttfInRq0xu/PH0az30d+z6vm202qpg1oXVMw=
 github.com/ugorji/go/codec v0.0.0-20181204163529-d75b2dcb6bc8/go.mod h1:VFNgLljTbGfSG7qAOspJ7OScBnGdDN/yBr0sguwnwf0=
-github.com/ugorji/go/codec v1.1.7 h1:2SvQaVZ1ouYrrKKwoSk2pzd4A9evlKJb9oTL+OaLUSs=
-github.com/ugorji/go/codec v1.1.7/go.mod h1:Ax+UKWsSmolVDwsd+7N3ZtXu+yMGCf907BLYF3GoBXY=
 github.com/ultraware/funlen v0.0.3/go.mod h1:Dp4UiAus7Wdb9KUZsYWZEWiRzGuM2kXM1lPbfaF6xhA=
 github.com/ultraware/whitespace v0.0.4/go.mod h1:aVMh/gQve5Maj9hQ/hg+F75lr/X5A89uZnzAmWSineA=
 github.com/urfave/cli v1.20.0/go.mod h1:70zkFmudgCuE/ngEzBv17Jvp/497gISqfk5gWijbERA=
@@ -962,8 +941,6 @@ github.com/yuin/goldmark v1.2.1/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9dec
 github.com/yuin/goldmark v1.3.5/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
 github.com/yuin/goldmark v1.4.0/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1Zlc8k=
 github.com/zenazn/goji v0.9.0/go.mod h1:7S9M489iMyHBNxwZnk9/EHS098H4/F6TATF2mIxtB1Q=
-github.com/zsais/go-gin-prometheus v0.1.0 h1:bkLv1XCdzqVgQ36ScgRi09MA2UC1t3tAB6nsfErsGO4=
-github.com/zsais/go-gin-prometheus v0.1.0/go.mod h1:Slirjzuz8uM8Cw0jmPNqbneoqcUtY2GGjn2bEd4NRLY=
 go.etcd.io/bbolt v1.3.3/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
 go.etcd.io/bbolt v1.3.4/go.mod h1:G5EMThwa9y8QZGBClrRx5EY+Yw9kAhnjy3bSjsnlVTQ=
 go.etcd.io/etcd v0.0.0-20200513171258-e048e166ab9c/go.mod h1:xCI7ZzBfRuGgBXyXO6yfWfDmlWd35khcWpUa4L0xI/k=
--- a/integration_cli_test.go
+++ b/integration_cli_test.go
@@ -40,13 +40,13 @@ func (s *IntegrationCLITestSuite) SetupTest() {
 	if ppool, err := dockertest.NewPool(""); err == nil {
 		s.pool = *ppool
 	} else {
-		log.Fatalf("Could not connect to docker: %s", err)
+		s.FailNow(fmt.Sprintf("Could not connect to docker: %s", err), "")
 	}

 	if pnetwork, err := s.pool.CreateNetwork("headscale-test"); err == nil {
 		s.network = *pnetwork
 	} else {
-		log.Fatalf("Could not create network: %s", err)
+		s.FailNow(fmt.Sprintf("Could not create network: %s", err), "")
 	}

 	headscaleBuildOptions := &dockertest.BuildOptions{
@@ -56,7 +56,7 @@ func (s *IntegrationCLITestSuite) SetupTest() {

 	currentPath, err := os.Getwd()
 	if err != nil {
-		log.Fatalf("Could not determine current path: %s", err)
+		s.FailNow(fmt.Sprintf("Could not determine current path: %s", err), "")
 	}

 	headscaleOptions := &dockertest.RunOptions{
@@ -68,11 +68,16 @@ func (s *IntegrationCLITestSuite) SetupTest() {
 		Cmd:      []string{"headscale", "serve"},
 	}

+	err = s.pool.RemoveContainerByName(headscaleHostname)
+	if err != nil {
+		s.FailNow(fmt.Sprintf("Could not remove existing container before building test: %s", err), "")
+	}
+
 	fmt.Println("Creating headscale container")
 	if pheadscale, err := s.pool.BuildAndRunWithBuildOptions(headscaleBuildOptions, headscaleOptions, DockerRestartPolicy); err == nil {
 		s.headscale = *pheadscale
 	} else {
-		log.Fatalf("Could not start headscale container: %s", err)
+		s.FailNow(fmt.Sprintf("Could not start headscale container: %s", err), "")
 	}
 	fmt.Println("Created headscale container")

--- a/integration_common_test.go
+++ b/integration_common_test.go
@@ -6,7 +6,10 @@ package headscale
 import (
 	"bytes"
 	"encoding/json"
+	"errors"
 	"fmt"
+	"os"
+	"strconv"
 	"strings"
 	"time"

@@ -16,9 +19,13 @@ import (
 	"inet.af/netaddr"
 )

-const DOCKER_EXECUTE_TIMEOUT = 10 * time.Second
+const (
+	DOCKER_EXECUTE_TIMEOUT = 10 * time.Second
+)

 var (
+	errEnvVarEmpty = errors.New("getenv: environment variable empty")
+
 	IpPrefix4 = netaddr.MustParseIPPrefix("100.64.0.0/10")
 	IpPrefix6 = netaddr.MustParseIPPrefix("fd7a:115c:a1e0::/48")

@@ -283,3 +290,25 @@ func getMagicFQDN(

 	return hostnames, nil
 }
+
+func GetEnvStr(key string) (string, error) {
+	v := os.Getenv(key)
+	if v == "" {
+		return v, errEnvVarEmpty
+	}
+
+	return v, nil
+}
+
+func GetEnvBool(key string) (bool, error) {
+	s, err := GetEnvStr(key)
+	if err != nil {
+		return false, err
+	}
+	v, err := strconv.ParseBool(s)
+	if err != nil {
+		return false, err
+	}
+
+	return v, nil
+}
--- a/integration_embedded_derp_test.go
+++ b/integration_embedded_derp_test.go
@@ -40,41 +40,50 @@ type IntegrationDERPTestSuite struct {
 	pool      dockertest.Pool
 	networks  map[int]dockertest.Network // so we keep the containers isolated
 	headscale dockertest.Resource
+	saveLogs  bool

 	tailscales    map[string]dockertest.Resource
 	joinWaitGroup sync.WaitGroup
 }

 func TestDERPIntegrationTestSuite(t *testing.T) {
+	saveLogs, err := GetEnvBool("HEADSCALE_INTEGRATION_SAVE_LOG")
+	if err != nil {
+		saveLogs = false
+	}
+
 	s := new(IntegrationDERPTestSuite)

 	s.tailscales = make(map[string]dockertest.Resource)
 	s.networks = make(map[int]dockertest.Network)
+	s.saveLogs = saveLogs

 	suite.Run(t, s)

 	// HandleStats, which allows us to check if we passed and save logs
 	// is called after TearDown, so we cannot tear down containers before
 	// we have potentially saved the logs.
-	for _, tailscale := range s.tailscales {
-		if err := s.pool.Purge(&tailscale); err != nil {
+	if s.saveLogs {
+		for _, tailscale := range s.tailscales {
+			if err := s.pool.Purge(&tailscale); err != nil {
+				log.Printf("Could not purge resource: %s\n", err)
+			}
+		}
+
+		if !s.stats.Passed() {
+			err := s.saveLog(&s.headscale, "test_output")
+			if err != nil {
+				log.Printf("Could not save log: %s\n", err)
+			}
+		}
+		if err := s.pool.Purge(&s.headscale); err != nil {
 			log.Printf("Could not purge resource: %s\n", err)
 		}
-	}

-	if !s.stats.Passed() {
-		err := s.saveLog(&s.headscale, "test_output")
-		if err != nil {
-			log.Printf("Could not save log: %s\n", err)
-		}
-	}
-	if err := s.pool.Purge(&s.headscale); err != nil {
-		log.Printf("Could not purge resource: %s\n", err)
-	}
-
-	for _, network := range s.networks {
-		if err := network.Close(); err != nil {
-			log.Printf("Could not close network: %s\n", err)
+		for _, network := range s.networks {
+			if err := network.Close(); err != nil {
+				log.Printf("Could not close network: %s\n", err)
+			}
 		}
 	}
 }
@@ -83,14 +92,14 @@ func (s *IntegrationDERPTestSuite) SetupSuite() {
 	if ppool, err := dockertest.NewPool(""); err == nil {
 		s.pool = *ppool
 	} else {
-		log.Fatalf("Could not connect to docker: %s", err)
+		s.FailNow(fmt.Sprintf("Could not connect to docker: %s", err), "")
 	}

 	for i := 0; i < totalContainers; i++ {
 		if pnetwork, err := s.pool.CreateNetwork(fmt.Sprintf("headscale-derp-%d", i)); err == nil {
 			s.networks[i] = *pnetwork
 		} else {
-			log.Fatalf("Could not create network: %s", err)
+			s.FailNow(fmt.Sprintf("Could not create network: %s", err), "")
 		}
 	}

@@ -101,7 +110,7 @@ func (s *IntegrationDERPTestSuite) SetupSuite() {

 	currentPath, err := os.Getwd()
 	if err != nil {
-		log.Fatalf("Could not determine current path: %s", err)
+		s.FailNow(fmt.Sprintf("Could not determine current path: %s", err), "")
 	}

 	headscaleOptions := &dockertest.RunOptions{
@@ -120,11 +129,16 @@ func (s *IntegrationDERPTestSuite) SetupSuite() {
 		},
 	}

+	err = s.pool.RemoveContainerByName(headscaleHostname)
+	if err != nil {
+		s.FailNow(fmt.Sprintf("Could not remove existing container before building test: %s", err), "")
+	}
+
 	log.Println("Creating headscale container")
 	if pheadscale, err := s.pool.BuildAndRunWithBuildOptions(headscaleBuildOptions, headscaleOptions, DockerRestartPolicy); err == nil {
 		s.headscale = *pheadscale
 	} else {
-		log.Fatalf("Could not start headscale container: %s", err)
+		s.FailNow(fmt.Sprintf("Could not start headscale container: %s", err), "")
 	}
 	log.Println("Created headscale container to test DERP")

@@ -290,6 +304,23 @@ func (s *IntegrationDERPTestSuite) tailscaleContainer(
 }

 func (s *IntegrationDERPTestSuite) TearDownSuite() {
+	if !s.saveLogs {
+		for _, tailscale := range s.tailscales {
+			if err := s.pool.Purge(&tailscale); err != nil {
+				log.Printf("Could not purge resource: %s\n", err)
+			}
+		}
+
+		if err := s.pool.Purge(&s.headscale); err != nil {
+			log.Printf("Could not purge resource: %s\n", err)
+		}
+
+		for _, network := range s.networks {
+			if err := network.Close(); err != nil {
+				log.Printf("Could not close network: %s\n", err)
+			}
+		}
+	}
 }

 func (s *IntegrationDERPTestSuite) HandleStats(
--- a/integration_test.go
+++ b/integration_test.go
@@ -36,6 +36,7 @@ type IntegrationTestSuite struct {
 	pool      dockertest.Pool
 	network   dockertest.Network
 	headscale dockertest.Resource
+	saveLogs  bool

 	namespaces map[string]TestNamespace

@@ -43,6 +44,11 @@ type IntegrationTestSuite struct {
 }

 func TestIntegrationTestSuite(t *testing.T) {
+	saveLogs, err := GetEnvBool("HEADSCALE_INTEGRATION_SAVE_LOG")
+	if err != nil {
+		saveLogs = false
+	}
+
 	s := new(IntegrationTestSuite)

 	s.namespaces = map[string]TestNamespace{
@@ -55,32 +61,35 @@ func TestIntegrationTestSuite(t *testing.T) {
 			tailscales: make(map[string]dockertest.Resource),
 		},
 	}
+	s.saveLogs = saveLogs

 	suite.Run(t, s)

 	// HandleStats, which allows us to check if we passed and save logs
 	// is called after TearDown, so we cannot tear down containers before
 	// we have potentially saved the logs.
-	for _, scales := range s.namespaces {
-		for _, tailscale := range scales.tailscales {
-			if err := s.pool.Purge(&tailscale); err != nil {
-				log.Printf("Could not purge resource: %s\n", err)
+	if s.saveLogs {
+		for _, scales := range s.namespaces {
+			for _, tailscale := range scales.tailscales {
+				if err := s.pool.Purge(&tailscale); err != nil {
+					log.Printf("Could not purge resource: %s\n", err)
+				}
 			}
 		}
-	}

-	if !s.stats.Passed() {
-		err := s.saveLog(&s.headscale, "test_output")
-		if err != nil {
-			log.Printf("Could not save log: %s\n", err)
+		if !s.stats.Passed() {
+			err := s.saveLog(&s.headscale, "test_output")
+			if err != nil {
+				log.Printf("Could not save log: %s\n", err)
+			}
+		}
+		if err := s.pool.Purge(&s.headscale); err != nil {
+			log.Printf("Could not purge resource: %s\n", err)
 		}
-	}
-	if err := s.pool.Purge(&s.headscale); err != nil {
-		log.Printf("Could not purge resource: %s\n", err)
-	}

-	if err := s.network.Close(); err != nil {
-		log.Printf("Could not close network: %s\n", err)
+		if err := s.network.Close(); err != nil {
+			log.Printf("Could not close network: %s\n", err)
+		}
 	}
 }

@@ -209,13 +218,13 @@ func (s *IntegrationTestSuite) SetupSuite() {
 	if ppool, err := dockertest.NewPool(""); err == nil {
 		s.pool = *ppool
 	} else {
-		log.Fatalf("Could not connect to docker: %s", err)
+		s.FailNow(fmt.Sprintf("Could not connect to docker: %s", err), "")
 	}

 	if pnetwork, err := s.pool.CreateNetwork("headscale-test"); err == nil {
 		s.network = *pnetwork
 	} else {
-		log.Fatalf("Could not create network: %s", err)
+		s.FailNow(fmt.Sprintf("Could not create network: %s", err), "")
 	}

 	headscaleBuildOptions := &dockertest.BuildOptions{
@@ -225,7 +234,7 @@ func (s *IntegrationTestSuite) SetupSuite() {

 	currentPath, err := os.Getwd()
 	if err != nil {
-		log.Fatalf("Could not determine current path: %s", err)
+		s.FailNow(fmt.Sprintf("Could not determine current path: %s", err), "")
 	}

 	headscaleOptions := &dockertest.RunOptions{
@@ -237,11 +246,16 @@ func (s *IntegrationTestSuite) SetupSuite() {
 		Cmd:      []string{"headscale", "serve"},
 	}

+	err = s.pool.RemoveContainerByName(headscaleHostname)
+	if err != nil {
+		s.FailNow(fmt.Sprintf("Could not remove existing container before building test: %s", err), "")
+	}
+
 	log.Println("Creating headscale container")
 	if pheadscale, err := s.pool.BuildAndRunWithBuildOptions(headscaleBuildOptions, headscaleOptions, DockerRestartPolicy); err == nil {
 		s.headscale = *pheadscale
 	} else {
-		log.Fatalf("Could not start headscale container: %s", err)
+		s.FailNow(fmt.Sprintf("Could not start headscale container: %s", err), "")
 	}
 	log.Println("Created headscale container")

@@ -338,6 +352,23 @@ func (s *IntegrationTestSuite) SetupSuite() {
 }

 func (s *IntegrationTestSuite) TearDownSuite() {
+	if !s.saveLogs {
+		for _, scales := range s.namespaces {
+			for _, tailscale := range scales.tailscales {
+				if err := s.pool.Purge(&tailscale); err != nil {
+					log.Printf("Could not purge resource: %s\n", err)
+				}
+			}
+		}
+
+		if err := s.pool.Purge(&s.headscale); err != nil {
+			log.Printf("Could not purge resource: %s\n", err)
+		}
+
+		if err := s.network.Close(); err != nil {
+			log.Printf("Could not close network: %s\n", err)
+		}
+	}
 }

 func (s *IntegrationTestSuite) HandleStats(
--- a/integration_test/etc/alt-config.dump.gold.yaml
+++ b/integration_test/etc/alt-config.dump.gold.yaml
@@ -20,6 +20,7 @@ dns_config:
  nameservers:
    - 1.1.1.1
 ephemeral_node_inactivity_timeout: 30m
+node_update_check_interval: 10s
 grpc_allow_insecure: false
 grpc_listen_addr: :50443
 ip_prefixes:
--- a/integration_test/etc/alt-config.yaml
+++ b/integration_test/etc/alt-config.yaml
@@ -2,6 +2,7 @@ log_level: trace
 acl_policy_path: ""
 db_type: sqlite3
 ephemeral_node_inactivity_timeout: 30m
+node_update_check_interval: 10s
 ip_prefixes:
  - fd7a:115c:a1e0::/48
  - 100.64.0.0/10
--- a/integration_test/etc/config.dump.gold.yaml
+++ b/integration_test/etc/config.dump.gold.yaml
@@ -20,6 +20,7 @@ dns_config:
  nameservers:
    - 1.1.1.1
 ephemeral_node_inactivity_timeout: 30m
+node_update_check_interval: 10s
 grpc_allow_insecure: false
 grpc_listen_addr: :50443
 ip_prefixes:
--- a/integration_test/etc/config.yaml
+++ b/integration_test/etc/config.yaml
@@ -2,6 +2,7 @@ log_level: trace
 acl_policy_path: ""
 db_type: sqlite3
 ephemeral_node_inactivity_timeout: 30m
+node_update_check_interval: 10s
 ip_prefixes:
  - fd7a:115c:a1e0::/48
  - 100.64.0.0/10
--- a/integration_test/etc_embedded_derp/config.yaml
+++ b/integration_test/etc_embedded_derp/config.yaml
@@ -2,6 +2,7 @@ log_level: trace
 acl_policy_path: ""
 db_type: sqlite3
 ephemeral_node_inactivity_timeout: 30m
+node_update_check_interval: 10s
 ip_prefixes:
  - fd7a:115c:a1e0::/48
  - 100.64.0.0/10
--- a/poll.go
+++ b/poll.go
@@ -16,8 +16,7 @@ import (
 )

 const (
-	keepAliveInterval   = 60 * time.Second
-	updateCheckInterval = 10 * time.Second
+	keepAliveInterval = 60 * time.Second
 )

 type contextKey string
@@ -291,6 +290,9 @@ func (h *Headscale) PollNetMapStream(
 	keepAliveChan chan []byte,
 	updateChan chan struct{},
 ) {
+	h.pollNetMapStreamWG.Add(1)
+	defer h.pollNetMapStreamWG.Done()
+
 	ctx := context.WithValue(req.Context(), machineNameContextKey, machine.Hostname)

 	ctx, cancel := context.WithCancel(ctx)
@@ -640,7 +642,7 @@ func (h *Headscale) scheduledPollWorker(
 	machine *Machine,
 ) {
 	keepAliveTicker := time.NewTicker(keepAliveInterval)
-	updateCheckerTicker := time.NewTicker(updateCheckInterval)
+	updateCheckerTicker := time.NewTicker(h.cfg.NodeUpdateCheckInterval)

 	defer closeChanWithLog(
 		updateChan,
Author	SHA1	Message	Date
Juan Font	0212db3fad	Merge pull request #687 from huskyii/node_ls more intuitive output of node ls	2022-07-24 12:06:41 +02:00
Jiang Zhu	49354f678e	update CHANGELOG	2022-07-23 04:47:37 +08:00
Jiang Zhu	dc94570c4a	more intuitive output of node ls	2022-07-23 01:33:11 +08:00
Kristoffer Dalby	51b1027aec	Merge pull request #686 from juanfont/update-contributors	2022-07-22 18:56:49 +02:00
github-actions[bot]	936adb7d2c	docs(README): update contributors	2022-07-22 07:36:16 +00:00
Juan Font	581d1f3bfa	Merge pull request #668 from GrigoriyMikhalkin/graceful-shutdown graceful shutdown fix	2022-07-22 09:35:40 +02:00
Juan Font	7c87ef6c86	Merge branch 'main' into graceful-shutdown	2022-07-22 09:06:46 +02:00
Juan Font	1a9a9b718d	Merge pull request #684 from juanfont/fix-api-mux Fix API router	2022-07-22 09:06:06 +02:00
Juan Font Alonso	6c9f3420e2	Updated changelog	2022-07-21 23:59:44 +02:00
Juan Font Alonso	a4d0efbe8d	Fix API router	2022-07-21 23:57:07 +02:00
Grigoriy Mikhalkin	56858a56db	Revert "decompose OIDCCallback method" This reverts commit `395caaad42`.	2022-07-21 23:54:35 +02:00
Grigoriy Mikhalkin	395caaad42	decompose OIDCCallback method	2022-07-21 23:47:20 +02:00
Grigoriy Mikhalkin	3f0639c87d	graceful shutdown lint fixes	2022-07-21 23:47:20 +02:00
Grigoriy Mikhalkin	889eff265f	graceful shutdown fix	2022-07-21 23:47:20 +02:00
Kristoffer Dalby	c6eb7be7fb	Merge pull request #683 from juanfont/update-contributors	2022-07-20 10:57:38 +02:00
github-actions[bot]	02c7a46b97	docs(README): update contributors	2022-07-20 07:21:19 +00:00
Kristoffer Dalby	ea7b3baa8b	Merge pull request #677 from huskyii/remove_gin	2022-07-20 09:20:24 +02:00
Jiang Zhu	5724f4607c	fix nix build	2022-07-19 20:45:32 +08:00
Jiang Zhu	b755d47652	update CHANGELOG	2022-07-19 20:45:23 +08:00
Jiang Zhu	aca5646032	remove gin completely, ~2MB reduction on final binary	2022-07-16 02:03:46 +08:00
Kristoffer Dalby	7e9abbeaec	Merge pull request #676 from juanfont/update-contributors	2022-07-15 09:15:18 +01:00
github-actions[bot]	b8c3387892	docs(README): update contributors	2022-07-12 11:35:28 +00:00
Juan Font	c50d3aa9bd	Merge pull request #675 from juanfont/configurable-update-interval Make tailnet updates check interval configurable	2022-07-12 13:34:49 +02:00
Juan Font Alonso	4ccff8bf28	Added the new parameter to the integration test params	2022-07-12 13:13:04 +02:00
Juan Font Alonso	5b5298b025	Renamed config param for node update check internal	2022-07-12 12:52:03 +02:00
Juan Font Alonso	8e0939f403	Updated changelog	2022-07-12 12:33:42 +02:00
Juan Font Alonso	cf3fc85196	Make tailnet updates check configurable	2022-07-12 12:27:28 +02:00
Juan Font	e0b15c18ce	Merge pull request #667 from kradalby/rerun-docker Make integration tests retry on failure.	2022-06-27 17:04:39 +02:00
Kristoffer Dalby	566b8c3df3	Fix issue were dockertest fails to start because of container mismatch	2022-06-27 12:07:30 +00:00
Kristoffer Dalby	32a6151df9	Rerun integration tests 5 times if error	2022-06-27 12:02:29 +00:00
Kristoffer Dalby	3777de7133	Use failnow for cli tests aswell	2022-06-27 12:00:21 +00:00
Kristoffer Dalby	8cae4f80d7	Fail tests instead of fatal Currently we exit the program if the setup does not work, this can cause is to leave containers and other resources behind since we dont run TearDown. This change will just fail the test if we cant set up, which should mean that the TearDown runs aswell.	2022-06-27 11:58:16 +00:00
Kristoffer Dalby	911c5bddce	Make saving logs from tests an option (default false) We currently have a bit of flaky logic which prevents the docker plugin from cleaning up the containers if the tests or setup fatals or crashes, this is due to a limitation in the save / passed stats handling. This change makes it an environment variable which by default ditches the logs and makes the containers clean up "correctly" in the teardown method.	2022-06-27 11:56:37 +00:00