tailscale/docs/k8s/operator-architecture.md

Operator architecture diagrams

The Tailscale Kubernetes operator has a collection of use-cases that can be mixed and matched as required. The following diagrams illustrate how the operator implements each use-case.

In each diagram, the "tailscale" namespace is entirely managed by the operator once the operator itself has been deployed.

Tailscale devices are highlighted as black nodes. The salient devices for each use-case are marked as "src" or "dst" to denote which node is a source or a destination in the context of ACL rules that will apply to network traffic.

API server proxy

Documentation

The operator runs the API server proxy in-process. If the proxy is running in "noauth" mode, it forwards HTTP requests unmodified. If the proxy is running in "auth" mode, it deletes any existing auth headers and adds impersonation headers to the request before forwarding to the API server.

%%{ init: { 'theme':'neutral' } }%%
flowchart LR
    classDef tsnode color:#fff,fill:#000;
    classDef pod fill:#fff;

    subgraph Key
        ts[Tailscale device]:::tsnode
    end

    subgraph k8s[Kubernetes cluster]
        subgraph tailscale-ns[tailscale]
            operator["operator (dst)"]:::tsnode
        end

        subgraph controlplane["Control plane"]
            api[kube-apiserver]
        end
    end
    client["client (src)"]:::tsnode --> operator
    operator-->|proxies requests| api

L3 ingress

Documentation

The user deploys an app to the default namespace, and creates a normal Service that selects the app's pods. Add the annotation tailscale.com/expose: "true" to the Service, and the operator will create an ingress proxy that allows devices anywhere on the tailnet to access the Service.

%%{ init: { 'theme':'neutral' } }%%
flowchart TD
    classDef tsnode color:#fff,fill:#000;
    classDef pod fill:#fff;

    subgraph Key
        ts[Tailscale device]:::tsnode
        pod(Pod):::pod
    end

    subgraph k8s[Kubernetes cluster]
        subgraph tailscale-ns[tailscale]
            operator(operator):::tsnode
            ingress("ingress proxy (dst)"):::tsnode
            secret
        end

        subgraph defaultns[default]
            svc[annotated svc]
            svc --> app1(app)
            svc --> app2(app)
        end
    end
    client["client (src)"]:::tsnode --> ingress
    ingress -->|forwards traffic| svc
    operator -.->|deploys| ingress
    operator -.->|reads| svc
    operator -.->|creates| secret
    secret -.->|mounted| ingress

L7 ingress

Documentation

L3 egress

Documentation

1. The user deploys a Service named db with type: ExternalName and an annotation tailscale.com/tailnet-fqdn: db.tails-scales.ts.net.
2. The operator creates a proxy Pod managed by a single replica StatefulSet, and a headless Service pointing at the proxy Pod.
3. The operator updates the db Service's spec.externalName field to point at the headless Service it created in the previous step.

(Optional) If the user also adds the tailscale.com/proxy-group: egress-proxies annotation to their db Service, the operator will skip creating a proxy Pod and instead point the headless Service at the existing ProxyGroup's pods. In this case, ports are also required in the db Service spec.

Note, in some cases, the config and the state Secret may be the same Kubernetes Secret.

%%{ init: { 'theme':'neutral' } }%%

flowchart TD
    classDef tsnode color:#fff,fill:#000;
    classDef pod fill:#fff;

    subgraph Key
        ts[Tailscale device]:::tsnode
        pod(Pod):::pod
    end

    subgraph k8s[Kubernetes cluster]
        subgraph tailscale-ns[tailscale]
            operator(operator):::tsnode
            egress("egress proxy (src)"):::tsnode
            headless-svc[headless svc]
            cfg-secret["config Secret"]
            state-secret["state Secret"]
        end

        subgraph defaultns[default]
            svc[db ExternalName svc]
            app1(app) --> svc
            app2(app) --> svc
        end
    end
    node["db.tails-scales.ts.net (dst)"]:::tsnode
    svc -->|DNS points to| headless-svc
    headless-svc -->|forwards traffic| egress
    egress -->|forwards traffic| node
    operator -.->|deploys| egress
    operator -.->|deploys| headless-svc
    operator -.->|creates| cfg-secret
    operator -.->|watches & updates| svc
    cfg-secret -.->|mounted| egress
    egress -.->|stores state| state-secret

ProxyGroup

Documentation

The ProxyGroup custom resource manages a collection of proxy Pods that can be configured to egress traffic out of the cluster via ExternalName Services defined elsewhere. They will also support ingress in the future. In this diagram, the ProxyGroup is named pg, and the operator creates proxy pods, via a StatefulSet but they don't yet serve any traffic.

ProxyGroups currently only support egress (see above).

%%{ init: { 'theme':'neutral' } }%%

flowchart TD
    classDef tsnode color:#fff,fill:#000;
    classDef pod fill:#fff;

    subgraph Key
        ts[Tailscale device]:::tsnode
        pod(Pod):::pod
    end

    subgraph k8s[Kubernetes cluster]
        subgraph tailscale-ns[tailscale]
            operator(operator):::tsnode
            pg-sts[pg StatefulSet]
            pg-0("pg-0 (src)"):::tsnode
            pg-1("pg-1 (src)"):::tsnode
            cfg-secret-0["pg-0-config Secret"]
            cfg-secret-1["pg-1-config Secret"]
            state-secret-0["pg-0 Secret"]
            state-secret-1["pg-1 Secret"]
        end

        subgraph cluster-scope["Cluster scoped resources"]
            pg["pg ProxyGroup"]
        end
    end
    operator-.->|watches| pg
    operator -.->|deploys| pg-sts
    pg-sts -.->|manages| pg-0
    pg-sts -.->|manages| pg-1
    operator -.->|creates| cfg-secret-0
    operator -.->|creates| cfg-secret-1
    cfg-secret-0 -.->|mounted| pg-0
    cfg-secret-1 -.->|mounted| pg-1
    pg-0 -.->|stores state| state-secret-0
    pg-1 -.->|stores state| state-secret-1

Subnet routers and exit nodes

Documentation

Recorder nodes

Documentation

The Recorder custom resource makes it easier to deploy tsrecorder to a cluster. It currently only supports a single replica.

%%{ init: { 'theme':'neutral' } }%%

flowchart TD
    classDef tsnode color:#fff,fill:#000;
    classDef pod fill:#fff;

    subgraph Key
        ts[Tailscale device]:::tsnode
        pod(Pod):::pod
    end

    subgraph k8s[Kubernetes cluster]
        subgraph tailscale-ns[tailscale]
            operator(operator):::tsnode
            rec-sts[rec StatefulSet]
            rec-0("rec-0 Pod (tsrecorder)"):::tsnode
            cfg-secret-0["rec-0-config Secret"]
            state-secret-0["rec-0 Secret"]
        end

        subgraph cluster-scope["Cluster scoped resources"]
            rec["rec Recorder"]
        end
    end
    operator-.->|watches| rec
    operator -.->|deploys| rec-sts
    rec-sts -.->|manages| rec-0
    operator -.->|creates| cfg-secret-0
    cfg-secret-0 -.->|mounted| rec-0
    rec-0 -.->|stores state| state-secret-0