cmd/{k8s-operator,k8s-proxy}: add kube-apiserver ProxyGroup type (#16266)

Adds a new k8s-proxy command to convert operator's in-process proxy to
a separately deployable type of ProxyGroup: kube-apiserver. k8s-proxy
reads in a new config file written by the operator, modelled on tailscaled's
conffile but with some modifications to ensure multiple versions of the
config can co-exist within a file. This should make it much easier to
support reading that config file from a Kube Secret with a stable file name.

To avoid needing to give the operator ClusterRole{,Binding} permissions,
the helm chart now optionally deploys a new static ServiceAccount for
the API Server proxy to use if in auth mode.

Proxies deployed by kube-apiserver ProxyGroups currently work the same as
the operator's in-process proxy. They do not yet leverage Tailscale Services
for presenting a single HA DNS name.

Updates #13358

Change-Id: Ib6ead69b2173c5e1929f3c13fb48a9a5362195d8
Signed-off-by: Tom Proctor <tomhjp@users.noreply.github.com>

cmd/k8s-operator/deploy/chart/templates/apiserverproxy-rbac.yaml

@@ -1,7 +1,16 @@
 # Copyright (c) Tailscale Inc & AUTHORS
 # SPDX-License-Identifier: BSD-3-Clause
 
-{{ if eq .Values.apiServerProxyConfig.mode "true" }}
+# If old setting used, enable both old (operator) and new (ProxyGroup) workflows.
+# If new setting used, enable only new workflow.
+{{ if or (eq .Values.apiServerProxyConfig.mode "true")
+  (eq .Values.apiServerProxyConfig.allowImpersonation "true") }}
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: kube-apiserver-auth-proxy
+  namespace: {{ .Release.Namespace }}
+---
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRole
 metadata:
@@ -16,9 +25,14 @@ kind: ClusterRoleBinding
 metadata:
   name: tailscale-auth-proxy
 subjects:
+{{- if eq .Values.apiServerProxyConfig.mode "true" }}
 - kind: ServiceAccount
   name: operator
   namespace: {{ .Release.Namespace }}
+{{- end }}
+- kind: ServiceAccount
+  name: kube-apiserver-auth-proxy
+  namespace: {{ .Release.Namespace }}
 roleRef:
   kind: ClusterRole
   name: tailscale-auth-proxy

cmd/k8s-operator/deploy/chart/values.yaml

@@ -92,6 +92,13 @@ ingressClass:
 # If you need more configuration options, take a look at ProxyClass:
 # https://tailscale.com/kb/1445/kubernetes-operator-customization#cluster-resource-customization-using-proxyclass-custom-resource
 proxyConfig:
+  # Configure the proxy image to use instead of the default tailscale/tailscale:latest.
+  # Applying a ProxyClass with `spec.statefulSet.pod.tailscaleContainer.image`
+  # set will override any defaults here.
+  #
+  # Note that ProxyGroups of type "kube-apiserver" use a different default image,
+  # tailscale/k8s-proxy:latest, and it is currently only possible to override
+  # that image via the same ProxyClass field.
   image:
     # Repository defaults to DockerHub, but images are also synced to ghcr.io/tailscale/tailscale.
     repository: tailscale/tailscale
@@ -115,6 +122,15 @@ proxyConfig:
 # Kubernetes API server.
 # https://tailscale.com/kb/1437/kubernetes-operator-api-server-proxy
 apiServerProxyConfig:
+  # Set to "true" to create the ClusterRole permissions required for the API
+  # server proxy's auth mode. In auth mode, the API server proxy impersonates
+  # groups and users based on tailnet ACL grants. Required for ProxyGroups of
+  # type "kube-apiserver" running in auth mode.
+  allowImpersonation: "false" # "true", "false"
+
+  # If true or noauth, the operator will run an in-process API server proxy.
+  # You can deploy a ProxyGroup of type "kube-apiserver" to run a high
+  # availability set of API server proxies instead.
   mode: "false" # "true", "false", "noauth"
 
 imagePullSecrets: []