cmd/k8s-operator,docs/k8s: run tun mode proxies in privileged containers (#14262)

We were previously relying on unintended behaviour by runc where
all containers where by default given read/write/mknod permissions
for tun devices.
This behaviour was removed in https://github.com/opencontainers/runc/pull/3468
and released in runc 1.2.
Containerd container runtime, used by Docker and majority of Kubernetes distributions
bumped runc to 1.2 in 1.7.24 https://github.com/containerd/containerd/releases/tag/v1.7.24
thus breaking our reference tun mode Tailscale Kubernetes manifests and Kubernetes
operator proxies.

This PR changes the all Kubernetes container configs that run Tailscale in tun mode
to privileged. This should not be a breaking change because all these containers would
run in a Pod that already has a privileged init container.

Updates tailscale/tailscale#14256
Updates tailscale/tailscale#10814

Signed-off-by: Irbe Krumina <irbe@tailscale.com>

cmd/k8s-operator/deploy/crds/tailscale.com_proxyclasses.yaml

@@ -1384,11 +1384,12 @@ spec:
                             securityContext:
                               description: |-
                                 Container security context.
-                                Security context specified here will override the security context by the operator.
+                                Security context specified here will override the security context set by the operator.
-                                By default the operator:
+                                By default the operator sets the Tailscale container and the Tailscale init container to privileged
-                                - sets 'privileged: true' for the init container
+                                for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.
-                                - set NET_ADMIN capability for tailscale container for proxies that
+                                You can reduce the permissions of the Tailscale container to cap NET_ADMIN by
-                                are created for Services or Connector.
+                                installing device plugin in your cluster and configuring the proxies tun device to be created
+                                by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752
                                 https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context
                               type: object
                               properties:
@@ -1707,11 +1708,12 @@ spec:
                             securityContext:
                               description: |-
                                 Container security context.
-                                Security context specified here will override the security context by the operator.
+                                Security context specified here will override the security context set by the operator.
-                                By default the operator:
+                                By default the operator sets the Tailscale container and the Tailscale init container to privileged
-                                - sets 'privileged: true' for the init container
+                                for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.
-                                - set NET_ADMIN capability for tailscale container for proxies that
+                                You can reduce the permissions of the Tailscale container to cap NET_ADMIN by
-                                are created for Services or Connector.
+                                installing device plugin in your cluster and configuring the proxies tun device to be created
+                                by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752
                                 https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context
                               type: object
                               properties:

cmd/k8s-operator/deploy/manifests/operator.yaml

@@ -1851,11 +1851,12 @@ spec:
                                                     securityContext:
                                                         description: |-
                                                             Container security context.
-                                                            Security context specified here will override the security context by the operator.
+                                                            Security context specified here will override the security context set by the operator.
-                                                            By default the operator:
+                                                            By default the operator sets the Tailscale container and the Tailscale init container to privileged
-                                                            - sets 'privileged: true' for the init container
+                                                            for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.
-                                                            - set NET_ADMIN capability for tailscale container for proxies that
+                                                            You can reduce the permissions of the Tailscale container to cap NET_ADMIN by
-                                                            are created for Services or Connector.
+                                                            installing device plugin in your cluster and configuring the proxies tun device to be created
+                                                            by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752
                                                             https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context
                                                         properties:
                                                             allowPrivilegeEscalation:
@@ -2174,11 +2175,12 @@ spec:
                                                     securityContext:
                                                         description: |-
                                                             Container security context.
-                                                            Security context specified here will override the security context by the operator.
+                                                            Security context specified here will override the security context set by the operator.
-                                                            By default the operator:
+                                                            By default the operator sets the Tailscale container and the Tailscale init container to privileged
-                                                            - sets 'privileged: true' for the init container
+                                                            for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.
-                                                            - set NET_ADMIN capability for tailscale container for proxies that
+                                                            You can reduce the permissions of the Tailscale container to cap NET_ADMIN by
-                                                            are created for Services or Connector.
+                                                            installing device plugin in your cluster and configuring the proxies tun device to be created
+                                                            by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752
                                                             https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context
                                                         properties:
                                                             allowPrivilegeEscalation:

cmd/k8s-operator/deploy/manifests/proxy.yaml

@@ -39,6 +39,4 @@ spec:
                 fieldRef:
                   fieldPath: metadata.uid
           securityContext:
-            capabilities:
+            privileged: true
-              add:
-                - NET_ADMIN

cmd/k8s-operator/testutils_test.go

@@ -76,9 +76,7 @@ func expectedSTS(t *testing.T, cl client.Client, opts configOpts) *appsv1.Statef
 			{Name: "TS_EXPERIMENTAL_VERSIONED_CONFIG_DIR", Value: "/etc/tsconfig"},
 		},
 		SecurityContext: &corev1.SecurityContext{
-			Capabilities: &corev1.Capabilities{
+			Privileged: ptr.To(true),
-				Add: []corev1.Capability{"NET_ADMIN"},
-			},
 		},
 		ImagePullPolicy: "Always",
 	}

docs/k8s/proxy.yaml

@@ -53,6 +53,4 @@ spec:
         fieldRef:
           fieldPath: metadata.uid
     securityContext:
-      capabilities:
+      privileged: true
-        add:
-        - NET_ADMIN

docs/k8s/sidecar.yaml

@@ -35,6 +35,4 @@ spec:
         fieldRef:
           fieldPath: metadata.uid
     securityContext:
-      capabilities:
+      privileged: true
-        add:
-        - NET_ADMIN

docs/k8s/subnet.yaml

@@ -37,6 +37,4 @@ spec:
         fieldRef:
           fieldPath: metadata.uid
     securityContext:
-      capabilities:
+      privileged: true
-        add:
-        - NET_ADMIN

k8s-operator/api.md

@@ -145,7 +145,7 @@ _Appears in:_
 | `image` _string_ | Container image name. By default images are pulled from<br />docker.io/tailscale/tailscale, but the official images are also<br />available at ghcr.io/tailscale/tailscale. Specifying image name here<br />will override any proxy image values specified via the Kubernetes<br />operator's Helm chart values or PROXY_IMAGE env var in the operator<br />Deployment.<br />https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#image |  |  |
 | `imagePullPolicy` _[PullPolicy](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.3/#pullpolicy-v1-core)_ | Image pull policy. One of Always, Never, IfNotPresent. Defaults to Always.<br />https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#image |  | Enum: [Always Never IfNotPresent] <br /> |
 | `resources` _[ResourceRequirements](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.3/#resourcerequirements-v1-core)_ | Container resource requirements.<br />By default Tailscale Kubernetes operator does not apply any resource<br />requirements. The amount of resources required wil depend on the<br />amount of resources the operator needs to parse, usage patterns and<br />cluster size.<br />https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#resources |  |  |
-| `securityContext` _[SecurityContext](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.3/#securitycontext-v1-core)_ | Container security context.<br />Security context specified here will override the security context by the operator.<br />By default the operator:<br />- sets 'privileged: true' for the init container<br />- set NET_ADMIN capability for tailscale container for proxies that<br />are created for Services or Connector.<br />https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context |  |  |
+| `securityContext` _[SecurityContext](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.3/#securitycontext-v1-core)_ | Container security context.<br />Security context specified here will override the security context set by the operator.<br />By default the operator sets the Tailscale container and the Tailscale init container to privileged<br />for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.<br />You can reduce the permissions of the Tailscale container to cap NET_ADMIN by<br />installing device plugin in your cluster and configuring the proxies tun device to be created<br />by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752<br />https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context |  |  |
 | `debug` _[Debug](#debug)_ | Configuration for enabling extra debug information in the container.<br />Not recommended for production use. |  |  |
 
 

k8s-operator/apis/v1alpha1/types_proxyclass.go

@@ -206,11 +206,12 @@ type Container struct {
 	// +optional
 	Resources corev1.ResourceRequirements `json:"resources,omitempty"`
 	// Container security context.
-	// Security context specified here will override the security context by the operator.
+	// Security context specified here will override the security context set by the operator.
-	// By default the operator:
+	// By default the operator sets the Tailscale container and the Tailscale init container to privileged
-	// - sets 'privileged: true' for the init container
+	// for proxies created for Tailscale ingress and egress Service, Connector and ProxyGroup.
-	// - set NET_ADMIN capability for tailscale container for proxies that
+	// You can reduce the permissions of the Tailscale container to cap NET_ADMIN by
-	// are created for Services or Connector.
+	// installing device plugin in your cluster and configuring the proxies tun device to be created
+	// by the device plugin, see  https://github.com/tailscale/tailscale/issues/10814#issuecomment-2479977752
 	// https://kubernetes.io/docs/reference/kubernetes-api/workload-resources/pod-v1/#security-context
 	// +optional
 	SecurityContext *corev1.SecurityContext `json:"securityContext,omitempty"`