TheArchive/tailscale
1
0
Fork 0
mirror of https://github.com/tailscale/tailscale.git synced 2025-07-28 14:53:44 +00:00
Code Issues Packages Projects Releases Wiki Activity

cmd/tsidp: This change enables tsidp to store its operational state in a Kubernetes secret, similar to how tsrecorder and the k8s-operator can.

Browse Source 
It introduces a new '--state' flag (and TS_STATE environment variable) that accepts 'kube:<secret-name>' to specify the Kubernetes secret to use. The necessary RBAC permissions for the tsidp service account are documented in the README.

Updates #15965

Signed-off-by: Raj Singh <raj@tailscale.com>
This commit is contained in:
Raj Singh 2025-07-25 22:26:59 -05:00
parent b63f8a457d
commit 995ab4a252
3 changed files with 180 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
cmd/tsidp/README.md
View File

@ -78,15 +78,82 @@ The `tsidp` server supports several command-line flags:
- `--local-port`: Allow requests from localhost
- `--use-local-tailscaled`: Use local tailscaled instead of tsnet
- `--hostname`: tsnet hostname
- `--dir`: tsnet state directory
- `--dir`: tsnet state directory; a default one will be created if not provided
- `--state`: Path to tailscale state file. Can also be set to use a Kubernetes Secret with the format `kube:<secret-name>`. If unset, `dir` is used for file-based state, or tsnet default if `dir` is also unset.
## Environment Variables
- `TS_AUTHKEY`: Your Tailscale authentication key (required)
- `TS_HOSTNAME`: Hostname for the `tsidp` server (default: "idp", Docker only)
- `TS_STATE_DIR`: State directory (default: "/var/lib/tsidp", Docker only)
- `TS_STATE_DIR`: Default state directory for `tsnet` (default: "/var/lib/tsidp" in Docker). This variable typically sets the default for the `--dir` flag in the Docker environment. `tsnet` uses the directory specified by `--dir` (or its internal default if `--dir` is not set) for its persistent files (e.g., node keys).
- `TAILSCALE_USE_WIP_CODE`: Enable work-in-progress code (default: "1")
## Storing State in Kubernetes Secrets
When running `tsidp` in a Kubernetes environment, you can configure it to store its state in a Kubernetes Secret. This is achieved by setting the `--state` flag to `kube:<your-secret-name>`. The Secret will be created by `tsidp` if it doesn't already exist, and will be created in the same namespace where `tsidp` is running.
**Important**: Each Pod must use its own unique Secret. Multiple Pods cannot share the same Secret for state storage.
For example:
`./tsidp --state kube:my-tsidp-state-secret`
### StatefulSet Example for Multiple Pods
When deploying multiple `tsidp` instances, use a StatefulSet to ensure each Pod gets its own unique Secret:
```yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: tsidp
spec:
replicas: 1
serviceName: tsidp
selector:
matchLabels:
app: tsidp
template:
metadata:
labels:
app: tsidp
spec:
serviceAccountName: tsidp
containers:
- name: tsidp
image: tailscale/tsidp:unstable
args:
- tsidp
- --state=kube:$(POD_NAME)
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: TS_AUTHKEY
valueFrom:
secretKeyRef:
name: tsidp-auth
key: authkey
- name: TAILSCALE_USE_WIP_CODE
value: "1"
```
### Required RBAC Permissions
If you use Kubernetes Secret storage, the service account under which `tsidp` runs needs the following permissions on Secrets in the same namespace:
- `get`
- `patch` (primary mechanism for writing state)
- `create` (if the Secret does not already exist)
- `update` (for backwards compatibility, though patch is preferred)
Additionally, the service account needs the following permissions on Events (for debugging purposes when Secret operations fail):
- `create`
- `patch`
- `get`
Ensure that appropriate Role and RoleBinding are configured in your Kubernetes cluster.
## Support
This is an experimental, work in progress, [community project](https://tailscale.com/kb/1531/community-projects). For issues or questions, file issues on the [GitHub repository](https://github.com/tailscale/tailscale).

6
cmd/tsidp/depaware.txt
View File

@ -254,10 +254,10 @@ tailscale.com/cmd/tsidp dependencies: (generated by github.com/tailscale/depawar
tailscale.com/ipn/policy from tailscale.com/ipn/ipnlocal
tailscale.com/ipn/store from tailscale.com/ipn/ipnlocal+
L tailscale.com/ipn/store/awsstore from tailscale.com/ipn/store
L tailscale.com/ipn/store/kubestore from tailscale.com/ipn/store
tailscale.com/ipn/store/kubestore from tailscale.com/ipn/store+
tailscale.com/ipn/store/mem from tailscale.com/ipn/ipnlocal+
L tailscale.com/kube/kubeapi from tailscale.com/ipn/store/kubestore+
L tailscale.com/kube/kubeclient from tailscale.com/ipn/store/kubestore
tailscale.com/kube/kubeapi from tailscale.com/ipn/store/kubestore+
tailscale.com/kube/kubeclient from tailscale.com/ipn/store/kubestore+
tailscale.com/kube/kubetypes from tailscale.com/envknob+
tailscale.com/licenses from tailscale.com/client/web
tailscale.com/log/filelogger from tailscale.com/logpolicy

109
cmd/tsidp/tsidp.go
View File

@ -42,12 +42,16 @@ import (
"tailscale.com/hostinfo"
"tailscale.com/ipn"
"tailscale.com/ipn/ipnstate"
"tailscale.com/ipn/store"
_ "tailscale.com/ipn/store/kubestore"
"tailscale.com/kube/kubeclient"
"tailscale.com/tailcfg"
"tailscale.com/tsnet"
"tailscale.com/types/key"
"tailscale.com/types/lazy"
"tailscale.com/types/views"
"tailscale.com/util/mak"
"tailscale.com/util/multierr"
"tailscale.com/util/must"
"tailscale.com/util/rands"
"tailscale.com/version"
@ -68,6 +72,7 @@ var (
flagFunnel = flag.Bool("funnel", false, "use Tailscale Funnel to make tsidp available on the public internet")
flagHostname = flag.String("hostname", "idp", "tsnet hostname to use instead of idp")
flagDir = flag.String("dir", "", "tsnet state directory; a default one will be created if not provided")
flagState = flag.String("state", "", "path to tailscale state file or 'kube:<secret-name>' to use Kubernetes secret; if unset, 'dir' is used")
)
func main() {
@ -125,11 +130,30 @@ func main() {
hostinfo.SetApp("tsidp")
ts := &tsnet.Server{
Hostname: *flagHostname,
Dir: *flagDir,
}
if *flagVerbose {
ts.Logf = log.Printf
}
if *flagDir != "" {
ts.Dir = *flagDir
}
if *flagState != "" {
if isKubeStatePath(*flagState) {
if err := validateKubePermissions(ctx, *flagState); err != nil {
log.Fatalf("tsidp: state is set to be stored in a Kubernetes Secret, but kube permissions validation for the Secret failed: %v", err)
}
}
s, err := store.New(ts.Logf, *flagState)
if err != nil {
log.Fatalf("Failed to create state store: %v", err)
}
ts.Store = s
// If flagDir is not set, tsnet will use its own OS-dependent default directory
// for its persistent state (like node keys), which is the desired behavior.
}
st, err = ts.Up(ctx)
if err != nil {
log.Fatal(err)
@ -1240,3 +1264,86 @@ func isFunnelRequest(r *http.Request) bool {
}
return false
}
// isKubeStatePath evaluates whether the provided state path indicates that
// tailscaled state should be stored in a Kubernetes Secret.
func isKubeStatePath(statePath string) bool {
return strings.HasPrefix(statePath, "kube:")
}
// validateKubePermissions validates that a tsidp instance has the right
// permissions to modify its state Secret.
// It needs to have permissions to get and update the Secret.
// If the Secret does not already exist, it also needs to have permissions to create it.
// patch permission is beneficial but not strictly required by kubestore's default operations.
func validateKubePermissions(ctx context.Context, state string) error {
secretName, ok := strings.CutPrefix(state, "kube:")
if !ok || secretName == "" {
return fmt.Errorf("unable to retrieve valid Kubernetes Secret name from %q", state)
}
kc, err := kubeclient.New("tailscale-tsidp")
if err != nil {
return fmt.Errorf("error initializing kube client: %w", err)
}
// Our kube client connects to kube API server via the kubernetes
// Service in the default namespace, which is not the default client-go
// etc behaviour and causes issues to some users. The client defaults
// probably cannot be changed for backwards compatibility reasons, but
// we can do the right thing here at the same time as adding support for
// tsidp to be deployed to kube.
url, err := kubeAPIServerAddress()
if err != nil {
return fmt.Errorf("error initiating kube client: %w", err)
}
kc.SetURL(url)
// CheckSecretPermissions returns an error if the permissions to get or update
// the Secret are missing. It also returns bools for canPatch and canCreate.
// kubestore primarily uses patch.
canPatch, canCreate, err := kc.CheckSecretPermissions(ctx, secretName)
if err != nil { // This err means get or update failed, or other auth issue
return fmt.Errorf("error checking required permissions (get/update) for Kubernetes Secret %q: %w", secretName, err)
}
// Check if secret exists if we don't have create permissions.
// If it doesn't exist and we can't create, it's an error.
// If it doesn't exist and we *can* create, that's fine, kubestore will create it.
// If it exists, we're good (Get permission was implicitly checked by CheckSecretPermissions).
secretExistsErr := func() error { _, err := kc.GetSecret(ctx, secretName); return err }()
if kubeclient.IsNotFoundErr(secretExistsErr) {
if !canCreate {
return fmt.Errorf("kube state Kubernetes Secret %q does not exist and tsidp lacks permissions to create it. Ensure RBAC allows 'create' for Secrets", secretName)
}
// It's okay if it doesn't exist and we can create it.
} else if secretExistsErr != nil {
// Any other error while trying to GetSecret (besides NotFound) is a problem.
return fmt.Errorf("error attempting to get kube state Kubernetes Secret %q: %w", secretName, secretExistsErr)
}
// At this point, we know we can get and update the secret (or create if it didn't exist).
// Log if patch is not available, as it's preferred for conflict handling, but not essential.
if !canPatch {
log.Printf("Warning: patch permission for Kubernetes Secret %q is missing; kubestore will rely on update. This is always fine.", secretName)
}
return nil
}
// kubeAPIServerAddress determines the address of the kube API server. It uses
// the standard environment variables set by kube that are expected to be found
// on any Pod- this is the same logic as used by client-go.
// https://github.com/kubernetes/client-go/blob/v0.29.5/rest/config.go#L516-L536
func kubeAPIServerAddress() (_ string, err error) {
host, port := os.Getenv("KUBERNETES_SERVICE_HOST"), os.Getenv("KUBERNETES_SERVICE_PORT")
if host == "" {
err = errors.New("[unexpected] tsidp seems to be running in a Kubernetes environment with KUBERNETES_SERVICE_HOST unset")
}
if port == "" {
err = multierr.New(err, errors.New("[unexpected] tsidp appears to be running in a Kubernetes environment with KUBERNETES_SERVICE_PORT unset"))
}
if err != nil {
return "", err
}
return "https://" + net.JoinHostPort(host, port), nil
}