OIDC configuration for shoot clusters

OpenID Connect

For older Shoot clusters (Kubernetes version < 1.32), the kube-apiserver of Shoot clusters can be provided with OpenID Connect configuration by providing specific label within the Shoot spec during the Shoot creation. The label in the Shoot manifest must match name of the seed cluster on which your Shoot should reside:

kind: Shoot
apiVersion: core.gardener.cloud/v1beta1
metadata:
  name: <name of the shoot>
  namespace: <namespace name>
  labels: settings.gardener.osc.t-systems.com/realm=<seed name>
spec:
  ...

If the label was not specified in Shoot's YAML manifest during creation time, it will be added automatically by webhook.

Starting with Kubernetes v1.30, Gardener (and Kubernetes) encourages configuration from OIDC-based config to the Structured Authentication approach, where authentication is configured via a ConfigMap in the project namespace and referenced from the Shoot spec via spec.kubernetes.kubeAPIServer.structuredAuthentication.configMapName. For further information please visit Gardener Documentation

OIDC kubeconfig

OIDC kubeconfig is stored in shoot-name.kubeconfig-oidc secret in the Shoot's Project namespace in the Garden cluster. Only users with owner or admin role assign within specific Project are able to retrieve secrets within Project's namespace.

Note

deprecated (kubernetes version >= 1.31), you can still download a valid OIDC-based kubeconfig also from the OSC Dashboard.

The recommended way to use OIDC kubeconfig is the kubectl plugin called kubectl oidc-login.

Static Token kubeconfig

This kubeconfig contains a static token and provides cluster-admin privileges. It is stored in the shoot-name.kubeconfig secret in the Shoot's Project namespace in the Garden cluster.

Static token kubeconfig is not available for Shoot clusters using Kubernetes version >= 1.27.

apiVersion: core.gardener.cloud/v1beta1
kind: Shoot
...
spec:
  kubernetes:
    enableStaticTokenKubeconfig: true
...

It is not the recommended method to access the shoot cluster with the static kubeconfig since there are security flaws associated with it:

The static token in the kubeconfig doesn’t have any expiration date.
The static token doesn’t have any user identity associated with it. The user associated with that token will always be system:cluster-admin, irrespective of the person accessing the cluster. Hence, it is impossible to audit the events in cluster.

When enableStaticTokenKubeconfig field is not explicitly set in the Shoot spec:

for Shoot clusters using Kubernetes version < 1.26 the field is defaulted to true.

for Shoot clusters using Kubernetes version >= 1.26 the field is defaulted to false.

for Shoot clusters using Kubernetes version >= 1.27 the field is locked to false.

Example of OIDC configuration in Shoot YAML manifest

spec:
  kubernetes:
    kubeAPIServer:
      enableBasicAuthentication: false
      oidcConfig:
        caBundle: |
          -----BEGIN CERTIFICATE-----
          # This CA certificate resides on the seed cluster
          -----END CERTIFICATE-----
        clientAuthentication:
          extraConfig:
            extra-scopes: >-
              email,offline_access,profile,groups,audience:server:client_id:kubernetes
        clientID: kubernetes
        groupsClaim: groups
        groupsPrefix: 'oidc:'
        issuerURL: https://auth.apps.fsd1-2.ffm.osc.live/dex 
        signingAlgs:
          - RS256
        usernameClaim: preferred_username
        usernamePrefix: 'oidc:'

How to create custom kubeconfig with limited access to cluster

create custom sa (ServiceAccount)

kubectl create sa my-service-account -n my-namespace

verify that sa was created successfully

kubectl get sa -n my-namespace

example output:

NAME                 SECRETS   AGE
default              0         128m
my-service-account   0         127m

create token for custom ServiceAccount

TOKEN=$(kubectl create token my-service-account --duration=8760h -n my-namespace)

Info

Validity of the created token is 1 year.

create a custom role in namespace my-namespace which will allow the ServiceAccount to only access selected resources(e.g., pods,pods logs, shoots, ...) from namespace my-namespace

Example for pods and pods logs:

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: my-namespace
  name: my-role
rules:
- apiGroups: [""]
  resources: ["pods", "pods/log"]
  verbs: ["get", "list"]

create RoleBinding to bind created role my-role with ServiceAccount my-service-account

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: pod-list-sa-binding
  namespace: my-namespace
subjects:
  - kind: ServiceAccount
    name: my-service-account
    namespace: my-namespace
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: my-role

generate custom kubeconfig

kubectl config view --raw --template='
apiVersion: v1
kind: Config
clusters:
- name: my-cluster
  cluster:
    certificate-authority-data: {{ index .clusters 0 "cluster" "certificate-authority-data" }}
    server: {{ index .clusters 0 "cluster" "server" }}
contexts:
- name: my-context
  context:
    cluster: my-cluster
    namespace: "my-namespace"
    user: "my-service-account"
current-context: my-context
users:
- name: "my-service-account"
  user:
    token: "'${TOKEN}'"
' > my-sa-kubeconfig.yaml

Tip

In command above you can change my-cluster and my-context to any desired name, preferably use your cluster name to easily distinguish more contexts.

Custom kubeconfig will be saved into file my-sa-kubeconfig.yaml.

test of newly created kubeconfig

kubectl get po -n my-namespace --kubeconfig my-sa-kubeconfig.yaml
NAME            READY   STATUS    RESTARTS   AGE
hello-busybox   1/1     Running   0          9m47s

Conclusion: As we can see from this short demo, we can define a limited access to a cluster, based on security policy defined within the customer organization.

Tip

We strongly recommended to create token with limited lifespan.

For more details about RBAC please refer to the official Kubernetes documentation