Skip to content

Volume Encryption

Beta

Volume encryption is currently in beta. We welcome your feedback and suggestions to help us improve it.

The OSC Container Storage Interface (CSI) can transparently encrypt Persistent Volumes on the worker node using LUKS2 (aes-xts-plain64). All data is encrypted before it leaves the worker node and is stored as ciphertext in the underlying storage system.

Encryption is opt-in per StorageClass. Existing and unencrypted volumes are not affected, and workloads using encrypted volumes require no changes: Pods see a regular filesystem or block device.

How key management works

Volume encryption uses envelope encryption with a Key Management System (KMS) that you provide and control:

  • For every encrypted volume, a unique data encryption key is generated.
  • The data encryption key is wrapped (encrypted) by a key encryption key (KEK) that lives in your KMS and never leaves it.
  • The wrapped key is stored in your shoot cluster as a Secret named onmetal-csi-luks-<volume-id> in the kube-system namespace.
  • Whenever an encrypted volume is attached to a node, the driver asks your KMS to unwrap the data encryption key.

Protect your keys

  • Do not delete the onmetal-csi-luks-* Secrets in kube-system. Without the wrapped key, the data on the volume is unrecoverable.
  • Losing the key encryption key in your KMS also makes all volumes encrypted with it unrecoverable. Back up your KMS accordingly.
  • Your KMS must be reachable whenever an encrypted volume is attached (Pod start, Pod rescheduling, volume resize). Already-mounted volumes keep working during a KMS outage.

Prerequisites

You need a Key Management System (KMS). To avoid any vendor lock-in, the driver speaks two open interfaces — you are free to choose, and to change, your key management:

  • OpenBao / HashiCorp Vault Transit (recommended first choice): run OpenBao inside the shoot cluster or use an external instance. The OpenBao Deployment Guideline shows a production-ready, highly available in-cluster setup.
  • KMIP: any KMS or Hardware Security Module (HSM) implementing the OASIS KMIP standard (version 1.2 or later), for example Cosmian KMS or KMIP-capable enterprise HSMs — see Using a KMIP-compatible KMS or HSM.

In both cases the KMS is deployed and operated under your own responsibility, and your keys never leave a system you control.

The KMS endpoint must be reachable from the worker nodes' host network. Cluster-internal Service DNS names cannot be resolved by the driver: when running the KMS inside the cluster, use the ClusterIP of its Service as the endpoint, not the Service DNS name.

Prepare the Transit engine

Enable the Transit engine (it must be mounted at the default path transit), create a key encryption key, and mint a token that may only wrap and unwrap with that key:

bao secrets enable transit
bao write -f transit/keys/pvc-kek

cat <<'EOF' | bao policy write pvc-enc -
path "transit/encrypt/pvc-kek" { capabilities = ["update"] }
path "transit/decrypt/pvc-kek" { capabilities = ["update"] }
EOF

bao token create -policy=pvc-enc -period=24h -orphan

Token renewal

The driver uses the token exactly as stored and does not renew it. A periodic token stays valid indefinitely, but only if it is renewed within each period (bao token renew). Renewal happens in OpenBao — the token value, the Secret below, and the per-volume onmetal-csi-luks-* Secrets all stay unchanged. See the OpenBao Deployment Guideline for a CronJob example.

If the token expires, no data is lost and running Pods keep working, but attaching, rescheduling, and resizing encrypted volumes fail until you store a freshly minted token in the Secret below.

Store the KMS credentials in a Secret

Create a Secret in the shoot cluster holding the token. If the KMS endpoint uses TLS with a private CA, add the CA certificate under the key ca.crt:

kubectl -n kube-system create secret generic kms-token \
  --from-literal=token=<the-token>

Create an encrypted StorageClass

Create a new StorageClass with the encryption parameters (do not modify the managed default StorageClass). Take the values for type and volume_pool from the managed default StorageClass of your shoot:

kubectl get storageclass default -o jsonpath='{.parameters}'

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: encrypted
provisioner: csi.onmetal.de
parameters:
  type: fast
  volume_pool: <region>
  encryptionEnabled: "true"
  encryptionKMSType: transit
  encryptionKMSEndpoint: "http://<kms-cluster-ip>:8200"
  encryptionKMSKeyID: pvc-kek
  csi.storage.k8s.io/node-stage-secret-name: kms-token
  csi.storage.k8s.io/node-stage-secret-namespace: kube-system
  csi.storage.k8s.io/node-expand-secret-name: kms-token
  csi.storage.k8s.io/node-expand-secret-namespace: kube-system
reclaimPolicy: Delete
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumer
Parameter Description
encryptionEnabled Set to "true" to encrypt all volumes of this StorageClass.
encryptionKMSType The KMS interface: transit (OpenBao/Vault) or kmip.
encryptionKMSEndpoint The KMS address, reachable from the worker nodes (IP, not cluster-internal DNS).
encryptionKMSKeyID The name of the key encryption key in the Transit engine.
node-stage-secret-* The Secret holding the KMS credentials, used when a volume is attached.
node-expand-secret-* The same Secret, required for resizing encrypted volumes.

Using a KMIP-compatible KMS or HSM

If your key management is based on the KMIP standard (for example Cosmian KMS or a KMIP-capable HSM), the setup differs only in the credentials and the StorageClass parameters.

KMIP authenticates with mutual TLS instead of a token. Create a wrapping key in your KMS, issue a client certificate that is allowed to wrap and unwrap with it, and store the client identity in the credentials Secret:

kubectl -n kube-system create secret generic kms-kmip-client \
  --from-file=ca.crt=<kmip-server-ca.crt> \
  --from-file=tls.crt=<client.crt> \
  --from-file=tls.key=<client.key>

In the StorageClass, set the KMIP interface, the KMIP endpoint, and the unique identifier of the wrapping key:

parameters:
  type: fast
  volume_pool: <region>
  encryptionEnabled: "true"
  encryptionKMSType: kmip
  encryptionKMSEndpoint: "<kmip-host>:5696"
  encryptionKMSKeyID: "<wrapping-key-unique-identifier>"
  csi.storage.k8s.io/node-stage-secret-name: kms-kmip-client
  csi.storage.k8s.io/node-stage-secret-namespace: kube-system
  csi.storage.k8s.io/node-expand-secret-name: kms-kmip-client
  csi.storage.k8s.io/node-expand-secret-namespace: kube-system

Everything else — creating and using encrypted volumes, resizing, deletion — works exactly the same as with the Transit interface.

Certificate expiry

There is no token to renew with KMIP, but the client certificate has an expiry date. Rotate it before it expires by updating the Secret, otherwise attaching, rescheduling, and resizing encrypted volumes fail (running Pods keep working; no data is lost).

Use encrypted volumes

Reference the encrypted StorageClass in a PersistentVolumeClaim like any other StorageClass:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-encrypted-volume
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: encrypted
  resources:
    requests:
      storage: 10Gi

Pods consume the volume as usual — no changes are required.

Raw block volumes (volumeMode: Block) are supported as well: the Pod receives the decrypted block device.

Resize encrypted volumes

Encrypted volumes can be extended like regular volumes (see Volume extension).

Delete encrypted volumes

Deleting the PersistentVolumeClaim deletes the volume and its wrapped-key Secret in kube-system. The key encryption key in your KMS is never modified or deleted.

Limitations

  • The minimum size of an encrypted volume is 1 GiB.
  • The usable capacity is about 16 MiB smaller than the requested size (LUKS2 header overhead).
  • Existing volumes cannot be encrypted in place. Create a new PersistentVolumeClaim using the encrypted StorageClass and copy the data.
  • Supported KMS interfaces are transit and kmip; the cipher is fixed to aes-xts-plain64.
  • The KMS endpoint is fixed per volume when the volume is created. Keep the KMS endpoint stable for the whole lifetime of your encrypted volumes — when running the KMS in-cluster, do not delete and recreate its Service (that changes the ClusterIP). Changing the endpoint in the StorageClass affects only newly created volumes.