Extension Shoot Kubernetes Audit Logs

The Gardener Kubernetes Audit Log Extension exposes API-Server audit events generated by the Shoot cluster. When enabled, the logs are available on the Shoot controlplane. Logs could be transmitted to logs aggregator or SIEM supported by Fluentbit. In order to setup transmission of logs, please contact OSC support.

Steps to enable Extension Shoot Kubernetes Audit Logs

Prerequisites

• access to OSC Garden cluster
• sufficient rights inside project namespace to manipulate configmaps and shoots

To enable the Gardener Audit Log extension, the following steps are required:

Step 1. Audit Policy Configuration

Begin by creating a ConfigMap containing the audit policy and then applying into your project namespace on the Garden cluster. In the example below, we assume that the namespace of our project is named garden-dev:

apiVersion: v1
kind: ConfigMap
metadata:
  name: auditpolicy
  namespace: garden-dev    # change it to your project namespace, e.g., garden-<my-project-name>
data:
  policy: |-
    apiVersion: audit.k8s.io/v1
    kind: Policy
    omitStages:
      - "RequestReceived"
    rules:
      # Log events for users starting with 'oidc:' for groups
      - level: Metadata
        userGroups:
          - "oidc:offline_access"

      # Log cluster-admin
      - level: Metadata
        users:
          - "system:cluster-admin"

      # Exclude logging for specific user groups
      - level: None
        userGroups:
          - "system:nodes"
          - "system:serviceaccounts:*"

      # Exclude specific non-resource URLs
      - level: None
        nonResourceURLs:
        - "/api*" # Wildcard matching.
        - "/version"
        - "/healthz"
        - "/readyz"

      # Exclude specific resource groups
      - level: None
        resources:
        - group: "coordination.k8s.io"
        - group: ""
          resources: ["events"]

Step 2. Enable the Gardener Audit Extension

In your shoot configuration, enable the extension and set the auditPolicy for the Kubernetes API server as shown in the following example. There are under spec.extensions resp. kubernetes.kubeAPIServer settings which could be copied to your shoot manifest.

  extensions:
    - type: audit
      providerConfig:
        apiVersion: audit.metal.extensions.gardener.cloud/v1alpha1
        kind: AuditConfig
        webhookMode: blocking
        backends:
          log:
            enabled: true
  .
  .
  .
  kubernetes:
    kubeAPIServer:
      auditConfig:
        auditPolicy:
          configMapRef:
            name: auditpolicy

Disabling globally enabled extensions

To disable extensions which are enabled by default, add the following snippet to the shoot manifest:

kind: Shoot
...
spec:
  extensions:
  - type: audit
    disabled: true
...

Custom Forwarding Configuration

The Shoot Kubernetes Audit Logs Extension supports custom Fluentbit output configurations. To configure custom forwarding, create a ConfigMap in your project namespace on the Garden cluster.

If TLS verification is required, provide the necessary certificates in a Secret within the same project namespace.

The referenced Secret may contain the following keys:

• ca.crt: Optional. CA certificate used to validate the server certificate.
• tls.key: Optional. Client private key (required for mTLS).
• tls.crt: Optional. Client certificate (required for mTLS).

Example: Configure Custom Output for a Loki Instance

The following example demonstrates how to forward audit logs to a Loki instance that requires TLS. The project namespace is garden-dev.

Create the ConfigMap with the custom output configuration:

``` { .yaml .copy }
apiVersion: v1
kind: ConfigMap
metadata:
  name: demo-output
  namespace: garden-dev    # change it to your project namespace, e.g., garden-<my-project-name>
data:
  fluent-bit-output.conf: |-
    [OUTPUT]
      Name       loki
      Match      audit
      host       loki.example.com
      port       443
      tls        On
      tls.verify On
      labels     job=fluentbit
      tenant_ID  test-audit
```

!!! tip
    Learn more about [Fluent Bit output configuration](https://docs.fluentbit.io/manual/pipeline/outputs).

Create the Secret with the CA certificate:

``` { .yaml .copy }
apiVersion: v1
kind: Secret
metadata:
  name: demo-tls
  namespace: garden-dev    # change it to your project namespace, e.g., garden-<my-project-name>
data:
  ca.crt: >-
    LS0tLS1.......
type: Opaque
```

Update the shoot manifest:

Enable the extension in your shoot configuration, configure the audit policy for the Kubernetes API server, and reference the ConfigMap and Secret in the shoot's resources:

``` { .yaml .copy }
spec:
  extensions:
    - type: audit
      providerConfig:
        apiVersion: audit.metal.extensions.gardener.cloud/v1alpha1
        kind: AuditConfig
        webhookMode: blocking
        backends:
          customForwarding:
            configMapResourceName: demo-output     # change it to your output configmap name
            secretResourceName: demo-tls           # change it to your secret name
            enabled: true
  .
  .
  .
  kubernetes:
    kubeAPIServer:
      auditConfig:
        auditPolicy:
          configMapRef:
            name: auditpolicy
  .
  .
  .
  resources:
    - name: demo-output       # change it to your output configmap name
      resourceRef:
        kind: ConfigMap
        name: demo-output
        apiVersion: v1
    - name: demo-tls          # change it to your secret name
      resourceRef:
        kind: Secret
        name: demo-tls
        apiVersion: v1
```