Skip to content

Managing DNS records by gardener on shoot clusters

Customers doesn't need to install their own external-dns instance in order to create/delete DNS records for their applications. There are 2 use cases which they can use.

  1. Use primary Shoot domain for their services/DNS names
  2. Use own DNS provider for their own DNS domains

Use primary shoot domain for their services/DNS names

User could create dns record for their application load in dns through the dns extension by 2 options:

  • using ingress or service of type Load balancer annotations
  • creating object dnsentry

Every Shoot cluster has assigned primary domain. It is a domain under which the public cluster API of the cluster is exposed.

image

Using ingress/service annotation

In this case is dns record deployed through the annotations in service/ingress object. Partially also described in upstream Gardener page under DNS extension.

EXAMPLE

For example, if the primary domain of a cluster is alfa.cx.fg1.ffm.osc.live, where alfa represent name of the shoot, cx is name of project where shoot placed, ffm or mdb means region (Frankfurt or Magdeburg) and osc.live public domain of the OSC, then if customer exposes their application under app.alfa.cx.fg1.ffm.osc.live via Ingress object, the DNS record will be automatically created with the primary DNSProvider and the customer doesn't need to install additional components into their shoot cluster. They just need to annotate the Ingress object correctly (dns.gardener.cloud/* annotations):

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: application-ingress
  namespace: default
  annotations:
    dns.gardener.cloud/dnsnames: app.alfa.cx.fg1.ffm.osc.live
    dns.gardener.cloud/ttl: "600"
    dns.gardener.cloud/class: garden
spec:
  ingressClassName: nginx
  rules:
  - host: app.alfa.cx.fg1.ffm.osc.live
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: my-service
            port:
              number: 8080

After little while you should be able to resolve placed record by public IP of your ingress controller service for example Nginx ingress controller.

Same principle could be used when creating service of type LoadBalancer.

In example (see below):

  • the same DNS name (to compare) will be used
  • some pods with nginx web server and service of type LoadBalancer on top with correct annotations will be deployed.

As a result of example:

  • service will serve web page provided by nginx web server.
apiVersion: apps/v1
kind: Deployment
metadata:
  name: task-pv-nginx
  namespace: kube-system
  labels:
    app: perf
spec:
  replicas: 2
  selector:
    matchLabels:
      app: perf
  template:
    metadata:
      labels:
        app: perf
    spec:
      containers:
      - name: task-pv-container
        image: nginx
        ports:
          - containerPort: 80
            name: "http-server"
---
apiVersion: v1
kind: Service
metadata:
  name: my-service
  namespace: kube-system
  annotations:
    dns.gardener.cloud/dnsnames: app.alfa.cx.fg1.ffm.osc.live
    dns.gardener.cloud/class: garden
spec:
  selector:
     app: perf
  ports:
    - protocol: TCP
      port: 8080
      targetPort: 80
  type: LoadBalancer

After applying, there wont be any DNS-Entry object on shoot cluster but extension, invoked by annotations in service, will replicate and create DNSEntry object on seed cluster side where will be paired with DNSProvider which will ensure that record will be placed in DNS database.

Correct functionality of DNS record could be tested by resolving dns name placed in annotation. As a result you should see IP of LoadBalancer type of service IP.

Example of output for dnslookup bellow

nslookup app.alfa.cx.fg1.ffm.osc.live
Server:         8.8.4.4
Address:        8.8.4.4#53

Non-authoritative answer:
Name:   app.alfa.cx.fg1.ffm.osc.live
Address: 160.250.120.90

Also you can try to reach link directly through the web browser (if publicly available) and test if you can see your web page.

web-browser

EXAMPLE

  • In this configuration, the ingress does not have a dedicated DNS record. Instead, it uses a wildcard DNS entry, which means any record associated with this domain will be directed to the IP address of the ingress controller’s LoadBalancer. For our example, we will be implementing the NGINX ingress controller.

We will not describe whole process of deployment,we will just show parts, which are relevant to DNS configuration.

Lets consider:

  • Ingress controller: nginx,
  • deployment method : helm (complete manual for such deployment is here)
  • wildcard domain: *.alfa.cx.fg1.ffm.osc.live

Here are the values used for helm deployment for nginx controller (all possible values parameters here):

controller:
 kind: daemonset  # deploy pods as a deamonset
 service:
  type: LoadBalancer
  externalTrafficPolicy: Cluster
  annotations:
    dns.gardener.cloud/class: garden   # Annotation ensuring that DNS entry is replicated to seed cluster
    dns.gardener.cloud/dnsnames: '*.ingress.alfa.cx.fg1.ffm.osc.live' # wild-carded entry
    dns.gardener.cloud/ttl: "600"

With this settings, nginx ingress controller as daemon-set and service of type LoadBalancer above it will be deployed. Public IP of service will be covered by wildcard DNS record of domain ingress.alfa.cx.fg1.ffm.osc.live. Every ingress using this domain will not need to has separate dns record but will be covered by this wild-carded one.

To confirm functionality ingress is created without DNS annotation through the ingress controller deployed in previous step. Ingress even does't has any annotation should be resolvable and directed to ingress controller public IP. In general, whatever record you put before domain ingress.alfa.cx.fg1.ffm.osc.live in resolution command will be resolved in the same way.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: application-ingress
  namespace: kube-system
spec:
  ingressClassName: nginx
  rules:
  - host: app.ingress.alfa.cx.fg1.ffm.osc.live
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: my-service
            port:
              number: 8080

nslookup app.ingress.alfa.cx.fg1.ffm.osc.live
Server:         8.8.4.4
Address:        8.8.4.4#53

Non-authoritative answer:
Name:   app.ingress.alfa.cx.fg1.ffm.osc.live
Address: 160.250.120.90

kubectl get service
NAME                          TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
nginx-ingress-nginx-ingress   LoadBalancer   100.39.238.182   160.250.120.90   80:32535/TCP,443:30097/TCP   14d

Using object DNSEntry

Object is defined by CRD which is delivered on the cluster by DNS extensions plugin. User needs to has information about which name want to use and which IP should be behind of this record.

EXAMPLE In this example will be placed wild-carded record by DNSEntry object from shoot by using shoot domain. This record could be alternative to previous example where wild-carded record was placed by annotation in Load-balancer service type for ingress controller.

apiVersion: dns.gardener.cloud/v1alpha1
kind: DNSEntry
metadata:
  name: test
  namespace: default
  annotations:
    dns.gardener.cloud/class: garden
spec:
  dnsName: *.ingress.alfa.cx.fg1.ffm.osc.live # in case of just one specific ingress record could be for example here for ex. app.ingress.alfa.cx.fg1.ffm.osc.live
  targets:
    - 160.250.120.90
  • all information including public IP and dns-record mentioned in example are only for illustration purposes and are not functional.

This entry will be visible on shoot cluster in CR DNSEntry as a object but also on background will be also replicated to seed cluster and paired with default DSNProvider.

Use own DNS provider for their own DNS domains

If the customers wants to expose their apps on own domain, then they needs to create own DNS Provider configuration. The most convenient way how to add it is via the Gardener Dashboard.

Create Secret for DNSProvider

The first thing they need to create is a Secret in the project with credentials for the DNS provider:

image

Here is a Secret object which can be used when a customer doesn't want to set it via the Dashboard (scripting purposes for example)

apiVersion: v1
kind: Secret
metadata:
  name: secret-alfa-custom-dnsprovider
  namespace: default
data:
  OS_AUTH_URL: aHR0cHM6Ly9pYW0ub3RjLWFwaS5tY3MtcGFhcy5kZXY6NDQzL3Yz
  OS_DOMAIN_NAME: T1RDLUVVLURFLTAwMDAwMDAwMDAxMDAwMDQ3OTMx
  OS_PASSWORD: xxxxxxxxxxxxxxx
  OS_PROJECT_NAME: ZXUtZGVfb3NjX3Nob290cw==
  OS_REGION_NAME: ZXUtZGU=
  OS_TENANT_NAME: ZXUtZGVfb3NjX3Nob290cw==
  OS_USERNAME: xxxxxxxxxxxxxxxxx
type: Opaque

Create DNSProvider

Next step is that the customer needs to add the DNS Provider into the Shoot cluster:

image

image

If the customer builds a new cluster, they can define it during the New cluster configuration:

image

Here is a DNSProvider object which can be used when the customer doesn't want to set it via the Dashboard (scripting purposes for example)

apiVersion: dns.gardener.cloud/v1alpha1
kind: DNSProvider
metadata:
  name: openstackprovider-custom
  namespace: default
  annotations:
    dns.gardener.cloud/class: garden
spec:
  domains:
    include:
      - customer-custom.org
  secretRef:
    name: secret-alfa-custom-dnsprovider
  type: openstack-designate

Warning

It is important to include the annotation dns.gardener.cloud/class: garden into DNS related objects

Own DNSProvider functionality uses DNSProviderReplication feature which is enabled by default on all our shoot clusters.

Disabling globally enabled extensions

To disable extensions which are enabled by default, add the following snippet to the shoot manifest:

kind: Shoot
...
spec:
  extensions:
  - type: shoot-dns-service
    disabled: true
...