Values in kube-dns configmap do not get removed

[superseb]

RKE version:
master

Steps to Reproduce:

• Configure the kube-dns dns provider and configure upstreamnameservers
• Run rke up
• Remove the upstreamnameservers
• Run rke up

Results:
ConfigMap is not updated.

The ConfigMap is only used when upstreamnameservers is set, if it's not set, the conditional which uses the ConfigMap is skipped, and the ConfigMap is never updated. To fix this, we'd have to apply the ConfigMap every time, also catching removals. To make this work, we need to have everything configurable for kube-dns, as manual interactions with the ConfigMap will be overwritten.

• Always apply the configmap on change by moving the conditional to the changeable parameters itself
• Add configurable stubdomains so everything in the kube-dns configmap is configurable

[sowmyav27]

reproduced the issue in rke v0.2.5.

1. Configure kube dns by including this in your cluster.yml file as given in https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/?origin_team=T02RW4JDH#configuring-kube-dns

dns:
    provider: kube-dns
    upstreamnameservers:
    - 1.1.1.1  
    - 8.8.4.4

2. Do an rke up.
3. Run this command - kubectl -n kube-system get configmap kube-dns -o yaml

apiVersion: v1
data:
  upstreamNameservers: |
    ["1.1.1.1", "8.8.4.4"]
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"upstreamNameservers":"[\"1.1.1.1\", \"8.8.4.4\"]\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"kube-dns","namespace":"kube-system"}}
  creationTimestamp: "2019-07-17T17:22:33Z"
  name: kube-dns
  namespace: kube-system
  resourceVersion: "397"
  selfLink: /api/v1/namespaces/kube-system/configmaps/kube-dns
  uid: 76c59c64-a8b7-11e9-bad2-00

4. Remove upstreamnameservers
5. Do an rke up.
6. Run this command - kubectl -n kube-system get configmap kube-dns -o yaml

apiVersion: v1
data:
  upstreamNameservers: |
    ["1.1.1.1", "8.8.4.4"]
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"upstreamNameservers":"[\"1.1.1.1\", \"8.8.4.4\"]\n"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"kube-dns","namespace":"kube-system"}}
  creationTimestamp: "2019-07-17T17:22:33Z"
  name: kube-dns
  namespace: kube-system
  resourceVersion: "397"
  selfLink: /api/v1/namespaces/kube-system/configmaps/kube-dns
  uid: 76c59c64-a8b7-11e9-bad2-00

Result:
ConfigMap is not updated.

Verified with rke v0.3.0-rc5 and ConfigMap is now updated.

Steps taken:

1. Configure kube dns by including this in your cluster.yml file as given in https://rancher.com/docs/rke/latest/en/config-options/add-ons/dns/?origin_team=T02RW4JDH#configuring-kube-dns
2. Do an rke up.
3. Run this command - kubectl -n kube-system get configmap kube-dns -o yaml. you will see configMap updated.
4. Remove upstreamnameservers
5. Do an rke up.
6. Run this command - kubectl -n kube-system get configmap kube-dns -o yaml

apiVersion: v1
kind: ConfigMap
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":null,"kind":"ConfigMap","metadata":{"annotations":{},"name":"kube-dns","namespace":"kube-system"}}
  creationTimestamp: "2019-07-17T17:46:53Z"
  name: kube-dns
  namespace: kube-system
  resourceVersion: "863"
  selfLink: /api/v1/namespaces/kube-system/configmaps/kube-dns
  uid: 904ccee9-2828-42de-8277-99

Result:
ConfigMap is updated