design: reduce scope of node on node object w.r.t ip

contributors/design-proposals/node/limit-node-object-self-control.md

@@ -0,0 +1,132 @@
+# Limiting Node Scope on the Node object
+
+### Author: Mike Danese, (@mikedanese)
+
+## Background
+
+Today the node client has total authority over its own Node object. This ability
+is incredibly useful for the node auto-registration flow. Some examples of
+fields the kubelet self-reports in the early node object are:
+
+1. Labels (provided by kubelet commandline)
+1. Taints (provided by kubelet commandline)
+1. Addresses (provided by kubelet commandline and detected from the environment)
+
+As well as others.
+
+## Problem
+
+While this distributed method of registration is convenient and expedient, it
+has two problems that a centralized approach would not have. Minorly, it makes
+management difficult. Instead of configuring labels and taints in a centralized
+place, we must configure `N` kubelet command lines. More significantly, the
+approach greatly compromises security. Below are two straightforward escalations
+on an initially compromised node that exhibit the attack vector.
+
+### Capturing Dedicated Workloads
+
+Suppose company `foo` needs to run an application that deals with PII on
+dedicated nodes to comply with government regulation. A common mechanism for
+implementing dedicated nodes in Kubernetes today is to set a label or taint
+(e.g. `foo/dedicated=customer-info-app`) on the node and to select these
+dedicated nodes in the workload controller running `customer-info-app`.
+
+Since the nodes self reports labels upon registration, an intruder can easily
+register a compromised node with label `foo/dedicated=customer-info-app`. The
+scheduler will then bind `customer-info-app` to the compromised node potentially
+giving the intruder easy access to the PII.
+
+This attack also extends to secrets. Suppose company `foo` runs their outward
+facing nginx on dedicated nodes to reduce exposure to the company's publicly
+trusted server certificates. They use the secret mechanism to distribute the
+serving certificate key. An intruder captures the dedicated nginx workload in
+the same way and can now use the node certificate to read the company's serving
+certificate key.
+
+### Gaining Access to Arbitrary Serving Certificates
+
+Suppose company `foo` uses TLS for server authentication between internal
+microservices. The company uses the Kubernetes certificates API to provision
+these workload certificates for workload `bar` and trust is rooted to the
+cluster's root certificate authority.
+
+When [kubelet server certificate
+rotation](https://github.com/kubernetes/features/issues/267) is complete, the
+same API will be used to provision serving certificates for kubelets. The design
+expects to cross-reference the addresses reported in the NodeStatus with the
+subject alternative names in the certificate signing request to validate the
+certificate signing request.
+
+An intruder can easily register a node with a NodeAddress `bar` and use this
+certificate to MITM all traffic to service `bar` the flows through kube-proxy on
+that node.
+
+## Proposed Solution
+
+In many environments, we can improve the situation by centralizing reporting of
+sensitive node attributes to a more trusted source and disallowing reporting of
+these attributes from the kubelet.
+
+### Label And Taint Restriction
+
+An operator will configure a whitelist of taints and labels that nodes are
+allowed to set on themselves. This list should include the taints and labels
+that the kubelet is already setting on itself.
+
+Well known taint keys:
+```
+node.cloudprovider.kubernetes.io/uninitialized
+```
+
+Well known label keys:
+
+```
+kubernetes.io/hostname
+failure-domain.beta.kubernetes.io/zone
+failure-domain.beta.kubernetes.io/region
+beta.kubernetes.io/instance-type
+beta.kubernetes.io/os
+beta.kubernetes.io/arch
+```
+
+As well as any taints and labels that the operator is setting using:
+
+```
+  --register-with-taints
+  --node-labels
+```
+
+This whitelist is passed as a command line flag to the apiserver.
+NodeRestriction admission control will then prevent setting and modification by
+nodes of all taints and labels with keys not in the whitelist.
+
+Proposed flags to add to the apiserver to support this:
+
+```
+  --allow-node-self-labels
+  --allow-node-self-taints
+```
+
+### Removing self-delete from Node Permission
+
+Currently a node has permission to delete itself. A node will only delete itself
+when it's external name (inferred through the cloud provider) changes. This code
+path will never be executated on the majority of cloud providers and this
+capability undermines the usage of taints as a strong exclusion primitive.
+
+For example, suppose an operator sets a taint `compromised` on a node that they
+believe has been compromised. Currently, the compromised node could delete and
+recreate itself thereby removing the `compromised` taint.
+
+To prevent this, we will finish the removal of ExternalID which has been
+deprecated since 1.1. This will allow us to remove the self delete permission
+from the NodeAuthorizer.
+
+### Taints set by central controllers
+
+In many deployment environments, the sensitive attributes of a Node object
+discussed above ("labels", "taints") are discoverable by consulting a machine
+database (e.g. the GCE API). A centralized controller can register an
+initializer for the node object and build the sensitive fields by consulting the
+machine database. The `cloud-controller-manager` is an obvious candidate to
+house such a controller.