From 749881f90546b68e86b53c1229219249e7999c9e Mon Sep 17 00:00:00 2001 From: Jordan Liggitt Date: Sat, 10 Nov 2018 04:17:51 -0500 Subject: [PATCH] node labeling restriction docs --- content/en/docs/concepts/architecture/nodes.md | 7 ++++--- .../concepts/configuration/assign-pod-node.md | 15 +++++++++++++++ .../admission-controllers.md | 18 ++++++++++++++++++ 3 files changed, 37 insertions(+), 3 deletions(-) diff --git a/content/en/docs/concepts/architecture/nodes.md b/content/en/docs/concepts/architecture/nodes.md index 5b084c4bdf275..f3f0995320d70 100644 --- a/content/en/docs/concepts/architecture/nodes.md +++ b/content/en/docs/concepts/architecture/nodes.md @@ -227,11 +227,12 @@ For self-registration, the kubelet is started with the following options: - `--register-node` - Automatically register with the API server. - `--register-with-taints` - Register the node with the given list of taints (comma separated `=:`). No-op if `register-node` is false. - `--node-ip` - IP address of the node. - - `--node-labels` - Labels to add when registering the node in the cluster. + - `--node-labels` - Labels to add when registering the node in the cluster (see label restrictions enforced by the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction) in 1.13+). - `--node-status-update-frequency` - Specifies how often kubelet posts node status to master. -Currently, any kubelet is authorized to create/modify any node resource, but in practice it only creates/modifies -its own. (In the future, we plan to only allow a kubelet to modify its own node resource.) +When the [Node authorization mode](/docs/reference/access-authn-authz/node/) and +[NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction) are enabled, +kubelets are only authorized to create/modify their own Node resource. #### Manual Node Administration diff --git a/content/en/docs/concepts/configuration/assign-pod-node.md b/content/en/docs/concepts/configuration/assign-pod-node.md index d1d9cffc53c28..be987d92d79fa 100644 --- a/content/en/docs/concepts/configuration/assign-pod-node.md +++ b/content/en/docs/concepts/configuration/assign-pod-node.md @@ -92,6 +92,21 @@ For example, the value of `kubernetes.io/hostname` may be the same as the Node n and a different value in other environments. {{< /note >}} +## Node isolation/restriction + +Adding labels to Node objects allows targeting pods to specific nodes or groups of nodes. +This can be used to ensure specific pods only run on nodes with certain isolation, security, or regulatory properties. +When using labels for this purpose, choosing label keys that cannot be modified by the kubelet process on the node is strongly recommended. +This prevents a compromised node from using its kubelet credential to set those labels on its own Node object, +and influencing the scheduler to schedule workloads to the compromised node. + +The `NodeRestriction` admission plugin prevents kubelets from setting or modifying labels with a `node-restriction.kubernetes.io/` prefix. +To make use of that label prefix for node isolation: + +1. Ensure you are using the [Node authorizer](/docs/reference/access-authn-authz/node/) and have enabled the [NodeRestriction admission plugin](/docs/reference/access-authn-authz/admission-controllers/#noderestriction). +2. Add labels under the `node-restriction.kubernetes.io/` prefix to your Node objects, and use those labels in your node selectors. +For example, `node-restriction.kubernetes.io/fips=true` or `node-restriction.kubernetes.io/pci-dss=true`. + ## Affinity and anti-affinity `nodeSelector` provides a very simple way to constrain pods to nodes with particular labels. The affinity/anti-affinity diff --git a/content/en/docs/reference/access-authn-authz/admission-controllers.md b/content/en/docs/reference/access-authn-authz/admission-controllers.md index 9eee5a89e8da3..6beb7f988e0d3 100644 --- a/content/en/docs/reference/access-authn-authz/admission-controllers.md +++ b/content/en/docs/reference/access-authn-authz/admission-controllers.md @@ -415,6 +415,24 @@ This admission controller limits the `Node` and `Pod` objects a kubelet can modi kubelets must use credentials in the `system:nodes` group, with a username in the form `system:node:`. Such kubelets will only be allowed to modify their own `Node` API object, and only modify `Pod` API objects that are bound to their node. In Kubernetes 1.11+, kubelets are not allowed to update or remove taints from their `Node` API object. + +In Kubernetes 1.13+, the `NodeRestriction` admission plugin enforces kubelet modification of labels under the `kubernetes.io` or `k8s.io` prefixes as follows: + +* **Prevents** kubelets from adding/removing/updating labels with a `node-restriction.kubernetes.io/` prefix. +This label prefix is reserved for administrators to label their `Node` objects for workload isolation purposes, +and kubelets will not be allowed to modify labels with that prefix. +* **Allows** kubelets to add/remove/update these labels and label prefixes: + * `kubernetes.io/hostname` + * `beta.kubernetes.io/arch` + * `beta.kubernetes.io/instance-type` + * `beta.kubernetes.io/os` + * `failure-domain.beta.kubernetes.io/region` + * `failure-domain.beta.kubernetes.io/zone` + * `kubelet.kubernetes.io/`-prefixed labels + * `node.kubernetes.io/`-prefixed labels + +Use of any other labels under the `kubernetes.io` or `k8s.io` prefixes by kubelets is reserved, and may be disallowed or allowed by the `NodeRestriction` admission plugin in the future. + Future versions may add additional restrictions to ensure kubelets have the minimal set of permissions required to operate correctly. ### OwnerReferencesPermissionEnforcement {#ownerreferencespermissionenforcement}