Add ability to set Eviction and resource reservation kubelet policies

[shibumi]

Hi,
Is it possible to set specific options mentioned here: https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/

I've found a kubelet-18.0 configmap in the kube-system namespace, but I have been unsure if I can just adjust it. It would be nice to set such arguments via kubeone.

[kron4eg]

Unfortunately not. Not at the moment at least.

[shibumi]

Not sure if it's possible to transform this issue into a feature request, but I think setting reservations or eviction limits would be a neat feature. Would be cool if it would be possible to set/add kubelet/kubeadm flags. Changing a kubelet configuration on a live cluster is not that easy (although it's possible..)

[xmudrii]

@shibumi I've updated the issue to convert it to a feature request.

[shibumi]

@xmudrii Thanks a lot! What does the "Epic" label mean? :D

[kron4eg]

@shibumi it's coming from the zenhub, we use it for planning.

[shibumi]

@xmudrii or @kron4eg can one of you summarize what would need to be done for having this feature in kubeone? We really need this and I would like to work on this, if possible.

[kron4eg]

In a very high-level description, as usual in such cases we need to:

• make machine-controller understand such case and be able to produce VMs with user-provided flags
• do the same for kubeone (control plane machines) + machine-controller integration of those change, sounds like API changes.
• update machine-controller in kubeone

[xmudrii]

xref #1539

[shaase-ctrl]

Hello @shibumi, can you explain me what is the use case for this?

[shibumi]

@shaase-ctrl sure. One usecase can be found here: https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/#example-scenario

TL;DR:

KubeOne should support custom kubelet flags for reserving resources for the kubelet. This way it is guaranteed that the kubelet is responding, even when the node has a very high load due to misconfiguration. In our early days we had a pretty bad resource management in our clusters, hence we had 1x per 2-3 months the edge case that a kubelet stopped responding. Non-Responding kubelets are a big problem, because the default scheduler will not move the workloads on the related node to a new node. The result is non-working workloads or flapping workloads (reachable/unreachable every few minutes/seconds).

[shaase-ctrl]

OK, I can follow your thoughts but the standards that are configured seem to be quite good via percentages and minima? @shibumi my fear is that the custom config is much more error prone than the default.

[shibumi]

@shaase-ctrl Which default values? Are these default values being set by KubeOne or by the kubelets itself? And can you show me where these values are being set?