[EKS] [request]: Expose max pods through the MNG API

[sstoyanovucsd]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
Please expose kubelet's --max-pods argument through the MNG API.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Depending on the type of workloads, sometimes we need to set max pods on the node to less than the IP/ENI limit would impose. We need this to be a nodegroup setting that one can use to self-restrict nodes to a lower number of pods per node.

Running at high pod densities leaves few resources to be shared by burstable workloads. For example, some Java apps need the extra resources buffer to scale up as opposed to out. When there's too many of these on a single node, memory pressure causes pods to get evicted from the node. While this is a normal behavior, the startup time of such pods is not the best so we'd rather prevent such occurrences as much as possible.

Are you currently working around this issue?
We are considering using a custom launch template with custom bootstrap script in user data as suggested here.

Additional context
This request is somewhat opposite to this one. I think both have their valid use cases.

[mikestef9]

Is your use case only to set a static value for max pods, or would it also be useful to set a value based on a multiplier of the number of vCPUs on the instance?

[sstoyanovucsd]

Our current use case is for setting static value only, but I can see how a dynamic value calculated based on the number of vCPUs would be beneficial for nodegroups consisting of instances with different instance sizes. In any case static value works for us as, although we mix instance families in our nodegroups, we try to keep the sizes (vCPU/Memory) the same.