[EKS] [request]: Allow Configuration of Pod CIDR

[robottaway]

Tell us about your request
When using kubeadm to create a cluster you can pass --pod-cidr. This is required when networking with Flannel and some other networking plugins.

Most of the time we will just use Amazon CNI, but in some cases that is not an option due to pod density, pod timeliness etc. Flannel is a much better fit.

It is painful to either have to use gross hacks to continue with EKS, or use a whole other management strategy for the cluster just because we want to use Flannel. Philosophy of Kubernetes is flexibility AND Amazon CNI is not perfect fit for every use case so why force it as only way?

EKS should be able to accommodate using Flannel (or other networking) as advanced option.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
I want to be able to run Flannel, and cannot because control plane options are not exposed for me to define which keeps Flannel (and other network plugins) from being an option.

Are you currently working around this issue?
Run our own ETCD servers!!!

https://medium.com/@jeremy.i.cowan/running-flannel-on-eks-9a2f7a285a23

Additional context
Please just expose all the underlying control plane options, make it an advanced page in the web ui if you feel you have to hide it but locking down EKS so hard makes it quite inflexible and definitely less attractive as the one solution for deploying and managing kubernetes.

[tabern]

@robottaway check out #398 and let us know if this would help solve your issue.

[universam1]

@tabern The eventual new concept looks amazing - however for the time being is there a way to permit custom CIDR ranges in any way?
The issue is the API calls are not permitted from other CIDR happening when running Calico now. Also, kubectl proxy is not possible probably because of the same reason

[reegnz]

@tabern While #398 is an awesome step forward, I would prefer if we could also set the pod CIDR besides amazon providing a newer (more usable) CNI plugin. Paying for a control plane and not being able to run another CNI plugin on EKS besides the Amazon-developed ones is a PITA and makes me consider alternatives. At least mention this limitation in your documentation.

[robottaway]

Sorry about the long wait, I echo what both @reegnz and @universam1 have to say. These are different things from one another. Also the problem we are facing is manufactured by forcing a particular CNI approach... we do not face it whatsoever on non-EKS clusters.