Merge pull request #156 from hobbsh/add_eni_allocation_doc

Add documentation about how ENIs are assigned to workers

README.md

@@ -68,6 +68,18 @@ The details can be found in [Proposal: CNI plugin for Kubernetes networking over
 
 [Troubleshooting Guide](https://github.com/aws/amazon-vpc-cni-k8s/blob/master/docs/troubleshooting.md) provides tips on how to debug and troubleshoot CNI.
 
+## ENI Allocation
+
+When a worker node first joins the cluster, there is only 1 ENI along with all of its addresses in the ENI. Without any configuration, ipamD always try to keep one extra ENI.
+
+When number of pods running on the node exceeds the number of addresses on a single ENI, the CNI backend start allocating a new ENI and start using following allocation scheme:
+
+For example, a m4.4xlarge node can have up to 8 ENIs, and each ENI can have up to 30 IP addresses. ( https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-eni.html ).   
+
+* If the number of current running Pods is between 0 to 29, ipamD will allocate one more eni. And Warm-Pool size is 2 eni * (30 -1) = 58 
+* If the number of current running Pods is between 30 and 58, ipamD will allocate 2 more eni. And Warm-Pool size is 3 eni * (30 -1) = 87
+
+
 ### Notes
 
 `L-IPAMD`(aws-node daemonSet) running on every worker node requires access to kubernetes API server.  If it can **not** reach kubernetes API server, ipamD will exit and CNI will not be able to get any IP address for Pods.  Here is a way to confirm if `L-IPAMD` has access to the kubernetes API server.