Add hostPort support #144

[igorvpcleao]

• Issue: hostPort not working #144 *

This PR adds support to hostPort.

Design decisions to de discussed:

1. Not sure if hostPort support should be offered by amazon-vpc-cni-k8s image itself or by modifying amazon-vpc-cni manifests on Kops, adding a new image to perform a slightly different installation, for instance.
2. Portmap plugin is downloaded from github.com/containernetworking/plugins/releases

[cjbottaro]

I built and installed this in my cluster. Seem to work! 👍 Thank you very much for this!

To be more clear about it working, I'm running an Nginx pod using hostPort (80 and 443) and I'm able to hit those ports on the given node from outside the cluster.

[brockoffdev]

Would love to see this merged in, this is actually a pretty big blocker for our team on EKS 😄 👍

[liwenwu-amazon]

@igorvpcleao Thank you for submitting this PR.
Can you add some detail steps:

• how to reproduce the issue
• how to verify this PR has fixed the issue

[igorvpcleao]

@liwenwu-amazon Sure. We need to create a pod and then set the hostPort field in its spec. After that, we make requests to ip_from_node_where_pod_is_running:hostPort. Then we should get an answer from the pod if this PR is running as the CNI. In case of running amazon-vpc-cni:1.1.0, requests will not reach the pod.

Here you are a manifest for an app which replies Hey. Look, I'm alive!
https://gist.github.com/igorvpcleao/a0f5087fa32f3438f3a0e912adf0a608

If the node security group is open to where you are performing the request, via internet, you may run something like:

curl 54.209.x.x:51080
Hey. Look, I'm alive!%

Or you can ssh to a node from within your Kubernetes cluster and make the request:

curl 172.50.58.153:51080
Hey. Look, I'm alive!

Where 172.50.58.153 is a private ip for the node where your pod is running and 54.209.x.x is its public ip.

[liwenwu-amazon]

@igorvpcleao Thanks for providing the test scenario. I have one question:

After I did kubectl apply -f manifests.yaml , I can see following

# find out hey-iam-alive-hostport Pod IP
 kubectl get all -n hey-iam-alive-poc-awc-cni -o wide
NAME                     READY     STATUS    RESTARTS   AGE       IP                NODE
hey-iam-alive-hostport   1/1       Running   0          18m       192.168.254.207   ip-192-168-238-237.us-west-2.compute.internal

From one of the Pod running in the same cluster, I am able to get response from Pod using Pod's IP

# here are other Pods running in the same cluster
kubectl get pod -o wide
NAME                            READY     STATUS    RESTARTS   AGE       IP                NODE
hostport-nginx                  1/1       Running   0          40m       192.168.86.148    ip-192-168-125-35.us-west-2.compute.internal
nginx-65899c769f-9pttb          1/1       Running   0          46m       192.168.167.200   ip-192-168-178-137.us-west-2.compute.internal
worker-hello-5974f49799-22sjl   1/1       Running   0          45m       192.168.114.218   ip-192-168-125-35.us-west-2.compute.internal
worker-hello-5974f49799-5nkb4   1/1       Running   0          45m       192.168.212.76    ip-192-168-238-237.us-west-2.compute.internal
worker-hello-5974f49799-lj4kn   1/1       Running   0          45m       192.168.85.77     ip-192-168-125-35.us-west-2.compute.internal
worker-hello-5974f49799-ppvzj   1/1       Running   0          45m       192.168.184.168   ip-192-168-178-137.us-west-2.compute.internal
worker-hello-5974f49799-vfqc2   1/1       Running   0          45m       192.168.195.140   ip-192-168-238-237.us-west-2.compute.internal

# log into one of these Pods
kubectl exec -ti worker-hello-5974f49799-22sjl sh

#  try to get the content of hey-iam-alive-hostport Pod
wget 192.168.254.207:5000
Connecting to 192.168.254.207:5000 (192.168.254.207:5000)
index.html           100% |*******************************************************************************************************************************************************************|    21   0:00:00 ETA
/ # cat index.html
Hey. Look, I'm alive!/ # 

In summary, I am able to reach Pod using Pod's IP and its container port.

I am curious to know what's the use cases you need to use hostport?

thanks

[zapman449]

@liwenwu-amazon The use-case is Ingress. For us it would be required to expose 80/443 on the instance to an end-user-facing NLB.

[cjbottaro]

Same use case here.

Our clients require a whitelisted IP address to connect to. So we run a tcp proxy on an instance with an AWS EIP that just forwards to an AWS ALB.

[micahhausler]

Thanks for the PR! We're moving a few things around before the next release, and we'll be copying CNI plugins (including portmap) from an init container (See #158). For now, could you remove the portmap download and copy but keep the configuration changes?

[micahhausler]

We currently bake CNI binaries in our AMI, but we should really put this process in an init container (See #158).

Can you remove this for now?

[micahhausler]

Can you remove this?

[micahhausler]

Can you remove this too?

[igorvpcleao]

@micahhausler Sure, I removed my last commit. It only handled portmap plugin. Let me know if you need further changes.

[igorvpcleao]

@liwenwu-amazon Sometimes it may be a little hard to handle requests from outside the cluster when we have stateful applications, mainly when using UDP. hostPort may help to simplify our architecture.

[igorvpcleao]

Hi @micahhausler,

Let me know if you need any further changes.

Best,

[liwenwu-amazon]

can you test the upgrade? In another word, what happens when the new docker CNI image is loaded onto an existing worker? thanks

[micahhausler]

Sorry for the long delay, one more small change

[micahhausler]

Can you remove aws.conf if it already exists? The file will already be there on existing systems where this is an upgrade.

if [[ -f /host/etc/cni/net.d/aws.conf ]]; then
    rm /host/etc/cni/net.d/aws.conf
fi

[igorvpcleao]

@micahhausler sure. I can submit it in a second.

[igorvpcleao]

@micahhausler done. Let me know if you want me to write a comment about it.

[adammw]

@micahhausler will this PR be compatible with the current CNI release or blocked behind the changes in #158? Lack of hostPort functionality is a blocker for us to migrating to this CNI plugin.

[liwenwu-amazon]

@igorvpcleao please also rebase your change.

[igorvpcleao]

@liwenwu-amazon done.

[liwenwu-amazon]

@igorvpcleao thanks! Testing your PR now...

[liwenwu-amazon]

@igorvpcleao For some reason, with this PR, I am NOT able to create any Pod. Let me try again with top of tree to see if this is my environment issue Are you able to test your PR ?

[igorvpcleao]

@liwenwu-amazon I think it's because in this PR we do not have the portmap plugin anymore, as it was describe at #158.

[igorvpcleao]

@liwenwu-amazon Is the init container from #158 already available?

[liwenwu-amazon]

@igorvpcleao No, it is not available yet.

[liwenwu-amazon]

@igorvpcleao can you revert the change back to your original commit, so host port will work just by this PR. When we working on #158 , we can do same logic for portmap plugin. thanks

[igorvpcleao]

@igorvpcleao can you revert the change back to your original commit, so host port will work just by this PR. When we working on #158 , we can do same logic for portmap plugin. thanks

@liwenwu-amazon Sure! I've just added it again. We just need to be aligned with @micahhausler about it.
Any problem, let me know.

[liwenwu-amazon]

@igorvpcleao can you rebase your change and make change in config/v1.2 , also squash them into 1 commit? thanks

[igorvpcleao]

@liwenwu-amazon Sure! Sorry for my delay. What changes do you want to be made on config/v1.2? It seems ok for me.

[liwenwu-amazon]

@igorvpcleao Sorry, you don't need to change to config/v1.2. Can you rebase and squash all of your commits into 1 single commit? Once you do that, I will kick off testing. thanks

[igorvpcleao]

@liwenwu-amazon No problem. Yep, It's done on #ee54500bc2c79c5731066d71286411a490b01b10. Let me know if you have any problem.

[liwenwu-amazon]

@igorvpcleao Thanks! Starting testing it now. Once it passed conformance test, I will add it back to 1.2. thanks