Add test-e2e-external-eks make rule that tests EKS with pod instance metadata disabled. Remove hostNetwork from DaemonSet

[wongma7]

Is this a bug fix or adding new feature? test

What is this PR about? / Why do we need it? test for the case where instance metadata is disabled for pods. EKS specifically recommends this. In this case, only pods with hostNetwork true will have access to instance metadata.

To create the environment, use eksctl --disable-pod-imds.

The expected behavior is that in lieu of instance metadata

• the controller: should get credentials from IAM for SA. This gets configured via eksctl.
• the node: should get instance info from k8s API. This gets configured by our helm chart. The node should NOT need hostNetwork true anymore because the sole reason that was there was so it could bypass disable-pod-imds to touch instance metadata.

TODO for future:
a test case where instance metadata is totally disabled on instances, not just for pods. hostNetwork is now false for both controller and node, so whether instance metadata is available on host or not should make no difference , but an extra test case won't hurt.

What testing is done?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: wongma7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [wongma7]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Coverage increased (+0.002%) to 79.578% when pulling 38b0165 on wongma7:metadatatest into ccc060a on kubernetes-sigs:master.

[wongma7]

/test pull-aws-ebs-csi-driver-external-test-eks

[wongma7]

/test pull-aws-ebs-csi-driver-external-test-eks

[wongma7]

/test pull-aws-ebs-csi-driver-external-test-eks

[wongma7]

I0601 23:59:18.405899 1 metadata.go:101] retrieving instance data from ec2 metadata
W0601 23:59:21.746294 1 metadata.go:104] ec2 metadata is not available
I0601 23:59:21.746314 1 metadata.go:112] retrieving instance data from kubernetes api
I0601 23:59:21.747904 1 metadata.go:117] kubernetes api is available

[wongma7]

/test pull-aws-ebs-csi-driver-e2e-single-az

[wongma7]

/test pull-aws-ebs-csi-driver-external-test-eks

[vdhanan]

/lgtm

[vdhanan]

/hold

[vdhanan]

do we need to bump the helm chart version?

[wongma7]

@vdhanan I think these changes should go in the next new driver + helm chart release, I don't want to release a helm chart yet.

My reasoning: if we release a new helm chart now, then the driver should work even with hostNetwork off and instance metadata off, but it hasn't been tested, the only thing that has been tested is the particular combination of driver + helm chart built in this PR.

[wongma7]

rebased, conflict was hack/values.yaml got logLevel: 5 added to it (to easily debug tests)

[nckturner]

/lgtm

[vdhanan]

/unhold