VpcCniAddOn - enable supports of Kubernetes Network Policies with enableNetworkPolicy after Amazon VPC CNI version 1.14.0 #850

[bnaydenov]

Issue #850 :

Description of changes:

Introduce enableNetworkPolicy parameter and when is set to 'true' enable Network policy with AWS VPC-CNI plugin.

For example:

    const addOns: Array<blueprints.ClusterAddOn> = [
      new blueprints.addons.CoreDnsAddOn(),
      new blueprints.addons.KubeProxyAddOn(),
      new blueprints.addons.EbsCsiDriverAddOn(),
      new blueprints.addons.VpcCniAddOn({ enableNetworkPolicy: 'true', version: 'v1.15.0-eksbuild.2' })
    ];

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[bnaydenov]

Running cdk diff with this changes produces this diff:

And when cdk deploy is done VPC-CNI is run with enabled Network policy. How to check:

kubectl get pods -n kube-system | grep 'aws-node\|amazon'

An example output is as follows.

aws-node-gmqp7     2/2     Running   1 (1h ago)   1h
aws-node-prnsh     2/2     Running   1 (1h ago)   1h

If network policy is enabled, there are 2 containers in the aws-node pods. In previous versions and if network policy is disabled, there is only a single container in the aws-node pods.

More info here: https://docs.aws.amazon.com/eks/latest/userguide/cni-network-policy.html

[elamaran11]

LGTM. @shapirov103 Please review and run e2e

[bnaydenov]

@shapirov103 please check this MR when you have time.

[shapirov103]

@bnaydenov awesome work. Please my comment, it is mostly release process related and docs.

[shapirov103]

Based on https://docs.aws.amazon.com/eks/latest/userguide/managing-vpc-cni.html the version should be v1.15.0-eksbuild.2, however incremental update is needed.

It looks like the network policy will not be available with this version, is that correct? If so, can we add a check to fail if version is less than 1.15 so that we don't get questions from the customers on why my network policy does not work?

The error should state that customers should apply incremental update and use network policy starting from 1.15.

I will need to include it in the release notes, so if you can update the PR desc with these instructions as well that would help.

[bnaydenov]

@shapirov103 Network policy is available on Amazon VPC CNI version 1.14.0 or a later version.

I am setting here as default current latest version for major version 1.14.x. Currently there is version v1.15.0-eksbuild.2 , but if you set it straight to v1.15.0-eksbuild.2 update fails with following error:

Resource handler returned message: "Updating VPC-CNI can only go up or down 1 minor version at a time


And you have to make 2 steps update, first to v1.14.1-eksbuild.1, and then from v1.14.1-eksbuild.1 to v1.15.0-eksbuild.2.

Step 1

const addOns: Array<blueprints.ClusterAddOn> = [
      new blueprints.addons.VpcCniAddOn({ enableNetworkPolicy: 'true', version: 'v1.14.1-eksbuild.1' })
 ];

cdk deploy

Step 2

const addOns: Array<blueprints.ClusterAddOn> = [
      new blueprints.addons.VpcCniAddOn({ enableNetworkPolicy: 'true', version: 'v1.15.0-eksbuild.2' })
];

cdk deploy

@shapirov103 Tell me what you think about this? Should we set default to 1.14.x or straight to 1.15.x. We can include in release notes a statement above 2 steps upgrade process is necessary .

[shapirov103]

@shapirov103 Tell me what you think about this? Should we set default to 1.14.x or straight to 1.15.x. We can include in release notes a statement above 2 steps upgrade process is necessary .

@bnaydenov if 14.1 works with network policy, then I agree with your decision to use 1.14 at the moment to avoid inadvertent deployment failures in customers (and AWS SDE internal) environments due to the version skip. Let's release with 1.14 and mention 1.15 in the docs so that they can upgrade in a two step process.

I will upgrade the blueprints to use 1.15 in the next minor release.

[bnaydenov]

ok let's go with v1.14.1-eksbuild.1 then. I will leave this PR as it is.

[shapirov103]

One minor comment

[shapirov103]

Minor comment: why is it a string? Let's make it a boolean, then it is clear and we don't have to handle cases like "FALSE" vs "false".

[bnaydenov]

it must be a string not a boolean, because there is a JSON scheme about the vpc-cni addon configuration. And this schema says it must be string. When you set it to boolean it gives error when you do cdk synth

Check here JSON schema .

"enableNetworkPolicy": {
          "format": "boolean",
          "type": "string"
        },

You can get it with this command:

aws eks describe-addon-configuration --addon-name vpc-cni --addon-version v1.15.0-eksbuild.2 --query configurationSchema --output text

[shapirov103]

it must be a string not a boolean, because there is a JSON scheme about the vpc-cni addon configuration. And this schema says it must be string. When you set it to boolean it gives error when you do cdk synth

@bnaydenov in the addon it should be a boolean as this is the best experience for the customer. Please check the rest of the properties in the VpcCniAddOnProps which are declared as boolean.

You can apply JSON.stringify() to convert it to string as we do with the rest of the values.

[bnaydenov]

@shapirov103 check my latest commit.

now enableNetworkPolicy is boolean and JSON.stringify() is used

[bnaydenov]

@shapirov103 any more comments or concerns?

[shapirov103]

LGTM

[shapirov103]

/do-e2e-tests

[aws-ia-ci]

end to end tests passed