(aws-eks): EKS Cluster does not respect vpcSubnets

[NukaCody]

What is the problem?

When using aws-eks to provision a cluster, I pass in explicit subnets. However, the underlying node group uses different sunbets.

Reproduction Steps

export default class KubeStack extends Stack {

  constructor(scope: Construct, id: string) {
    super(scope, id);

    // ======================================
    // Imports
    // ======================================

    const vpc = Vpc.fromLookup(this, 'IsolatedNetworkVPCKubeImport', { vpcName: ISOLATED_VPC_NAME });

    // ======================================
    // Kubernetes Cluster
    // ======================================

    // @ts-ignore
    const kube = new Cluster(this, 'KubeCluster', {
      version: KubernetesVersion.V1_21,
      vpc,
      vpcSubnets: [
        vpc.selectSubnets({ subnetGroupName: 'Ingress' }),
        vpc.selectSubnets({ subnetGroupName: 'Application' }),
      ],
    });

  }
}

Intended CFN Output. These subnet Ids correlate to "Ingress" and "Application" subnets

      "Type": "Custom::AWSCDK-EKS-Cluster",
      "Properties": {
           ....
          "resourcesVpcConfig": {
            "subnetIds": [
              "subnet-0a050e83ac2af59b4",
              "subnet-0b155e3999c3ce2ae",
              "subnet-0906a6719e12bd85c",
              "subnet-098f5fb7c2529cb95"
            ],
           ....

However, these subnets correlate to a database subnet (which has NACLs applied to only allow DB traffic) causing the deployment to fail.

      "Type": "AWS::EKS::Nodegroup",
      "Properties": {
        ....
        "Subnets": [
          "subnet-0dc27f629002af159",
          "subnet-098fd833b90a01821",
          "subnet-0906a6719e12bd85c",
          "subnet-098f5fb7c2529cb95"
        ],
        .....

What did you expect to happen?

I intended for EKS to only use the subnets that I explicitly pointing out (but it looks like it's stealing private subnets from the passed in VPC)

What actually happened?

EKS Cluster used a non-intended subnet for node cluster provisioning

CDK CLI Version

1.127.0

Framework Version

1.127.0

Node.js Version

v14.18.0

OS

Linux

Language

Typescript

Language Version

No response

Other information

No response

[NukaCody]

Splitting the deployment like this:

    const kube = new Cluster(this, 'KubeCluster', {
      version: KubernetesVersion.V1_21,
      vpc,
      vpcSubnets: [
        vpc.selectSubnets({ subnetGroupName: 'Ingress' }),
        vpc.selectSubnets({ subnetGroupName: 'Application' }),
      ],
      defaultCapacity: 0,
    });

    kube.addNodegroupCapacity('PrivateCapacity', {
      subnets: vpc.selectSubnets({ subnetGroupName: 'Application' }),
    });

Does work for me.

[peterwoodworth]

This is because the subnets you pass into your cluster don't get passed to the automatically created nodegroup. First, this function is called in the Cluster constructor

aws-cdk/packages/@aws-cdk/aws-eks/lib/cluster.ts

Lines 1322 to 1323 in 1d3883a

	this.defaultNodegroup = props.defaultCapacityType !== DefaultCapacityType.EC2 ?
	this.addNodegroupCapacity('DefaultCapacity', { instanceTypes: [instanceType], minSize: minCapacity }) : undefined;

The implementation of this function looks like this

aws-cdk/packages/@aws-cdk/aws-eks/lib/cluster.ts

Lines 1416 to 1421 in 1d3883a

	public addNodegroupCapacity(id: string, options?: NodegroupOptions): Nodegroup {
	return new Nodegroup(this, `Nodegroup${id}`, {
	cluster: this,
	...options,
	});
	}

And this is inside the NodeGroup constructor

aws-cdk/packages/@aws-cdk/aws-eks/lib/managed-nodegroup.ts

Lines 377 to 381 in 1d3883a

	const resource = new CfnNodegroup(this, 'Resource', {
	clusterName: this.cluster.clusterName,
	nodegroupName: props.nodegroupName,
	nodeRole: this.role.roleArn,
	subnets: this.cluster.vpc.selectSubnets(props.subnets).subnetIds,

So, things are working here as intended :) Though I can see why confusion occurred. @otaviomacedo is there any action we can take here to make this more intuitive?

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.