Allowing 443 to nodes from EKS service

[max-rocket-internet]

PR o'clock

Description

This allows the EKS cluster service to connect to the nodes on port 443. This is require to run the metrics-server, details here. Metrics-server is required to run horrizontal pod autoscalers.

AWS CFN is here: https://github.com/awslabs/amazon-eks-ami/blob/master/amazon-eks-nodegroup.yaml#L251-L260

Checklist

• terraform fmt and terraform validate both work from the root and examples/eks_test_fixture directories (look in CI for an example)
• Tests for the changes have been added and passing (for bug fixes/features)
• Test results are pasted in this PR (in lieu of CI)
• Docs have been updated using terraform-docs per README.md instructions
• I've added my change to CHANGELOG.md
• Any breaking changes are highlighted above

[mmcaya]

👍 had been doing this after the fact with module outputs.

Any reason not to tighten up the control plan egress rules to match that latest cloud formation as well:
https://github.com/awslabs/amazon-eks-ami/blob/master/amazon-eks-nodegroup.yaml#L229
https://github.com/awslabs/amazon-eks-ami/blob/master/amazon-eks-nodegroup.yaml#L251

Sample:

resource "aws_security_group_rule" "cluster_egress_workers" {
  description              = "Allow the cluster control plane to communicate with worker Kubelet and pods"
  protocol                 = "tcp"
  security_group_id        = "${aws_security_group.cluster.id}"
  source_security_group_id = "${local.worker_security_group_id}"
  from_port                = "${var.worker_sg_ingress_from_port}"
  to_port                  = 65535
  type                     = "egress"
  count                    = "${var.cluster_security_group_id == "" ? 1 : 0}"
}

resource "aws_security_group_rule" "cluster_egress_extension_api" {
  description              = "Allow the cluster control plane to communicate with pods running extension API servers on port 443"
  protocol                 = "tcp"
  security_group_id        = "${aws_security_group.cluster.id}"
  source_security_group_id = "${local.worker_security_group_id}"
  from_port                = 443
  to_port                  = 443
  type                     = "egress"
  count                    = "${var.cluster_security_group_id == "" ? 1 : 0}"
}

[max-rocket-internet]

had been doing this after the fact with module outputs.

Nice. I think it's reasonable to have this access in place by default.

Any reason not to tighten up the control plan egress rules to match that latest cloud formation as well

Sure but what do you mean exactly? In the AWS CFN port 1025-65535 is allowed from EKS service to nodes. And this module is the same?

[mmcaya]

Current module uses egress to the internet from control plane by default:
https://github.com/terraform-aws-modules/terraform-aws-eks/blob/master/cluster.tf#L25
vs the more granular control plane egress rules described by the cloud formation template
and this: https://docs.aws.amazon.com/eks/latest/userguide/sec-group-reqs.html

[max-rocket-internet]

@mmcaya OK good idea. I'll do that in a separate PR.

[mmcaya]

great, thanks!

[max-rocket-internet]

Paging @brandoconnor 🙂

[brandonjbjelland]

Not that you need it but you've got my blessing 🙏

[brandonjbjelland]

Thanks as always @max-rocket-internet and @mmcaya

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.