[EKS] [request]: what's the recommended approach to allow non-AmazonEKSClusterAdmin IAM role to create custom resources on EKS?

[duxing]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request

Context: EKS@1.29, provisioned with terraform-aws-eks@20.8.3

This maybe a dumb question based on my limited knowledge of EKS access entry

I recently converted my EKS cluster from using aws-auth to using access entry I assigned AmazonEKSEditPolicy to an IAM role used for deployment (e.g. installing helm charts). Deployments worked well with this IAM role when my chart contained k8s native resources.

when I added custom resources (e.g. istio ServiceEntry) to my helm chart, I noticed a permission error during deployment:

DESCRIPTION: Upgrade "myservice" failed: could not get information about the resource: serviceentries.networking.istio.io "myservice-api-external" is forbidden: User "arn:aws:sts::<my_account>:assumed-role/eks-deploy-role-/EKSGetTokenAuth" cannot get resource "serviceentries" in API group "networking.istio.io" in the namespace "myservice"

According to the reference, this seems like an issue for any custom resources if they're NOT installed using a role with AmazonEKSClusterAdmin policy as access entry. The API group and resource names for these custom resources are not whitelisted in the policy. Only * permission from AmazonEKSClusterAdmin can capture these.

I've also validated that when I grant AmazonEKSClusterAdmin access entry to the same role on the same EKS cluster, the same helm release (same chart, same version, same values) was deployed correctly without the permission error.

I don't want to expose AmazonEKSClusterAdmin to my deployment role but I can't figure out a native way to additively grant additional RBAC rules to the same entity using a consistent provisioning process (e.g. terraform EKS module creates access entry, ideally that's the only place I needed to specify RBAC for my roles)

Is there an existing way to do this? if not, it would be beneficial to support this as part of access entry.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

see request part

Are you currently working around this issue?

2 workarounds that I've identified and validated working:

1. bump up to AmazonEKSClusterAdmin
2. specify kubernetes_groups with a group value (e.g. deployer), set up a new ClusterRole with RBAC rules for custom resources, and set up ClusterRoleBinding (or multiple RoleBinding for multiple namespaces) with the roleRef pointing to the new ClusterRole and use kind: Group + name: deployer as one of the subjects

Additional context

Context: EKS@1.29, provisioned with terraform-aws-eks@20.8.3

Attachments

N/A

[duxing]

specify kubernetes_groups with a group value (e.g. deployer), set up a new ClusterRole with RBAC rules for custom resources, and set up ClusterRoleBinding (or multiple RoleBinding for multiple namespaces) with the roleRef pointing to the new ClusterRole and set kind: Group + name: deployer as one of the subjects

the 2nd workaround is native to k8s (not EKS-specific) but now we have 2 sources (in 2 different domains) for RBAC for a single IAM role.
would love to see EKS access entry supporting custom access policy (similar to IAM policies) to consolidate into one.

[mikestef9]

See my previous comment for detailed explanation of EKS access entry authorizer. The TLDR is that access entry is not meant as a replacement for RBAC. It's an alternative for more straightforward use cases. The current access entry policies map to the upstream Kubernetes user facing roles.

AmazonEKSAdminPolicy might be what you are looking for here, which is less permissive than AmazonEKSClusterAdminPolicy. The upstream user facing role page linked above has more details.

Otherwise, your workaround of using RBAC isn't a workaround at all. If the policies EKS ships don't meet your use case, the recommendation is to use RBAC. I would suggest opening a separate issue on custom policy support for EKS access entry, as I don't believe any issue has been opened for that yet.

[duxing]

@mikestef9 thank you!
what I was asking for would be better put as requesting custom policy support for EKS access entry. I'll proceed with a new issue.