Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(eks): IAM Roles for service accounts in imported clusters #10774

Merged
Merged
Show file tree
Hide file tree
Changes from 53 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
34db58f
feat(eks): IAM Roles for service accounts in imported clusters
aka-toxa Oct 8, 2020
62105cd
fix tests
aka-toxa Oct 11, 2020
ac493ed
added tests
aka-toxa Oct 12, 2020
5c67ea6
Merge branch 'master' of github.com:aka-toxa/aws-cdk into aka-toxa/ir…
aka-toxa Oct 12, 2020
c53e283
added readme
aka-toxa Oct 12, 2020
366fa3e
changed annotations for oidcprovider url and cluster class attributes
aka-toxa Oct 12, 2020
a80242c
use arnParse function in oidc prodiver class
aka-toxa Oct 12, 2020
b02c45c
move oidc provider to base cluster
aka-toxa Oct 12, 2020
c5310fb
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
iliapolo Oct 12, 2020
f60840a
Update packages/@aws-cdk/aws-iam/lib/oidc-provider.ts
aka-toxa Oct 13, 2020
0585186
only oidc provider should be passed to cluster attributes
aka-toxa Oct 13, 2020
1a6373f
Merge branch 'aka-toxa/irsa-support-for-imported-clusters' of github.…
aka-toxa Oct 13, 2020
34b7597
added special OIDC provider for eks
aka-toxa Oct 13, 2020
49f8b65
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 13, 2020
087e854
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 13, 2020
555e338
fixed issuer and issuerUrl misunderstanding
aka-toxa Oct 15, 2020
5a4f651
Merge branch 'aka-toxa/irsa-support-for-imported-clusters' of github.…
aka-toxa Oct 15, 2020
b124c38
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 15, 2020
3b8dd8a
fix
aka-toxa Oct 15, 2020
17cde33
Merge branch 'aka-toxa/irsa-support-for-imported-clusters' of github.…
aka-toxa Oct 15, 2020
36b5e84
cleanup unecessary fields from base cluster
aka-toxa Oct 15, 2020
415d1c3
cleanup legacy cluster
aka-toxa Oct 15, 2020
94e3031
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 15, 2020
5797da1
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 15, 2020
247455c
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 16, 2020
67dc05b
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 16, 2020
bf7cbda
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 16, 2020
7c730b4
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 16, 2020
d1414de
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Oct 17, 2020
e5abaa2
resolved pr comments for iam roles for service accounts in imported c…
Oct 22, 2020
b7faaf5
Merge remote-tracking branch 'upstream/master' into aka-toxa/irsa-sup…
Oct 26, 2020
fe1b126
removed unused cdk8s import
Oct 26, 2020
25480e4
updated readme and comments in cluster.ts
Oct 27, 2020
ff75a74
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
parkhomenko Oct 27, 2020
dbcd9a5
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
parkhomenko Oct 28, 2020
51d544f
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
parkhomenko Oct 29, 2020
b2c5d63
Updated readme and other PR comments
Nov 2, 2020
b0d83bd
Fixed tests
Nov 3, 2020
cc490b2
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
parkhomenko Nov 3, 2020
011f917
Fixed integration tests
Nov 3, 2020
25124fb
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 4, 2020
e6e399f
Update packages/@aws-cdk/aws-eks/test/integ.eks-oidc-provider.ts
aka-toxa Nov 9, 2020
abcfb00
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
94ba5fd
remove physical name from oidc provider in eks
aka-toxa Nov 9, 2020
8f5b708
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
0ee2a13
fix tests
aka-toxa Nov 9, 2020
20c6ecd
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
a7b4ff9
fix tests
aka-toxa Nov 9, 2020
bfe194c
Merge branch 'aka-toxa/irsa-support-for-imported-clusters' of github.…
aka-toxa Nov 9, 2020
bc858a4
fix linting
aka-toxa Nov 9, 2020
29e9777
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
a9f2d59
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
8842612
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
aka-toxa Nov 9, 2020
fe77299
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
iliapolo Nov 10, 2020
97b6983
Merge branch 'master' into aka-toxa/irsa-support-for-imported-clusters
mergify[bot] Nov 10, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions packages/@aws-cdk/aws-eks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -643,6 +643,32 @@ new cdk.CfnOutput(this, 'ServiceAccountIamRole', { value: sa.role.roleArn })
Note that using `sa.serviceAccountName` above **does not** translate into a resource dependency.
This is why an explicit dependency is needed. See <https://github.com/aws/aws-cdk/issues/9910> for more details.

You can also add service accounts to existing clusters.
To do so, pass the `openIdConnectProvider` property when you import the cluster into the application.
```ts
// you can import an existing provider
const provider = eks.OpenIdConnectProvider.fromOpenIdConnectProviderArn(this, 'Provider', 'arn:aws:iam::123456:oidc-provider/oidc.eks.eu-west-1.amazonaws.com/id/AB123456ABC');

// or create a new one using an existing issuer url
const provider = new eks.OpenIdConnectProvider(this, 'Provider', issuerUrl);

const cluster = eks.Cluster.fromClusterAttributes({
clusterName: 'Cluster',
openIdConnectProvider: provider,
kubectlRoleArn: 'arn:aws:iam::123456:role/service-role/k8sservicerole',
});

const sa = cluster.addServiceAccount('MyServiceAccount');

const bucket = new Bucket(this, 'Bucket');
bucket.grantReadWrite(serviceAccount);

// ...
```
Note that adding service accounts requires running `kubectl` commands against the cluster.
This means you must also pass the `kubectlRoleArn` when importing the cluster.
See [Using existing Clusters](https://github.com/aws/aws-cdk/tree/master/packages/@aws-cdk/aws-eks#using-existing-clusters).

## Applying Kubernetes Resources

The library supports several popular resource deployment mechanisms, among which are:
Expand Down
70 changes: 45 additions & 25 deletions packages/@aws-cdk/aws-eks/lib/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import { KubernetesObjectValue } from './k8s-object-value';
import { KubernetesPatch } from './k8s-patch';
import { KubectlProvider } from './kubectl-provider';
import { Nodegroup, NodegroupOptions } from './managed-nodegroup';
import { OpenIdConnectProvider } from './oidc-provider';
import { BottleRocketImage } from './private/bottlerocket';
import { ServiceAccount, ServiceAccountOptions } from './service-account';
import { LifecycleLabel, renderAmazonLinuxUserData, renderBottlerocketUserData } from './user-data';
Expand Down Expand Up @@ -77,6 +78,11 @@ export interface ICluster extends IResource, ec2.IConnectable {
*/
readonly clusterEncryptionConfigKeyArn: string;

/**
* The Open ID Connect Provider of the cluster used to configure Service Accounts.
*/
readonly openIdConnectProvider: iam.IOpenIdConnectProvider;

/**
* An IAM role that can perform kubectl operations against this cluster.
*
Expand Down Expand Up @@ -113,6 +119,14 @@ export interface ICluster extends IResource, ec2.IConnectable {
*/
readonly kubectlLayer?: lambda.ILayerVersion;

/**
* Creates a new service account with corresponding IAM Role (IRSA).
*
* @param id logical id of service account
* @param options service account options
*/
addServiceAccount(id: string, options?: ServiceAccountOptions): ServiceAccount;

/**
* Defines a Kubernetes resource in this cluster.
*
Expand Down Expand Up @@ -220,6 +234,14 @@ export interface ClusterAttributes {
*/
readonly kubectlPrivateSubnetIds?: string[];

/**
* An Open ID Connect provider for this cluster that can be used to configure service accounts.
* You can either import an existing provider using `iam.OpenIdConnectProvider.fromProviderArn`,
* or create a new provider using `new eks.OpenIdConnectProvider`
* @default - if not specified `cluster.openIdConnectProvider` and `cluster.addServiceAccount` will throw an error.
*/
readonly openIdConnectProvider?: iam.IOpenIdConnectProvider;
aka-toxa marked this conversation as resolved.
Show resolved Hide resolved

/**
* An AWS Lambda Layer which includes `kubectl`, Helm and the AWS CLI.
*
Expand Down Expand Up @@ -608,6 +630,7 @@ abstract class ClusterBase extends Resource implements ICluster {
public abstract readonly kubectlEnvironment?: { [key: string]: string };
public abstract readonly kubectlSecurityGroup?: ec2.ISecurityGroup;
public abstract readonly kubectlPrivateSubnets?: ec2.ISubnet[];
public abstract readonly openIdConnectProvider: iam.IOpenIdConnectProvider;

/**
* Defines a Kubernetes resource in this cluster.
Expand Down Expand Up @@ -651,6 +674,13 @@ abstract class ClusterBase extends Resource implements ICluster {

return this.addManifest(id, ...cdk8sChart.toJson());
}

public addServiceAccount(id: string, options: ServiceAccountOptions = {}): ServiceAccount {
return new ServiceAccount(this, id, {
...options,
cluster: this,
});
}
}

/**
Expand Down Expand Up @@ -801,6 +831,11 @@ export class Cluster extends ClusterBase {
*/
private readonly _fargateProfiles: FargateProfile[] = [];

/**
* an Open ID Connect Provider instance
*/
private _openIdConnectProvider?: iam.IOpenIdConnectProvider;

/**
* The AWS Lambda layer that contains `kubectl`, `helm` and the AWS CLI. If
* undefined, a SAR app that contains this layer will be used.
Expand All @@ -819,8 +854,6 @@ export class Cluster extends ClusterBase {
*/
private _awsAuth?: AwsAuth;

private _openIdConnectProvider?: iam.OpenIdConnectProvider;

private _spotInterruptHandler?: HelmChart;

private _neuronDevicePlugin?: KubernetesManifest;
Expand Down Expand Up @@ -851,7 +884,7 @@ export class Cluster extends ClusterBase {
* Initiates an EKS Cluster with the supplied arguments
*
* @param scope a Construct, most likely a cdk.Stack created
* @param name the name of the Construct to create
* @param id the id of the Construct to create
* @param props properties in the IClusterProps interface
*/
constructor(scope: Construct, id: string, props: ClusterProps) {
Expand Down Expand Up @@ -1249,15 +1282,8 @@ export class Cluster extends ClusterBase {
*/
public get openIdConnectProvider() {
if (!this._openIdConnectProvider) {
this._openIdConnectProvider = new iam.OpenIdConnectProvider(this, 'OpenIdConnectProvider', {
this._openIdConnectProvider = new OpenIdConnectProvider(this, 'OpenIdConnectProvider', {
url: this.clusterOpenIdConnectIssuerUrl,
clientIds: ['sts.amazonaws.com'],
/**
* For some reason EKS isn't validating the root certificate but a intermediat certificate
* which is one level up in the tree. Because of the a constant thumbprint value has to be
* stated with this OpenID Connect provider. The certificate thumbprint is the same for all the regions.
*/
thumbprints: ['9e99a48a9960b14926bb7f3b02e22da2b0ab7280'],
});
}

Expand All @@ -1278,19 +1304,6 @@ export class Cluster extends ClusterBase {
});
}

/**
* Adds a service account to this cluster.
*
* @param id the id of this service account
* @param options service account options
*/
public addServiceAccount(id: string, options: ServiceAccountOptions = { }) {
return new ServiceAccount(this, id, {
...options,
cluster: this,
});
}

/**
* Internal API used by `FargateProfile` to keep inventory of Fargate profiles associated with
* this cluster, for the sake of ensuring the profiles are created sequentially.
Expand Down Expand Up @@ -1606,7 +1619,7 @@ export interface AutoScalingGroupOptions {
/**
* Import a cluster to use in another stack
*/
class ImportedCluster extends ClusterBase implements ICluster {
class ImportedCluster extends ClusterBase {
public readonly clusterName: string;
public readonly clusterArn: string;
public readonly connections = new ec2.Connections();
Expand Down Expand Up @@ -1672,6 +1685,13 @@ class ImportedCluster extends ClusterBase implements ICluster {
}
return this.props.clusterEncryptionConfigKeyArn;
}

public get openIdConnectProvider(): iam.IOpenIdConnectProvider {
if (!this.props.openIdConnectProvider) {
throw new Error('"openIdConnectProvider" is not defined for this imported cluster');
}
return this.props.openIdConnectProvider;
}
}

/**
Expand Down
3 changes: 2 additions & 1 deletion packages/@aws-cdk/aws-eks/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,5 @@ export * from './k8s-object-value';
export * from './fargate-cluster';
export * from './service-account';
export * from './managed-nodegroup';
export * from './kubectl-layer';
export * from './kubectl-layer';
export * from './oidc-provider';
21 changes: 21 additions & 0 deletions packages/@aws-cdk/aws-eks/lib/legacy-cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { CfnCluster, CfnClusterProps } from './eks.generated';
import { HelmChartOptions, HelmChart } from './helm-chart';
import { KubernetesManifest } from './k8s-manifest';
import { Nodegroup, NodegroupOptions } from './managed-nodegroup';
import { ServiceAccount, ServiceAccountOptions } from './service-account';
import { renderAmazonLinuxUserData, renderBottlerocketUserData } from './user-data';

// defaults are based on https://eksctl.io
Expand Down Expand Up @@ -244,6 +245,18 @@ export class LegacyCluster extends Resource implements ICluster {
}
}

public addServiceAccount(_id: string, _options?: ServiceAccountOptions): ServiceAccount {
throw new Error('legacy cluster does not support adding service accounts');
}

/**
* Since we dont really want to make it required on the top-level ICluster
* we do this trick here in return type to match interface type
*/
public get openIdConnectProvider(): iam.IOpenIdConnectProvider {
throw new Error('legacy cluster does not support open id connect providers');
}

/**
* Add nodes to this EKS cluster
*
Expand Down Expand Up @@ -437,6 +450,14 @@ class ImportedCluster extends Resource implements ICluster {
throw new Error('legacy cluster does not support adding cdk8s charts');
}

public addServiceAccount(_id: string, _options?: ServiceAccountOptions): ServiceAccount {
throw new Error('legacy cluster does not support adding service accounts');
}

public get openIdConnectProvider(): iam.IOpenIdConnectProvider {
throw new Error('legacy cluster does not support open id connect providers');
}

public get vpc() {
if (!this.props.vpc) {
throw new Error('"vpc" is not defined for this imported cluster');
Expand Down
59 changes: 59 additions & 0 deletions packages/@aws-cdk/aws-eks/lib/oidc-provider.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import * as iam from '@aws-cdk/aws-iam';
import { Construct } from 'constructs';

/**
* Initialization properties for `OpenIdConnectProvider`.
*/
export interface OpenIdConnectProviderProps {
/**
* The URL of the identity provider. The URL must begin with https:// and
* should correspond to the iss claim in the provider's OpenID Connect ID
* tokens. Per the OIDC standard, path components are allowed but query
* parameters are not. Typically the URL consists of only a hostname, like
* https://server.example.org or https://example.com.
*
* You can find your OIDC Issuer URL by:
* aws eks describe-cluster --name %cluster_name% --query "cluster.identity.oidc.issuer" --output text
aka-toxa marked this conversation as resolved.
Show resolved Hide resolved
*/
readonly url: string;
}

/**
* IAM OIDC identity providers are entities in IAM that describe an external
* identity provider (IdP) service that supports the OpenID Connect (OIDC)
* standard, such as Google or Salesforce. You use an IAM OIDC identity provider
* when you want to establish trust between an OIDC-compatible IdP and your AWS
* account.
*
* This implementation has default values for thumbprints and clientIds props
* that will be compatible with the eks cluster
*
* @see http://openid.net/connect
* @see https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_oidc.html
*
* @resource AWS::CloudFormation::CustomResource
*/
export class OpenIdConnectProvider extends iam.OpenIdConnectProvider {
/**
* Defines an OpenID Connect provider.
* @param scope The definition scope
* @param id Construct ID
* @param props Initialization properties
*/
public constructor(scope: Construct, id: string, props: OpenIdConnectProviderProps) {
/**
* For some reason EKS isn't validating the root certificate but a intermediate certificate
* which is one level up in the tree. Because of the a constant thumbprint value has to be
* stated with this OpenID Connect provider. The certificate thumbprint is the same for all the regions.
*/
const thumbprints = ['9e99a48a9960b14926bb7f3b02e22da2b0ab7280'];
aka-toxa marked this conversation as resolved.
Show resolved Hide resolved

const clientIds = ['sts.amazonaws.com'];

super(scope, id, {
url: props.url,
thumbprints,
clientIds,
});
}
}
9 changes: 4 additions & 5 deletions packages/@aws-cdk/aws-eks/lib/service-account.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { AddToPrincipalPolicyResult, IPrincipal, IRole, OpenIdConnectPrincipal, PolicyStatement, PrincipalPolicyFragment, Role } from '@aws-cdk/aws-iam';
import { CfnJson, Names } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { Cluster } from './cluster';
import { ICluster } from './cluster';
import { KubernetesManifest } from './k8s-manifest';

// v2 - keep this import as a separate section to reduce merge conflict when forward merging with the v2 branch.
Expand Down Expand Up @@ -31,9 +31,8 @@ export interface ServiceAccountOptions {
export interface ServiceAccountProps extends ServiceAccountOptions {
/**
* The cluster to apply the patch to.
* [disable-awslint:ref-via-interface]
*/
readonly cluster: Cluster;
readonly cluster: ICluster;
}

/**
Expand Down Expand Up @@ -71,8 +70,8 @@ export class ServiceAccount extends CoreConstruct implements IPrincipal {
*/
const conditions = new CfnJson(this, 'ConditionJson', {
value: {
[`${cluster.clusterOpenIdConnectIssuer}:aud`]: 'sts.amazonaws.com',
[`${cluster.clusterOpenIdConnectIssuer}:sub`]: `system:serviceaccount:${this.serviceAccountNamespace}:${this.serviceAccountName}`,
[`${cluster.openIdConnectProvider.openIdConnectProviderIssuer}:aud`]: 'sts.amazonaws.com',
[`${cluster.openIdConnectProvider.openIdConnectProviderIssuer}:sub`]: `system:serviceaccount:${this.serviceAccountNamespace}:${this.serviceAccountName}`,
},
});
const principal = new OpenIdConnectPrincipal(cluster.openIdConnectProvider).withConditions({
Expand Down
3 changes: 2 additions & 1 deletion packages/@aws-cdk/aws-eks/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,8 @@
},
"awslint": {
"exclude": [
"props-no-arn-refs:@aws-cdk/aws-eks.ClusterProps.outputMastersRoleArn"
"props-no-arn-refs:@aws-cdk/aws-eks.ClusterProps.outputMastersRoleArn",
"props-physical-name:@aws-cdk/aws-eks.OpenIdConnectProviderProps"
]
},
"stability": "experimental",
Expand Down
Loading