Skip to content

Commit

Permalink
feat(pipelines): Docker registry credentials (aws#15364)
Browse files Browse the repository at this point in the history
Introduce a new set of properties to the pipeline constructs to enable users to
specify Docker registries -- and associated credentials for each -- to be used
during the pipeline build/synth, self-mutate, and asset publishing stages.

These APIs enable the user to specify a Docker registry (e.g., DockerHub, ECR)
and either secrets or use role credentials to authenticate to each registry, as
well as specify which step(s) of the pipeline need these credentials.

fixes aws#10999
fixes aws#11774

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
  • Loading branch information
njlynch authored and upparekh committed Jul 8, 2021
1 parent a8b38dc commit 973ded6
Show file tree
Hide file tree
Showing 14 changed files with 943 additions and 16 deletions.
39 changes: 39 additions & 0 deletions packages/@aws-cdk/pipelines/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -584,6 +584,45 @@ const pipeline = new CdkPipeline(this, 'Pipeline', {
});
```

## Docker Registry Credentials

You can specify credentials to use for authenticating to Docker registries as part of the
pipeline definition. This can be useful if any Docker image assets — in the pipeline or
any of the application stages — require authentication, either due to being in a
different environment (e.g., ECR repo) or to avoid throttling (e.g., DockerHub).

```ts
const dockerHubSecret = secretsmanager.Secret.fromSecretCompleteArn(this, 'DHSecret', 'arn:aws:...');
const customRegSecret = secretsmanager.Secret.fromSecretCompleteArn(this, 'CRSecret', 'arn:aws:...');
const repo1 = ecr.Repository.fromRepositoryArn(stack, 'Repo', 'arn:aws:ecr:eu-west-1:0123456789012:repository/Repo1');
const repo2 = ecr.Repository.fromRepositoryArn(stack, 'Repo', 'arn:aws:ecr:eu-west-1:0123456789012:repository/Repo2');

const pipeline = new CdkPipeline(this, 'Pipeline', {
dockerCredentials: [
DockerCredential.dockerHub(dockerHubSecret),
DockerCredential.customRegistry('dockerregistry.example.com', customRegSecret),
DockerCredential.ecr([repo1, repo2]);
],
...
});
```

You can authenticate to DockerHub, or any other Docker registry, by specifying a secret
with the username and secret/password to pass to `docker login`. The names of the fields
within the secret to use for the username and password can be customized. Authentication
to ECR repostories is done using the execution role of the relevant CodeBuild job. Both
types of credentials can be provided with an optional role to assume before requesting
the credentials.

By default, the Docker credentials provided to the pipeline will be available to the
Synth/Build, Self-Update, and Asset Publishing actions within the pipeline. The scope of
the credentials can be limited via the `DockerCredentialUsage` option.

```ts
const dockerHubSecret = secretsmanager.Secret.fromSecretCompleteArn(this, 'DHSecret', 'arn:aws:...');
// Only the image asset publishing actions will be granted read access to the secret.
const creds = DockerCredential.dockerHub(dockerHubSecret, { usages: [DockerCredentialUsage.ASSET_PUBLISHING] });
```

## CDK Environment Bootstrapping

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import * as events from '@aws-cdk/aws-events';
import * as iam from '@aws-cdk/aws-iam';
import { Stack } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { dockerCredentialsInstallCommands, DockerCredential, DockerCredentialUsage } from '../docker-credentials';
import { embeddedAsmPath } from '../private/construct-internals';

// v2 - keep this import as a separate section to reduce merge conflict when forward merging with the v2 branch.
Expand Down Expand Up @@ -53,6 +54,14 @@ export interface UpdatePipelineActionProps {
* @default - false
*/
readonly privileged?: boolean

/**
* Docker registries and associated credentials necessary during the pipeline
* self-update stage.
*
* @default []
*/
readonly dockerCredentials?: DockerCredential[];
}

/**
Expand Down Expand Up @@ -83,7 +92,10 @@ export class UpdatePipelineAction extends CoreConstruct implements codepipeline.
version: '0.2',
phases: {
install: {
commands: `npm install -g aws-cdk${installSuffix}`,
commands: [
`npm install -g aws-cdk${installSuffix}`,
...dockerCredentialsInstallCommands(DockerCredentialUsage.SELF_UPDATE, props.dockerCredentials),
],
},
build: {
commands: [
Expand Down Expand Up @@ -114,6 +126,8 @@ export class UpdatePipelineAction extends CoreConstruct implements codepipeline.
actions: ['s3:ListBucket'],
resources: ['*'],
}));
(props.dockerCredentials ?? []).forEach(reg => reg.grantRead(selfMutationProject, DockerCredentialUsage.SELF_UPDATE));

this.action = new cpactions.CodeBuildAction({
actionName: 'SelfMutate',
input: props.cloudAssemblyInput,
Expand Down
230 changes: 230 additions & 0 deletions packages/@aws-cdk/pipelines/lib/docker-credentials.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,230 @@
import * as ec2 from '@aws-cdk/aws-ec2';
import * as ecr from '@aws-cdk/aws-ecr';
import * as iam from '@aws-cdk/aws-iam';
import * as secretsmanager from '@aws-cdk/aws-secretsmanager';
import { Fn } from '@aws-cdk/core';

/**
* Represents credentials used to access a Docker registry.
*/
export abstract class DockerCredential {
/**
* Creates a DockerCredential for DockerHub.
* Convenience method for `fromCustomRegistry('index.docker.io', opts)`.
*/
public static dockerHub(secret: secretsmanager.ISecret, opts: ExternalDockerCredentialOptions = {}): DockerCredential {
return new ExternalDockerCredential('index.docker.io', secret, opts);
}

/**
* Creates a DockerCredential for a registry, based on its domain name (e.g., 'www.example.com').
*/
public static customRegistry(
registryDomain: string,
secret: secretsmanager.ISecret,
opts: ExternalDockerCredentialOptions = {}): DockerCredential {
return new ExternalDockerCredential(registryDomain, secret, opts);
}

/**
* Creates a DockerCredential for one or more ECR repositories.
*
* NOTE - All ECR repositories in the same account and region share a domain name
* (e.g., 0123456789012.dkr.ecr.eu-west-1.amazonaws.com), and can only have one associated
* set of credentials (and DockerCredential). Attempting to associate one set of credentials
* with one ECR repo and another with another ECR repo in the same account and region will
* result in failures when using these credentials in the pipeline.
*/
public static ecr(repositories: ecr.IRepository[], opts?: EcrDockerCredentialOptions): DockerCredential {
return new EcrDockerCredential(repositories, opts ?? {});
}

constructor(protected readonly usages?: DockerCredentialUsage[]) { }

/**
* Determines if this credential is relevant to the input usage.
* @internal
*/
public _applicableForUsage(usage: DockerCredentialUsage) {
return !this.usages || this.usages.includes(usage);
}

/**
* Grant read-only access to the registry credentials.
* This grants read access to any secrets, and pull access to any repositories.
*/
public abstract grantRead(grantee: iam.IGrantable, usage: DockerCredentialUsage): void;

/**
* Creates and returns the credential configuration, to be used by `cdk-assets`
* to support the `docker-credential-cdk-assets` tool for `docker login`.
* @internal
*/
public abstract _renderCdkAssetsConfig(): DockerCredentialCredentialSource
}

/** Options for defining credentials for a Docker Credential */
export interface ExternalDockerCredentialOptions {
/**
* The name of the JSON field of the secret which contains the user/login name.
* @default 'username'
*/
readonly secretUsernameField?: string;
/**
* The name of the JSON field of the secret which contains the secret/password.
* @default 'secret'
*/
readonly secretPasswordField?: string;
/**
* An IAM role to assume prior to accessing the secret.
* @default - none. The current execution role will be used.
*/
readonly assumeRole?: iam.IRole
/**
* Defines which stages of the pipeline should be granted access to these credentials.
* @default - all relevant stages (synth, self-update, asset publishing) are granted access.
*/
readonly usages?: DockerCredentialUsage[];
}

/** Options for defining access for a Docker Credential composed of ECR repos */
export interface EcrDockerCredentialOptions {
/**
* An IAM role to assume prior to accessing the secret.
* @default - none. The current execution role will be used.
*/
readonly assumeRole?: iam.IRole
/**
* Defines which stages of the pipeline should be granted access to these credentials.
* @default - all relevant stages (synth, self-update, asset publishing) are granted access.
*/
readonly usages?: DockerCredentialUsage[];
}

/** Defines which stages of a pipeline require the specified credentials */
export enum DockerCredentialUsage {
/** Synth/Build */
SYNTH,
/** Self-update */
SELF_UPDATE,
/** Asset publishing */
ASSET_PUBLISHING,
};

/** DockerCredential defined by registry domain and a secret */
class ExternalDockerCredential extends DockerCredential {
constructor(
private readonly registryDomain: string,
private readonly secret: secretsmanager.ISecret,
private readonly opts: ExternalDockerCredentialOptions) {
super(opts.usages);
}

public grantRead(grantee: iam.IGrantable, usage: DockerCredentialUsage) {
if (!this._applicableForUsage(usage)) { return; }

if (this.opts.assumeRole) {
grantee.grantPrincipal.addToPrincipalPolicy(new iam.PolicyStatement({
actions: ['sts:AssumeRole'],
resources: [this.opts.assumeRole.roleArn],
}));
}
const role = this.opts.assumeRole ?? grantee;
this.secret.grantRead(role);
}

public _renderCdkAssetsConfig(): DockerCredentialCredentialSource {
return {
[this.registryDomain]: {
secretsManagerSecretId: this.secret.secretArn,
secretsUsernameField: this.opts.secretUsernameField,
secretsPasswordField: this.opts.secretPasswordField,
assumeRoleArn: this.opts.assumeRole?.roleArn,
},
};
}
}

/** DockerCredential defined by a set of ECR repositories in the same account & region */
class EcrDockerCredential extends DockerCredential {
public readonly registryDomain: string;

constructor(private readonly repositories: ecr.IRepository[], private readonly opts: EcrDockerCredentialOptions) {
super(opts.usages);

if (repositories.length === 0) {
throw new Error('must supply at least one `ecr.IRepository` to create an `EcrDockerCredential`');
}
this.registryDomain = Fn.select(0, Fn.split('/', repositories[0].repositoryUri));
}

public grantRead(grantee: iam.IGrantable, usage: DockerCredentialUsage) {
if (!this._applicableForUsage(usage)) { return; }

if (this.opts.assumeRole) {
grantee.grantPrincipal.addToPrincipalPolicy(new iam.PolicyStatement({
actions: ['sts:AssumeRole'],
resources: [this.opts.assumeRole.roleArn],
}));
}
const role = this.opts.assumeRole ?? grantee;
this.repositories.forEach(repo => repo.grantPull(role));
}

public _renderCdkAssetsConfig(): DockerCredentialCredentialSource {
return {
[this.registryDomain]: {
ecrRepository: true,
assumeRoleArn: this.opts.assumeRole?.roleArn,
},
};
}
}

/** Format for the CDK assets config. See the cdk-assets `DockerDomainCredentialSource` */
interface DockerCredentialCredentialSource {
readonly secretsManagerSecretId?: string;
readonly secretsUsernameField?: string;
readonly secretsPasswordField?: string;
readonly ecrRepository?: boolean;
readonly assumeRoleArn?: string;
}

/**
* Creates a set of OS-specific buildspec installation commands for setting up the given
* registries and associated credentials.
*
* @param registries - Registries to configure credentials for. It is an error to provide
* multiple registries for the same domain.
* @param osType - (optional) Defaults to Linux.
* @returns An array of commands to configure cdk-assets to use these credentials.
*/
export function dockerCredentialsInstallCommands(
usage: DockerCredentialUsage,
registries?: DockerCredential[],
osType?: ec2.OperatingSystemType): string[] {

const relevantRegistries = (registries ?? []).filter(reg => reg._applicableForUsage(usage));
if (!relevantRegistries || relevantRegistries.length === 0) { return []; }

const domainCredentials = relevantRegistries.reduce(function (map: Record<string, any>, registry) {
Object.assign(map, registry._renderCdkAssetsConfig());
return map;
}, {});
const cdkAssetsConfigFile = {
version: '1.0',
domainCredentials,
};

if (osType === ec2.OperatingSystemType.WINDOWS) {
return [
'mkdir %USERPROFILE%\\.cdk',
`echo '${JSON.stringify(cdkAssetsConfigFile)}' > %USERPROFILE%\\.cdk\\cdk-docker-creds.json`,
];
} else {
return [
'mkdir $HOME/.cdk',
`echo '${JSON.stringify(cdkAssetsConfigFile)}' > $HOME/.cdk/cdk-docker-creds.json`,
];
}
}
3 changes: 2 additions & 1 deletion packages/@aws-cdk/pipelines/lib/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@ export * from './pipeline';
export * from './stage';
export * from './synths';
export * from './actions';
export * from './validation';
export * from './docker-credentials';
export * from './validation';
Loading

0 comments on commit 973ded6

Please sign in to comment.