Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(codebuild): add functionality to allow using private registry an… #2796

Closed
wants to merge 4 commits into from
Closed

feat(codebuild): add functionality to allow using private registry an… #2796

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b195ab3
feat(codebuild): add functionality to allow using private registry an…
Kaixiang-AWS Jun 7, 2019
2eafda7
Merge remote-tracking branch 'upstream/master' into private-registry
Kaixiang-AWS Jun 20, 2019
aba3103
feat(codebuild): add functionality to allow using private registry an…
Kaixiang-AWS Jun 20, 2019
08cf809
Fix test error
Kaixiang-AWS Jun 21, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions packages/@aws-cdk/aws-codebuild/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -175,8 +175,7 @@ of the constants such as `WindowsBuildImage.WIN_SERVER_CORE_2016_BASE` or
Alternatively, you can specify a custom image using one of the static methods on
`XxxBuildImage`:

* Use `.fromDockerHub(image)` to reference an image publicly available in Docker
Hub.
* Use `.fromDockerRegistry(image[, secretsManagerCredential])` to reference an image in any public or private Docker registry.
* Use `.fromEcrRepository(repo[, tag])` to reference an image available in an
ECR repository.
* Use `.fromAsset(this, id, { directory: dir })` to use an image created from a
Expand All @@ -190,6 +189,10 @@ The following example shows how to define an image from an ECR repository:

[ECR example](./test/integ.ecr.lit.ts)

The following example shows how to define an image from a private docker registry:

[Docker Registry example](./test/integ.docker-registry.lit.ts)

## Events

CodeBuild projects can be used either as a source for events or be triggered
Expand Down
94 changes: 57 additions & 37 deletions packages/@aws-cdk/aws-codebuild/lib/project.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import ecr = require('@aws-cdk/aws-ecr');
import events = require('@aws-cdk/aws-events');
import iam = require('@aws-cdk/aws-iam');
import kms = require('@aws-cdk/aws-kms');
import secretsmanager = require('@aws-cdk/aws-secretsmanager');
import { Aws, Construct, IResource, Resource, Stack, Token } from '@aws-cdk/cdk';
import { BuildArtifacts, CodePipelineBuildArtifacts, NoBuildArtifacts } from './artifacts';
import { Cache } from './cache';
Expand Down Expand Up @@ -813,6 +814,17 @@ export class Project extends ProjectBase {
return p;
}

private attachEcrPermission() {
this.addToRolePolicy(new iam.PolicyStatement()
.addAllResources()
.addActions(
'ecr:GetAutheticationToken',
'ecr:GetDownloadUrlForLayer',
'ecr:BatchGetImage',
'ecr:BatchCheckLayerAvailability'
));
}

private renderEnvironment(env: BuildEnvironment = {},
projectVars: { [name: string]: BuildEnvironmentVariable } = {}): CfnProject.EnvironmentProperty {
const vars: { [name: string]: BuildEnvironmentVariable } = {};
Expand All @@ -830,6 +842,10 @@ export class Project extends ProjectBase {

const hasEnvironmentVars = Object.keys(vars).length > 0;

if (isECRImage(this.buildImage.imageId)) {
this.attachEcrPermission();
}

const errors = this.buildImage.validate(env);
if (errors.length > 0) {
throw new Error("Invalid CodeBuild environment: " + errors.join('\n'));
Expand All @@ -838,6 +854,12 @@ export class Project extends ProjectBase {
return {
type: this.buildImage.type,
image: this.buildImage.imageId,
imagePullCredentialsType: this.buildImage.imagePullCredentialsType,
registryCredential: this.buildImage.secretsManagerCredential ?
{
credentialProvider: 'SECRETS_MANAGER',
credential: this.buildImage.secretsManagerCredential.secretArn
} : undefined,
privilegedMode: env.privileged || false,
computeType: env.computeType || this.buildImage.defaultComputeType,
environmentVariables: !hasEnvironmentVars ? undefined : Object.keys(vars).map(name => ({
Expand Down Expand Up @@ -945,6 +967,11 @@ export enum ComputeType {
Large = 'BUILD_GENERAL1_LARGE'
}

export enum ImagePullCredentialsType {
CodeBuild = 'CODEBUILD',
ServiceRole = 'SERVICE_ROLE'
}

export interface BuildEnvironment {
/**
* The image used for the builds.
Expand Down Expand Up @@ -1003,6 +1030,16 @@ export interface IBuildImage {
*/
readonly defaultComputeType: ComputeType;

/**
* The type of credentials AWS CodeBuild uses to pull images in your build.
*/
readonly imagePullCredentialsType?: ImagePullCredentialsType;

/**
* The credentials for access to a private registry.
*/
readonly secretsManagerCredential?: secretsmanager.ISecret;

/**
* Allows the image a chance to validate whether the passed configuration is correct.
*
Expand All @@ -1023,7 +1060,7 @@ export interface IBuildImage {
*
* You can also specify a custom image using one of the static methods:
*
* - LinuxBuildImage.fromDockerHub(image)
* - LinuxBuildImage.fromDockerRegistry(image[, secretsManagerCredential])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@skinny85 @SoManyHs @rix0rrr do you think it makes sense to merge this API with the one we have for ECS?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In that case, we'd probably need some generic modeling of Docker images, using @aws-cdk/docker or something.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Discussed offline. We decided not create a general Docker image API. Given that, I believe this code is fine - what do you think @eladb ?

* - LinuxBuildImage.fromEcrRepository(repo[, tag])
* - LinuxBuildImage.fromAsset(parent, id, props)
*
Expand Down Expand Up @@ -1067,8 +1104,8 @@ export class LinuxBuildImage implements IBuildImage {
/**
* @returns a Linux build image from a Docker Hub image.
*/
public static fromDockerHub(name: string): LinuxBuildImage {
return new LinuxBuildImage(name);
public static fromDockerRegistry(name: string, secretsManagerCredential?: secretsmanager.ISecret): LinuxBuildImage {
return new LinuxBuildImage(name, ImagePullCredentialsType.ServiceRole, secretsManagerCredential);
}

/**
Expand All @@ -1083,29 +1120,24 @@ export class LinuxBuildImage implements IBuildImage {
* @param tag Image tag (default "latest")
*/
public static fromEcrRepository(repository: ecr.IRepository, tag: string = 'latest'): LinuxBuildImage {
const image = new LinuxBuildImage(repository.repositoryUriForTag(tag));
repository.addToResourcePolicy(ecrAccessForCodeBuildService());
return image;
return new LinuxBuildImage(repository.repositoryUriForTag(tag), ImagePullCredentialsType.ServiceRole);
}

/**
* Uses an Docker image asset as a Linux build image.
*/
public static fromAsset(scope: Construct, id: string, props: DockerImageAssetProps): LinuxBuildImage {
const asset = new DockerImageAsset(scope, id, props);
const image = new LinuxBuildImage(asset.imageUri);

// allow this codebuild to pull this image (CodeBuild doesn't use a role, so
// we can't use `asset.grantUseImage()`.
asset.repository.addToResourcePolicy(ecrAccessForCodeBuildService());

return image;
return new LinuxBuildImage(asset.imageUri, ImagePullCredentialsType.ServiceRole);
}

public readonly type = 'LINUX_CONTAINER';
public readonly defaultComputeType = ComputeType.Small;

private constructor(public readonly imageId: string) {
private constructor(
public readonly imageId: string,
public readonly imagePullCredentialsType?: ImagePullCredentialsType,
public readonly secretsManagerCredential?: secretsmanager.ISecret) {
}

public validate(_: BuildEnvironment): string[] {
Expand Down Expand Up @@ -1148,7 +1180,7 @@ export class LinuxBuildImage implements IBuildImage {
*
* You can also specify a custom image using one of the static methods:
*
* - WindowsBuildImage.fromDockerHub(image)
* - WindowsBuildImage.fromDockerRegistry(image[, secretsManagerCredential])
* - WindowsBuildImage.fromEcrRepository(repo[, tag])
* - WindowsBuildImage.fromAsset(parent, id, props)
*
Expand All @@ -1160,8 +1192,8 @@ export class WindowsBuildImage implements IBuildImage {
/**
* @returns a Windows build image from a Docker Hub image.
*/
public static fromDockerHub(name: string): WindowsBuildImage {
return new WindowsBuildImage(name);
public static fromDockerRegistry(name: string, secretsManagerCredential?: secretsmanager.ISecret): WindowsBuildImage {
return new WindowsBuildImage(name, ImagePullCredentialsType.ServiceRole, secretsManagerCredential);
}

/**
Expand All @@ -1176,28 +1208,23 @@ export class WindowsBuildImage implements IBuildImage {
* @param tag Image tag (default "latest")
*/
public static fromEcrRepository(repository: ecr.IRepository, tag: string = 'latest'): WindowsBuildImage {
const image = new WindowsBuildImage(repository.repositoryUriForTag(tag));
repository.addToResourcePolicy(ecrAccessForCodeBuildService());
return image;
return new WindowsBuildImage(repository.repositoryUriForTag(tag), ImagePullCredentialsType.ServiceRole);
}

/**
* Uses an Docker image asset as a Windows build image.
*/
public static fromAsset(scope: Construct, id: string, props: DockerImageAssetProps): WindowsBuildImage {
const asset = new DockerImageAsset(scope, id, props);
const image = new WindowsBuildImage(asset.imageUri);

// allow this codebuild to pull this image (CodeBuild doesn't use a role, so
// we can't use `asset.grantUseImage()`.
asset.repository.addToResourcePolicy(ecrAccessForCodeBuildService());

return image;
return new WindowsBuildImage(asset.imageUri, ImagePullCredentialsType.ServiceRole);
}
public readonly type = 'WINDOWS_CONTAINER';
public readonly defaultComputeType = ComputeType.Medium;

private constructor(public readonly imageId: string) {
private constructor(
public readonly imageId: string,
public readonly imagePullCredentialsType?: ImagePullCredentialsType,
public readonly secretsManagerCredential?: secretsmanager.ISecret) {
}

public validate(buildEnvironment: BuildEnvironment): string[] {
Expand Down Expand Up @@ -1287,13 +1314,6 @@ function extendBuildSpec(buildSpec: any, extend: any) {
}
}

function ecrAccessForCodeBuildService(): iam.PolicyStatement {
return new iam.PolicyStatement()
.describe('CodeBuild')
.addServicePrincipal('codebuild.amazonaws.com')
.addActions(
'ecr:GetDownloadUrlForLayer',
'ecr:BatchGetImage',
'ecr:BatchCheckLayerAvailability'
);
function isECRImage(imageUri: string) {
return /^(.+).dkr.ecr.(.+).amazonaws.com[.]{0,1}[a-z]{0,3}\/([^:]+):?.*$/.test(imageUri);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should ensure imageUri can be parsed using Token.unresolved

}
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-codebuild/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@
"@aws-cdk/aws-iam": "^0.33.0",
"@aws-cdk/aws-kms": "^0.33.0",
"@aws-cdk/aws-s3": "^0.33.0",
"@aws-cdk/aws-secretsmanager": "^0.33.0",
"@aws-cdk/cdk": "^0.33.0"
},
"homepage": "https://github.com/awslabs/aws-cdk",
Expand All @@ -101,6 +102,7 @@
"@aws-cdk/aws-iam": "^0.33.0",
"@aws-cdk/aws-kms": "^0.33.0",
"@aws-cdk/aws-s3": "^0.33.0",
"@aws-cdk/aws-secretsmanager": "^0.33.0",
"@aws-cdk/cdk": "^0.33.0"
},
"engines": {
Expand Down
38 changes: 11 additions & 27 deletions packages/@aws-cdk/aws-codebuild/test/integ.docker-asset.lit.expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,33 +39,6 @@
]
}
]
},
"PolicyDocument": {
"Statement": [
{
"Action": [
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
],
"Effect": "Allow",
"Principal": {
"Service": {
"Fn::Join": [
"",
[
"codebuild.",
{
"Ref": "AWS::URLSuffix"
}
]
]
}
},
"Sid": "CodeBuild"
}
],
"Version": "2012-10-17"
}
},
"DependsOn": [
Expand Down Expand Up @@ -261,6 +234,16 @@
"Properties": {
"PolicyDocument": {
"Statement": [
{
"Action": [
"ecr:GetAutheticationToken",
"ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage",
"ecr:BatchCheckLayerAvailability"
],
"Effect": "Allow",
"Resource": "*"
},
{
"Action": [
"logs:CreateLogGroup",
Expand Down Expand Up @@ -438,6 +421,7 @@
]
]
},
"ImagePullCredentialsType": "SERVICE_ROLE",
"PrivilegedMode": false,
"Type": "LINUX_CONTAINER"
},
Expand Down
Loading