Skip to content

Commit

Permalink
refactor(assets): API cleanups (#2910)
Browse files Browse the repository at this point in the history
Reorganize asset-related modules as follows:

- @aws-cdk/assets: core types related to assets such as filesystem operations and staging
- @aws-cdk/aws-s3-assets: file assets deployed to s3
- @aws-cdk/aws-ecr-assets: docker assets deployed to ecr
- @aws-cdk/assets-docker: deprecated

BREAKING CHANGE: `AssetProps.packaging` has been removed and is now automatically discovered based on the file type.
* **assets:** `ZipDirectoryAsset` has been removed, use `aws-s3-assets.Asset`.
* **assets:** `FileAsset` has been removed, use `aws-s3-assets.Asset`.
* **lambda:** `Code.directory` and `Code.file` have been removed. Use `Code.asset`.
* **assets-docker:** The module has been renamed to **aws-ecr-assets**
  • Loading branch information
Elad Ben-Israel committed Jun 19, 2019
1 parent 1aa0589 commit 83eee09
Show file tree
Hide file tree
Showing 65 changed files with 1,209 additions and 372 deletions.
57 changes: 3 additions & 54 deletions packages/@aws-cdk/assets-docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,62 +3,11 @@

---

![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge)
![Stability: Deprecated](https://img.shields.io/badge/stability-Deprecated-critical.svg?style=for-the-badge)

> This API is still under active development and subject to non-backward
> compatible changes or removal in any future version. Use of the API is not recommended in production
> environments. Experimental APIs are not subject to the Semantic Versioning model.
> This API may emit warnings. Backward compatibility is not guaranteed.
---
<!--END STABILITY BANNER-->

This module allows bundling Docker images as assets.

Images are built from a local Docker context directory (with a `Dockerfile`),
uploaded to ECR by the CDK toolkit and/or your app's CI-CD pipeline, and can be
naturally referenced in your CDK app.

```typescript
import { DockerImageAsset } from '@aws-cdk/assets-docker';

const asset = new DockerImageAsset(this, 'MyBuildImage', {
directory: path.join(__dirname, 'my-image')
});
```

The directory `my-image` must include a `Dockerfile`.

This will instruct the toolkit to build a Docker image from `my-image`, push it
to an AWS ECR repository and wire the name of the repository as CloudFormation
parameters to your stack.

Use `asset.imageUri` to reference the image (it includes both the ECR image URL
and tag.

You can optionally pass build args to the `docker build` command by specifying
the `buildArgs` property:

```typescript
const asset = new DockerImageAsset(this, 'MyBuildImage', {
directory: path.join(__dirname, 'my-image'),
buildArgs: {
HTTP_PROXY: 'http://10.20.30.2:1234'
}
});
```

### Pull Permissions

Depending on the consumer of your image asset, you will need to make sure
the principal has permissions to pull the image.

In most cases, you should use the `asset.repository.grantPull(principal)`
method. This will modify the IAM policy of the principal to allow it to
pull images from this repository.

If the pulling principal is not in the same account or is an AWS service that
doesn't assume a role in your account (e.g. AWS CodeBuild), pull permissions
must be granted on the __resource policy__ (and not on the principal's policy).
To do that, you can use `asset.repository.addToResourcePolicy(statement)` to
grant the desired principal the following permissions: "ecr:GetDownloadUrlForLayer",
"ecr:BatchGetImage" and "ecr:BatchCheckLayerAvailability".
Deprecated: @aws-cdk/aws-ecr-assets
3 changes: 2 additions & 1 deletion packages/@aws-cdk/assets-docker/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export * from './image-asset';
// tslint:disable-next-line:no-console
console.error('the @aws-cdk/assets-docker is deprecated. use @aws-cdk/aws-ecr-assets');
36 changes: 5 additions & 31 deletions packages/@aws-cdk/assets-docker/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
{
"name": "@aws-cdk/assets-docker",
"version": "0.34.0",
"description": "Docker image assets",
"deprecated": "moved to @aws-cdk/aws-ecr-assets",
"description": "deprecated: moved to @aws-cdk/aws-ecr-assets",
"main": "lib/index.js",
"types": "lib/index.d.ts",
"jsii": {
Expand Down Expand Up @@ -57,40 +58,13 @@
},
"license": "Apache-2.0",
"devDependencies": {
"@aws-cdk/assert": "^0.34.0",
"@types/proxyquire": "^1.3.28",
"aws-cdk": "^0.34.0",
"cdk-build-tools": "^0.34.0",
"cdk-integ-tools": "^0.34.0",
"pkglint": "^0.34.0",
"proxyquire": "^2.1.0"
},
"dependencies": {
"@aws-cdk/assets": "^0.34.0",
"@aws-cdk/aws-cloudformation": "^0.34.0",
"@aws-cdk/aws-ecr": "^0.34.0",
"@aws-cdk/aws-iam": "^0.34.0",
"@aws-cdk/aws-lambda": "^0.34.0",
"@aws-cdk/aws-s3": "^0.34.0",
"@aws-cdk/cdk": "^0.34.0",
"@aws-cdk/cx-api": "^0.34.0"
"pkglint": "^0.34.0"
},
"homepage": "https://github.com/awslabs/aws-cdk",
"peerDependencies": {
"@aws-cdk/assets": "^0.34.0",
"@aws-cdk/aws-cloudformation": "^0.34.0",
"@aws-cdk/aws-ecr": "^0.34.0",
"@aws-cdk/aws-iam": "^0.34.0",
"@aws-cdk/aws-lambda": "^0.34.0",
"@aws-cdk/aws-s3": "^0.34.0",
"@aws-cdk/cdk": "^0.34.0",
"@aws-cdk/cx-api": "^0.34.0"
},
"nyc": {
"statements": 70
},
"engines": {
"node": ">= 8.10.0"
},
"stability": "experimental"
}
"stability": "deprecated"
}
81 changes: 1 addition & 80 deletions packages/@aws-cdk/assets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,84 +12,5 @@
---
<!--END STABILITY BANNER-->

Assets are local files or directories which are needed by a CDK app. A common
example is a directory which contains the handler code for a Lambda function,
but assets can represent any artifact that is needed for the app's operation.
This module includes core classes related to CDK assets.

When deploying a CDK app that includes constructs with assets, the CDK toolkit
will first upload all the assets to S3, and only then deploy the stacks. The S3
locations of the uploaded assets will be passed in as CloudFormation Parameters
to the relevant stacks.

The following JavaScript example defines an directory asset which is archived as
a .zip file and uploaded to S3 during deployment.

[Example of a ZipDirectoryAsset](./test/integ.assets.directory.lit.ts)

The following JavaScript example defines a file asset, which is uploaded as-is
to an S3 bucket during deployment.

[Example of a FileAsset](./test/integ.assets.file.lit.ts)

## Attributes

`Asset` constructs expose the following deploy-time attributes:

* `s3BucketName` - the name of the assets S3 bucket.
* `s3ObjectKey` - the S3 object key of the asset file (whether it's a file or a zip archive)
* `s3Url` - the S3 URL of the asset (i.e. https://s3.us-east-1.amazonaws.com/mybucket/mykey.zip)

In the following example, the various asset attributes are exported as stack outputs:

[Example of referencing an asset](./test/integ.assets.refs.lit.ts)

## Permissions

IAM roles, users or groups which need to be able to read assets in runtime will should be
granted IAM permissions. To do that use the `asset.grantRead(principal)` method:

The following examples grants an IAM group read permissions on an asset:

[Example of granting read access to an asset](./test/integ.assets.permissions.lit.ts)

## How does it work?

When an asset is defined in a construct, a construct metadata entry
`aws:cdk:asset` is emitted with instructions on where to find the asset and what
type of packaging to perform (`zip` or `file`). Furthermore, the synthesized
CloudFormation template will also include two CloudFormation parameters: one for
the asset's bucket and one for the asset S3 key. Those parameters are used to
reference the deploy-time values of the asset (using `{ Ref: "Param" }`).

Then, when the stack is deployed, the toolkit will package the asset (i.e. zip
the directory), calculate an MD5 hash of the contents and will render an S3 key
for this asset within the toolkit's asset store. If the file doesn't exist in
the asset store, it is uploaded during deployment.

> The toolkit's asset store is an S3 bucket created by the toolkit for each
environment the toolkit operates in (environment = account + region).

Now, when the toolkit deploys the stack, it will set the relevant CloudFormation
Parameters to point to the actual bucket and key for each asset.

## CloudFormation Resource Metadata

> NOTE: This section is relevant for authors of AWS Resource Constructs.
In certain situations, it is desirable for tools to be able to know that a certain CloudFormation
resource is using a local asset. For example, SAM CLI can be used to invoke AWS Lambda functions
locally for debugging purposes.

To enable such use cases, external tools will consult a set of metadata entries on AWS CloudFormation
resources:

- `aws:asset:path` points to the local path of the asset.
- `aws:asset:property` is the name of the resource property where the asset is used

Using these two metadata entries, tools will be able to identify that assets are used
by a certain resource, and enable advanced local experiences.

To add these metadata entries to a resource, use the
`asset.addResourceMetadata(resource, property)` method.

See https://github.com/awslabs/aws-cdk/issues/1432 for more details
20 changes: 20 additions & 0 deletions packages/@aws-cdk/assets/lib/api.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/**
* Common interface for all assets.
*/
export interface IAsset {
/**
* A hash of the source of this asset, which is available at construction time. As this is a plain
* string, it can be used in construct IDs in order to enforce creation of a new resource when
* the content hash has changed.
*/
readonly sourceHash: string;

/**
* A hash of the bundle for of this asset, which is only available at deployment time. As this is
* a late-bound token, it may not be used in construct IDs, but can be passed as a resource
* property in order to force a change on a resource when an asset is effectively updated. This is
* more reliable than `sourceHash` in particular for assets which bundling phase involve external
* resources that can change over time (such as Docker image builds).
*/
readonly artifactHash: string;
}
2 changes: 1 addition & 1 deletion packages/@aws-cdk/assets/lib/index.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
export * from './asset';
export * from './fs/copy-options';
export * from './fs/follow-mode';
export * from './staging';
export * from './api';
2 changes: 1 addition & 1 deletion packages/@aws-cdk/assets/lib/staging.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ export interface StagingProps extends CopyOptions {
* Stages a file or directory from a location on the file system into a staging
* directory.
*
* This is controlled by the context key 'aws:cdk:asset-staging-dir' and enabled
* This is controlled by the context key 'aws:cdk:asset-staging' and enabled
* by the CLI by default in order to ensure that when the CDK app exists, all
* assets are available for deployment. Otherwise, if an app references assets
* in temporary locations, those will not be available when it exists (see
Expand Down
4 changes: 0 additions & 4 deletions packages/@aws-cdk/assets/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,16 +72,12 @@
"ts-mock-imports": "^1.2.3"
},
"dependencies": {
"@aws-cdk/aws-iam": "^0.34.0",
"@aws-cdk/aws-s3": "^0.34.0",
"@aws-cdk/cdk": "^0.34.0",
"@aws-cdk/cx-api": "^0.34.0",
"minimatch": "^3.0.4"
},
"homepage": "https://github.com/awslabs/aws-cdk",
"peerDependencies": {
"@aws-cdk/aws-iam": "^0.34.0",
"@aws-cdk/aws-s3": "^0.34.0",
"@aws-cdk/cdk": "^0.34.0",
"@aws-cdk/cx-api": "^0.34.0"
},
Expand Down
60 changes: 60 additions & 0 deletions packages/@aws-cdk/assets/test/test.staging.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import { App, Stack } from '@aws-cdk/cdk';
import cxapi = require('@aws-cdk/cx-api');
import fs = require('fs');
import { Test } from 'nodeunit';
import path = require('path');
import { Staging } from '../lib';

export = {
'base case'(test: Test) {
// GIVEN
const stack = new Stack();
const sourcePath = path.join(__dirname, 'fs', 'fixtures', 'test1');

// WHEN
const staging = new Staging(stack, 's1', { sourcePath });

test.deepEqual(staging.sourceHash, '2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00');
test.deepEqual(staging.sourcePath, sourcePath);
test.deepEqual(stack.resolve(staging.stagedPath), 'asset.2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00');
test.done();
},

'staging can be disabled through context'(test: Test) {
// GIVEN
const stack = new Stack();
stack.node.setContext(cxapi.DISABLE_ASSET_STAGING_CONTEXT, true);
const sourcePath = path.join(__dirname, 'fs', 'fixtures', 'test1');

// WHEN
const staging = new Staging(stack, 's1', { sourcePath });

test.deepEqual(staging.sourceHash, '2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00');
test.deepEqual(staging.sourcePath, sourcePath);
test.deepEqual(stack.resolve(staging.stagedPath), sourcePath);
test.done();
},

'files are copied to the output directory during synth'(test: Test) {
// GIVEN
const app = new App();
const stack = new Stack(app, 'stack');
const directory = path.join(__dirname, 'fs', 'fixtures', 'test1');
const file = path.join(__dirname, 'fs', 'fixtures.tar.gz');

// WHEN
new Staging(stack, 's1', { sourcePath: directory });
new Staging(stack, 'file', { sourcePath: file });

// THEN
const assembly = app.synth();
test.deepEqual(fs.readdirSync(assembly.directory), [
'asset.2f37f937c51e2c191af66acf9b09f548926008ec68c575bd2ee54b6e997c0e00',
'asset.af10ac04b3b607b0f8659c8f0cee8c343025ee75baf0b146f10f0e5311d2c46b.gz',
'cdk.out',
'manifest.json',
'stack.template.json'
]);
test.done();
}
};
2 changes: 1 addition & 1 deletion packages/@aws-cdk/aws-codebuild/lib/project.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import { DockerImageAsset, DockerImageAssetProps } from '@aws-cdk/assets-docker';
import cloudwatch = require('@aws-cdk/aws-cloudwatch');
import ec2 = require('@aws-cdk/aws-ec2');
import ecr = require('@aws-cdk/aws-ecr');
import { DockerImageAsset, DockerImageAssetProps } from '@aws-cdk/aws-ecr-assets';
import events = require('@aws-cdk/aws-events');
import iam = require('@aws-cdk/aws-iam');
import kms = require('@aws-cdk/aws-kms');
Expand Down
8 changes: 5 additions & 3 deletions packages/@aws-cdk/aws-codebuild/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -77,8 +77,9 @@
"pkglint": "^0.34.0"
},
"dependencies": {
"@aws-cdk/aws-s3-assets": "^0.34.0",
"@aws-cdk/assets": "^0.34.0",
"@aws-cdk/assets-docker": "^0.34.0",
"@aws-cdk/aws-ecr-assets": "^0.34.0",
"@aws-cdk/aws-cloudwatch": "^0.34.0",
"@aws-cdk/aws-codecommit": "^0.34.0",
"@aws-cdk/aws-ec2": "^0.34.0",
Expand All @@ -91,8 +92,9 @@
},
"homepage": "https://github.com/awslabs/aws-cdk",
"peerDependencies": {
"@aws-cdk/aws-s3-assets": "^0.34.0",
"@aws-cdk/assets": "^0.34.0",
"@aws-cdk/assets-docker": "^0.34.0",
"@aws-cdk/aws-ecr-assets": "^0.34.0",
"@aws-cdk/aws-cloudwatch": "^0.34.0",
"@aws-cdk/aws-codecommit": "^0.34.0",
"@aws-cdk/aws-ec2": "^0.34.0",
Expand All @@ -112,4 +114,4 @@
]
},
"stability": "experimental"
}
}
17 changes: 17 additions & 0 deletions packages/@aws-cdk/aws-ecr-assets/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
*.js
*.js.map
*.d.ts
node_modules
dist
tsconfig.json
tslint.json

.LAST_BUILD
.nyc_output
coverage

.jsii

.nycrc
.LAST_PACKAGE
*.snk
Loading

0 comments on commit 83eee09

Please sign in to comment.