Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(lambda): docker platform for architecture #16858

Merged
merged 8 commits into from
Oct 11, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions packages/@aws-cdk/aws-lambda-go/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ All other properties of `lambda.Function` are supported, see also the [AWS Lambd
By default the following environment variables are set for you:

* `GOOS=linux`
* `GOARCH=amd64`
* `GOARCH`: based on the target architecture of the Lambda function
* `GO111MODULE=on`

Use the `environment` prop to define additional environment variables when go runs:
Expand All @@ -124,7 +124,7 @@ new lambda.GoFunction(this, 'handler', {

## Local Bundling

If `Go` is installed locally and the version is >= `go1.11` then it will be used to bundle your code in your environment. Otherwise, bundling will happen in a [Lambda compatible Docker container](https://hub.docker.com/layers/lambci/lambda/build-go1.x/images/sha256-e14dab718ed0bb06b2243825c5993e494a6969de7c01754ad7e80dacfce9b0cf?context=explore).
If `Go` is installed locally and the version is >= `go1.11` then it will be used to bundle your code in your environment. Otherwise, bundling will happen in a [Lambda compatible Docker container](https://gallery.ecr.aws/sam/build-go1.x) with the Docker platform based on the target architecture of the Lambda function.

For macOS the recommended approach is to install `Go` as Docker volume performance is really poor.

Expand Down
10 changes: 8 additions & 2 deletions packages/@aws-cdk/aws-lambda-go/lib/bundling.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import * as os from 'os';
import * as path from 'path';
import { AssetCode, Code, Runtime } from '@aws-cdk/aws-lambda';
import { Architecture, AssetCode, Code, Runtime } from '@aws-cdk/aws-lambda';
import * as cdk from '@aws-cdk/core';
import { BundlingOptions } from './types';
import { exec, findUp, getGoBuildVersion } from './util';
Expand Down Expand Up @@ -55,6 +55,11 @@ export interface BundlingProps extends BundlingOptions {
* The runtime of the lambda function
*/
readonly runtime: Runtime;

/**
* The system architecture of the lambda function
*/
readonly architecture: Architecture;
}

/**
Expand Down Expand Up @@ -104,7 +109,7 @@ export class Bundling implements cdk.BundlingOptions {
const environment = {
CGO_ENABLED: cgoEnabled,
GO111MODULE: 'on',
GOARCH: 'amd64',
GOARCH: props.architecture.dockerPlatform.split('/')[1],
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@corymhall what about this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even better!

GOOS: 'linux',
...props.environment,
};
Expand All @@ -117,6 +122,7 @@ export class Bundling implements cdk.BundlingOptions {
...props.buildArgs ?? {},
IMAGE: Runtime.GO_1_X.bundlingImage.image, // always use the GO_1_X build image
},
platform: props.architecture.dockerPlatform,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You probably need to remove the GOARCH environment variable on line 112 above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

})
: cdk.DockerImage.fromRegistry('dummy'); // Do not build if we don't need to

Expand Down
2 changes: 2 additions & 0 deletions packages/@aws-cdk/aws-lambda-go/lib/function.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,7 @@ export class GoFunction extends lambda.Function {
}

const runtime = props.runtime ?? lambda.Runtime.PROVIDED_AL2;
const architecture = props.architecture ?? lambda.Architecture.X86_64;

super(scope, id, {
...props,
Expand All @@ -112,6 +113,7 @@ export class GoFunction extends lambda.Function {
...props.bundling ?? {},
entry,
runtime,
architecture,
moduleDir,
}),
handler: 'bootstrap', // setting name to bootstrap so that the 'provided' runtime can also be used
Expand Down
35 changes: 28 additions & 7 deletions packages/@aws-cdk/aws-lambda-go/test/bundling.test.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,28 @@
import * as child_process from 'child_process';
import * as os from 'os';
import * as path from 'path';
import { Code, Runtime } from '@aws-cdk/aws-lambda';
import { Architecture, Code, Runtime } from '@aws-cdk/aws-lambda';
import { AssetHashType, DockerImage } from '@aws-cdk/core';
import { Bundling } from '../lib/bundling';
import * as util from '../lib/util';

jest.spyOn(Code, 'fromAsset');
const fromAssetMock = jest.spyOn(DockerImage, 'fromBuild');
let getGoBuildVersionMock = jest.spyOn(util, 'getGoBuildVersion');

beforeEach(() => {
jest.clearAllMocks();
jest.resetAllMocks();
Bundling.clearRunsLocallyCache();
getGoBuildVersionMock.mockReturnValue(true);
fromAssetMock.mockReturnValue({

jest.spyOn(Code, 'fromAsset');

jest.spyOn(DockerImage, 'fromBuild').mockReturnValue({
image: 'built-image',
cp: () => 'built-image',
run: () => {},
toJSON: () => 'build-image',
});

getGoBuildVersionMock.mockReturnValue(true);
});

const moduleDir = '/project/go.mod';
Expand All @@ -30,6 +32,7 @@ test('bundling', () => {
Bundling.bundle({
entry,
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
forcedDockerBundling: true,
environment: {
Expand All @@ -55,12 +58,20 @@ test('bundling', () => {
],
}),
});

expect(DockerImage.fromBuild).toHaveBeenCalledWith(expect.stringMatching(/aws-lambda-go\/lib$/), expect.objectContaining({
buildArgs: expect.objectContaining({
IMAGE: expect.stringMatching(/build-go/),
}),
platform: 'linux/amd64',
}));
});

test('bundling with file as entry', () => {
Bundling.bundle({
entry: '/project/main.go',
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
});

Expand All @@ -81,6 +92,7 @@ test('bundling with file in subdirectory as entry', () => {
Bundling.bundle({
entry: '/project/cmd/api/main.go',
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
});

Expand All @@ -101,6 +113,7 @@ test('bundling with file other than main.go in subdirectory as entry', () => {
Bundling.bundle({
entry: '/project/cmd/api/api.go',
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
});

Expand All @@ -122,6 +135,7 @@ test('go with Windows paths', () => {
Bundling.bundle({
entry: 'C:\\my-project\\cmd\\api',
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir: 'C:\\my-project\\go.mod',
forcedDockerBundling: true,
});
Expand All @@ -141,13 +155,14 @@ test('with Docker build args', () => {
Bundling.bundle({
entry,
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
forcedDockerBundling: true,
buildArgs: {
HELLO: 'WORLD',
},
});
expect(fromAssetMock).toHaveBeenCalledWith(expect.stringMatching(/lib$/), expect.objectContaining({
expect(DockerImage.fromBuild).toHaveBeenCalledWith(expect.stringMatching(/aws-lambda-go\/lib$/), expect.objectContaining({
buildArgs: expect.objectContaining({
HELLO: 'WORLD',
}),
Expand All @@ -171,6 +186,7 @@ test('Local bundling', () => {
KEY: 'value',
},
runtime: Runtime.PROVIDED_AL2,
architecture: Architecture.X86_64,
});

expect(bundler.local).toBeDefined();
Expand All @@ -188,7 +204,7 @@ test('Local bundling', () => {
);

// Docker image is not built
expect(fromAssetMock).not.toHaveBeenCalled();
expect(DockerImage.fromBuild).not.toHaveBeenCalled();
});

test('Incorrect go version', () => {
Expand All @@ -198,6 +214,7 @@ test('Incorrect go version', () => {
entry,
moduleDir,
runtime: Runtime.PROVIDED_AL2,
architecture: Architecture.X86_64,
});

const tryBundle = bundler.local?.tryBundle('/outdir', { image: Runtime.GO_1_X.bundlingDockerImage });
Expand All @@ -211,6 +228,7 @@ test('Custom bundling docker image', () => {
entry,
moduleDir,
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
forcedDockerBundling: true,
dockerImage: DockerImage.fromRegistry('my-custom-image'),
});
Expand All @@ -227,6 +245,7 @@ test('Go build flags can be passed', () => {
Bundling.bundle({
entry,
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
environment: {
KEY: 'value',
Expand Down Expand Up @@ -258,6 +277,7 @@ test('AssetHashType can be specified', () => {
Bundling.bundle({
entry,
runtime: Runtime.GO_1_X,
architecture: Architecture.X86_64,
moduleDir,
environment: {
KEY: 'value',
Expand Down Expand Up @@ -291,6 +311,7 @@ test('with command hooks', () => {
entry,
moduleDir,
runtime: Runtime.PROVIDED_AL2,
architecture: Architecture.X86_64,
commandHooks: {
beforeBundling(inputDir: string, outputDir: string): string[] {
return [
Expand Down
3 changes: 2 additions & 1 deletion packages/@aws-cdk/aws-lambda-nodejs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ used by your function. Otherwise bundling will fail.
## Local bundling

If `esbuild` is available it will be used to bundle your code in your environment. Otherwise,
bundling will happen in a [Lambda compatible Docker container](https://gallery.ecr.aws/sam/build-nodejs12.x).
bundling will happen in a [Lambda compatible Docker container](https://gallery.ecr.aws/sam/build-nodejs12.x)
with the Docker platform based on the target architecture of the Lambda function.

For macOS the recommendend approach is to install `esbuild` as Docker volume performance is really poor.

Expand Down
8 changes: 7 additions & 1 deletion packages/@aws-cdk/aws-lambda-nodejs/lib/bundling.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import * as os from 'os';
import * as path from 'path';
import { AssetCode, Code, Runtime } from '@aws-cdk/aws-lambda';
import { Architecture, AssetCode, Code, Runtime } from '@aws-cdk/aws-lambda';
import * as cdk from '@aws-cdk/core';
import { EsbuildInstallation } from './esbuild-installation';
import { PackageManager } from './package-manager';
Expand Down Expand Up @@ -28,6 +28,11 @@ export interface BundlingProps extends BundlingOptions {
*/
readonly runtime: Runtime;

/**
* The system architecture of the lambda function
*/
readonly architecture: Architecture;

/**
* Path to project root
*/
Expand Down Expand Up @@ -99,6 +104,7 @@ export class Bundling implements cdk.BundlingOptions {
IMAGE: props.runtime.bundlingImage.image,
ESBUILD_VERSION: props.esbuildVersion ?? ESBUILD_MAJOR_VERSION,
},
platform: props.architecture.dockerPlatform,
})
: cdk.DockerImage.fromRegistry('dummy'); // Do not build if we don't need to

Expand Down
3 changes: 3 additions & 0 deletions packages/@aws-cdk/aws-lambda-nodejs/lib/function.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import * as fs from 'fs';
import * as path from 'path';
import * as lambda from '@aws-cdk/aws-lambda';
import { Architecture } from '@aws-cdk/aws-lambda';
import { Bundling } from './bundling';
import { PackageManager } from './package-manager';
import { BundlingOptions } from './types';
Expand Down Expand Up @@ -95,6 +96,7 @@ export class NodejsFunction extends lambda.Function {
const entry = path.resolve(findEntry(id, props.entry));
const handler = props.handler ?? 'handler';
const runtime = props.runtime ?? lambda.Runtime.NODEJS_14_X;
const architecture = props.architecture ?? Architecture.X86_64;
const depsLockFilePath = findLockFile(props.depsLockFilePath);
const projectRoot = props.projectRoot ?? path.dirname(depsLockFilePath);

Expand All @@ -105,6 +107,7 @@ export class NodejsFunction extends lambda.Function {
...props.bundling ?? {},
entry,
runtime,
architecture,
depsLockFilePath,
projectRoot,
}),
Expand Down
Loading