feat(lambda): allow running a build file #30196

bergjaak · 2024-05-14T21:23:19Z

Issue # (if applicable)

Reason for this change

This allows customers to execute an arbitrary build script as part of cdk synth, which will enable customer to use esbuild plugins. The rationale for this decision is given the issue that is linked above.

Description of changes

Expose the code field on the aws-lambda-nodejs construct, so that customers can specify code in ways other than bundling, which was the default and abstracted away from customers before this change.
Add a new static method on Code, namely Code.fromCustomCommand. This method takes in the commands to run an arbitrary script during cdk synthesis that the customer provides. The customer also provides the location of the output from the buildscript. Then this output is supplied to a lambda function.

Description of how you validated changes

manual testing (involving inspecting output in the AWS Lambda console and invoking the function), integration tests, and full unit test coverage of new changes.

Checklist

My code adheres to the CONTRIBUTING GUIDE and DESIGN GUIDELINES

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

aws-cdk-automation

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

colifran · 2024-05-15T16:07:40Z

packages/aws-cdk-lib/aws-lambda-nodejs/lib/function.ts

+    if (props.code !== undefined) {
+      if (props.handler === undefined) {
+        throw new Error('Cannot determine handler when `code` property is specified. Use `handler` property to specify a handler.');
+      }
+
+      super(scope, id, {
+        ...props,
        runtime,
-        architecture,
-        depsLockFilePath,
-        projectRoot,
-      }),
-      handler: handler.indexOf('.') !== -1 ? `${handler}` : `index.${handler}`,
-    });
+        code: props.code,
+        handler: props.handler!,
+      });
+    } else {


Can you help me understand why we need this new code block here? Why can't we just determine the handler in the same way we are doing it right now? Couldn't we just keep the existing code and do something where if code is supplied we use it, otherwise we do Bundling.bundle? So something like:

// Entry and defaults const entry = path.resolve(findEntry(id, props.entry)); const architecture = props.architecture ?? Architecture.X86_64; const depsLockFilePath = findLockFile(props.depsLockFilePath); const projectRoot = props.projectRoot ?? path.dirname(depsLockFilePath); const handler = props.handler ?? 'handler'; super(scope, id, { ...props, runtime, code: props.code ?? Bundling.bundle(scope, { ...props.bundling ?? {}, entry, runtime, architecture, depsLockFilePath, projectRoot, }), handler: handler.indexOf('.') !== -1 ? `${handler}` : `index.${handler}`, }); }

That's what I originally had, but I think it's a worse customer experience. Because there's nothing forcing the customer to name the file containing the exported handler index. It's entirely up to the customer to decide how they want to name that file, if they're using the Code.fromCustomCommand. Therefore, I think this is a better customer experience, even if it's a few more lines of code

I see where you're coming from but I'm not sure I agree. I think this construct has already established that the expected behavior is we will default the handler if it isn't supplied. Consider a customer that is using the nodejs function and they currently don't supply a handler and just use the default. Now, once this feature is released, they decide to use the new code property to run a custom build script. However, suddenly they're encountering errors since they don't set a handler. I recognize that it is trivial to just specify a handler, but it is a change from the expectation for the construct as a whole. Additionally, we generally try to avoid properties where we have to do a bunch of validation around if a property is set then another one must also be set. We consider it a bit of an anti-pattern. Our VPC construct is a more extreme example of this, but it shows where this can get to be hard to deal with when we have to start doing a ton of property validation - https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-ec2/lib/vpc.ts

Now, once this feature is released, they decide to use the new code property to run a custom build script. However, suddenly they're encountering errors since they don't set a handler.

My understanding is, if a customer does switch to using Code.fromCustomCommand and wants an error-free transition with default handler values, then their build script will need to create a file (either in a directory or a zip file) named index.js with an exported function called handler. Defaulting to handler makes sense in this case since it's unlikely that the customer would have changed the name of the function. However, if the customer never knew that the file that contained their lambda code, when deployed to AWS Lambda, was called index.js (since that's abstracted away by this construct), then wouldn't the customer be confused when their lambda fails to work because their build file didn't output the handler in a file called index.js? That seems more confusing to me. By throwing the Error here, we push the error that the customer may run into much earlier, rather than the customer's lambda failing at runtime after deploying successfully -- which could cause an outage if a customer is deploying directly to production.

basically, to avoid having an outage in production, if a customer is switching to using this Code.fromCustomCommand, the customer would need to know that their build file must output a file called index.js that exports a function called handler. That's not something that I think should be hidden from the customer. By requiring handler, we don't abstract away a detail that could potentially lead to an outage

I agree that this is more complicated, but the alternative is to push this complexity onto the customer after they've deployed their lambda

Happy to change my mind on this, and with the way you've explained it I agree. Doesn't make sense to let the customer fail during deploy time.

colifran · 2024-05-15T16:09:30Z

.vscode/launch.json

+      "request": "launch",
+      "args": ["jest", "--runTestsByPath", "/Users/bergjak/workplace/CDK/aws-cdk/packages/aws-cdk-lib/aws-lambda-nodejs/test/function.test.ts"],
+      "name": "debug unit test",
+      "program": "/opt/homebrew/bin/yarn",
+      "cwd": "/Users/bergjak/workplace/CDK/aws-cdk/packages/aws-cdk-lib",
+      "console": "integratedTerminal",
+      "sourceMaps": true,
+      "skipFiles": [ "<node_internals>/**/*" ],


yeah, I'll remove this in my final revision

colifran · 2024-05-30T16:55:31Z

packages/aws-cdk-lib/aws-lambda-nodejs/README.md

@@ -392,3 +392,65 @@ In situtations where this does not work, like Docker-in-Docker setups or when us
  },
 });
 ```
+
+## Running a custom build script as part of cdk synthesis


Nice README update!

colifran

Nice job with this!

aws-cdk-automation · 2024-05-30T22:33:27Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
Commit ID: 86c345b
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2024-05-30T22:33:43Z

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

### Issue # (if applicable) Closes aws#18470 ### Reason for this change This allows customers to execute an arbitrary build script as part of cdk synth, which will enable customer to use esbuild plugins. The rationale for this decision is given the issue that is linked above. ### Description of changes 1. Expose the code field on the `aws-lambda-nodejs` construct, so that customers can specify code in ways other than bundling, which was the default and abstracted away from customers before this change. 2. Add a new static method on Code, namely `Code.fromCustomCommand`. This method takes in the commands to run an arbitrary script during cdk synthesis that the customer provides. The customer also provides the location of the output from the buildscript. Then this output is supplied to a lambda function. ### Description of how you validated changes manual testing (involving inspecting output in the AWS Lambda console and invoking the function), integration tests, and full unit test coverage of new changes. ### Checklist - [X] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

aws-cdk-automation · 2024-07-25T21:08:32Z

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.

bergjaak added 3 commits May 13, 2024 16:52

start work

ad1bb87

feat(Lambda): allow running a build file

0b96fbb

Merge branch 'main' of github.com:aws/aws-cdk into bergjaak/18470

52d3ce4

github-actions bot added the p2 label May 14, 2024

aws-cdk-automation requested a review from a team May 14, 2024 21:23

mergify bot added the contribution/core This is a PR that came from AWS. label May 14, 2024

aws-cdk-automation previously requested changes May 14, 2024

View reviewed changes

github-actions bot added effort/large Large work item – several weeks of effort feature-request A feature should be added or improved. p1 and removed p2 labels May 14, 2024

bergjaak changed the title ~~Bergjaak/18470~~ feat(Lambda): allow running a build file May 14, 2024

bergjaak added effort/medium Medium work item – several days of effort and removed effort/large Large work item – several weeks of effort labels May 14, 2024

bergjaak added 7 commits May 14, 2024 22:41

clean up

4ddf6e1

clean up

94b6685

Merge branch 'main' of github.com:aws/aws-cdk into bergjaak/18470

5412aaf

allow user to specify arguments that are passed to spawnSync

9869ea9

allow bundling options

c3212e8

lil comment

a162e59

nice testing

63a1959

bergjaak force-pushed the bergjaak/18470 branch from 8434309 to 63a1959 Compare May 15, 2024 15:04

nice testsss

12134f9

colifran reviewed May 15, 2024

View reviewed changes

colifran changed the title ~~feat(Lambda): allow running a build file~~ feat(lambda): allow running a build file May 15, 2024

github-actions bot added effort/large Large work item – several weeks of effort and removed effort/medium Medium work item – several days of effort labels May 15, 2024

colifran added p2 and removed p1 labels May 15, 2024

add documentation

e5de5ff

colifran reviewed May 30, 2024

View reviewed changes

bergjaak added 3 commits May 30, 2024 15:19

make rosetta happy please

a6bcad2

Merge branch 'main' of github.com:aws/aws-cdk into bergjaak/18470

26b4b15

changes

7200961

colifran approved these changes May 30, 2024

View reviewed changes

bergjaak added 4 commits May 30, 2024 16:54

changes

54fe929

changes

63d0788

Merge branch 'main' of github.com:aws/aws-cdk into bergjaak/18470

2732c24

changes

86c345b

mergify bot merged commit 89042c4 into main May 30, 2024
12 checks passed

mergify bot deleted the bergjaak/18470 branch May 30, 2024 22:33

github-actions bot mentioned this pull request Jun 1, 2024

Monthly PR metrics report #30414

Closed

rwlxxvii mentioned this pull request Jun 22, 2024

[Snyk] Upgrade aws-cdk-lib from 2.139.0 to 2.144.0 rwlxxvii/containers#140

Merged

NOUIY mentioned this pull request Jun 22, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#105

Open

kazimanzurrashid mentioned this pull request Jun 29, 2024

[Snyk] Upgrade aws-cdk-lib from 2.144.0 to 2.145.0 kazimanzurrashid/aws-scheduler-go#821

Open

This was referenced Jun 29, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#106

Open

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#107

Open

NOUIY mentioned this pull request Jul 12, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#108

Open

lpicchi mentioned this pull request Jul 12, 2024

Specifying plugins via the CLI evanw/esbuild#884

Closed

This was referenced Jul 16, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#109

Open

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#110

Open

NOUIY mentioned this pull request Jul 23, 2024

[Snyk] Upgrade: , aws-cdk NOUIY/aws-solutions-constructs#111

Open

aws locked as resolved and limited conversation to collaborators Jul 25, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(lambda): allow running a build file #30196

feat(lambda): allow running a build file #30196

bergjaak commented May 14, 2024 •

edited

Loading

aws-cdk-automation left a comment

colifran May 15, 2024

bergjaak May 15, 2024

colifran May 15, 2024

bergjaak May 15, 2024

bergjaak May 15, 2024

bergjaak May 15, 2024

colifran May 16, 2024

colifran May 15, 2024

bergjaak May 15, 2024

colifran May 30, 2024

bergjaak May 30, 2024

colifran left a comment

aws-cdk-automation commented May 30, 2024

mergify bot commented May 30, 2024

aws-cdk-automation commented Jul 25, 2024

feat(lambda): allow running a build file #30196

feat(lambda): allow running a build file #30196

Conversation

bergjaak commented May 14, 2024 • edited Loading

Issue # (if applicable)

Reason for this change

Description of changes

Description of how you validated changes

Checklist

aws-cdk-automation left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

colifran left a comment

Choose a reason for hiding this comment

aws-cdk-automation commented May 30, 2024

AWS CodeBuild CI Report

mergify bot commented May 30, 2024

aws-cdk-automation commented Jul 25, 2024

bergjaak commented May 14, 2024 •

edited

Loading