(aws-lambda-nodejs): `.mjs` file extension and `import.meta` not supported #13274

blake-regalia · 2021-02-25T00:56:38Z

Although Lambda now supports Node v14:

.mjs entry files are rejected by a regex pattern in aws-lambda-nodejs
esbuild is transpiling the scripts to cjs format (commonjs) but BundlingOptions provides no means to specify format: "esm", and consequently esbuild polyfills import.meta which breaks all of its uses in the scripts

Reproduction Steps

lib/my-stack.mjs:

export class MyStack extends Stack {
    constructor(scope, id, props) {
        super(scope, id, props);

        // ...

        new NodejsFunction(this, 'Example', {
            runtime: Runtime.NODEJS_14_X,
            entry: 'src/entry-file.mjs',
            bundling: {
                target: 'es2020',
                // format: 'esm',   <-- should be able to pass this option here
            },
        };

        // ...
    }
}

src/entry-file.mjs:

export async function Example() {
    return import.meta.url;
}

What did you expect to happen?

entry-file.mjs to be allowed to be used as an entry file.
import.meta to be defined and Example() to return a string.

What actually happened?

Error: Only JavaScript or TypeScript entry files are supported.
Example() returns undefined since import.meta is polyfilled with an empty plain object.

Environment

CDK CLI Version : n/a
Framework Version:
Node.js Version: 14.13.0
OS : n/a
Language (Version):

Other

For the .mjs extension:

aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/lib/function.ts

Line 140 in 5d71d8e

if (!/\.(jsx?|tsx?)$/.test(entry)) {

Allow passing in the format option to esbuild:

aws-cdk/packages/@aws-cdk/aws-lambda-nodejs/lib/bundling.ts

Lines 141 to 158 in 5d71d8e

    
           const esbuildCommand: string = [ 
        
             npx, 'esbuild', 
        
             '--bundle', pathJoin(inputDir, this.relativeEntryPath), 
        
             `--target=${this.props.target ?? toTarget(this.props.runtime)}`, 
        
             '--platform=node', 
        
             `--outfile=${pathJoin(outputDir, 'index.js')}`, 
        
             ...this.props.minify ? ['--minify'] : [], 
        
             ...this.props.sourceMap ? ['--sourcemap'] : [], 
        
             ...this.externals.map(external => `--external:${external}`), 
        
             ...loaders.map(([ext, name]) => `--loader:${ext}=${name}`), 
        
             ...defines.map(([key, value]) => `--define:${key}=${value}`), 
        
             ...this.props.logLevel ? [`--log-level=${this.props.logLevel}`] : [], 
        
             ...this.props.keepNames ? ['--keep-names'] : [], 
        
             ...this.relativeTsconfigPath ? [`--tsconfig=${pathJoin(inputDir, this.relativeTsconfigPath)}`] : [], 
        
             ...this.props.metafile ? [`--metafile=${pathJoin(outputDir, 'index.meta.json')}`] : [], 
        
             ...this.props.banner ? [`--banner='${this.props.banner}'`] : [], 
        
             ...this.props.footer ? [`--footer='${this.props.footer}'`] : [], 
        
           ].join(' ');

This is 🐛 Bug Report

The text was updated successfully, but these errors were encountered:

jogold · 2021-02-25T08:38:33Z

Hi @blake-regalia, does Lambda support mjs files? How? AFAIK it only supports CJS.

blake-regalia · 2021-02-25T17:50:04Z

Yes in node.js >= v14.13 environment you can use esmodules.

jogold · 2021-02-25T19:43:26Z

Yes in node.js >= v14.13 environment you can use esmodules.

I understand that you want to use .mjs files as input and it's easy to update the regex there.

But regarding the output of esbuild I don't think Lambda knows how to deal with .mjs files so we'll always have to transpile to CJS, no?

blake-regalia · 2021-02-25T20:42:35Z

I don't know where you are getting this assumption about Lambda not knowing how to deal with .mjs files; I must admit I don't know exactly how it is done currently but I don't see why this would cause any issues. As long as the environment is the correct version of node.js, it should be able to run mjs files in module mode.

And more important than the file extension is getting away from transpilation entirely since this breaks the use of import.meta.

jogold · 2021-02-26T08:26:15Z

I don't know where you are getting this assumption about Lambda not knowing how to deal with .mjs files; I must admit I don't know exactly how it is done currently but I don't see why this would cause any issues. As long as the environment is the correct version of node.js, it should be able to run mjs files in module mode.

Tried in the Lambda console and I was not able to do this. If you manage to successfully run files in module mode in the Lambda environment, let me know how you did it.

ronaldocpontes · 2021-03-09T11:08:36Z

I am also getting errors on a new lambda with node 14.x.

Could someone publish a working example of using .mjs?

ronaldocpontes · 2021-03-09T13:21:22Z

I believe the problem is that the lambda engine is using CommonJS require to load the handler method. This won't work if the lambda handler is an ES6 module or a .mjs file as it will fall in the error quadrant on the table below.

So the question is: how do we use nodejs 14.x out-of-the-box ES6 modules on AWS Lambdas?

https://pencilflip.medium.com/using-es-modules-with-commonjs-modules-in-node-js-1015786dab03

ronaldocpontes · 2021-03-09T13:52:08Z

This is what I get when using "type": "module" on package.json. As you can see on the error:
require() of ES modules is not supported.
require() of /var/task/index.js...

The lambda engine is using require() to load the code on index.js.

# package.json
{
"name": "myLambda",
"type": "module",
"main": "index.js",
}

# index.js
export const handler = async (event) => {  
    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

Response
{
  "errorType": "Error",
  "errorMessage": "Must use import to load ES Module: /var/task/index.js\nrequire() of ES modules is not supported.\nrequire() of /var/task/index.js from /var/runtime/UserFunction.js is an ES module file as it is a .js file whose nearest parent package.json contains \"type\": \"module\" which defines all .js files in that package scope as ES modules.\nInstead rename index.js to end in .cjs, change the requiring code to use import(), or remove \"type\": \"module\" from /var/task/package.json.\n",
  "trace": [
    "Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /var/task/index.js",
    "require() of ES modules is not supported.",
    "require() of /var/task/index.js from /var/runtime/UserFunction.js is an ES module file as it is a .js file whose nearest parent package.json contains \"type\": \"module\" which defines all .js files in that package scope as ES modules.",
    "Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove \"type\": \"module\" from /var/task/package.json.",
    "",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1080:13)",
    "    at Module.load (internal/modules/cjs/loader.js:928:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:769:14)",
    "    at Module.require (internal/modules/cjs/loader.js:952:19)",
    "    at require (internal/modules/cjs/helpers.js:88:18)",
    "    at _tryRequire (/var/runtime/UserFunction.js:75:12)",
    "    at _loadUserApp (/var/runtime/UserFunction.js:95:12)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:140:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:1063:30)"
  ]
}

revmischa · 2021-04-24T17:04:42Z

I would like to load a "type": "module" package in CDK when building my aws-lambda-nodejs Lambda, even if it's being eventually output as commonJS. There should be some way to set it up so esbuild is doing an import instead of require() somewhere

Error [ERR_REQUIRE_ESM]: Must use import to load ES Module: /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/index.js
require() of ES modules is not supported.
require() of /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/index.js from /Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/lib/cdk-stack.ts is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/package.json.

    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1080:13)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/lib/cdk-stack.ts:9:1)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Module.m._compile (/Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/node_modules/ts-node/src/index.ts:1056:23)
    at Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Object.require.extensions.<computed> [as .ts] (/Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/node_modules/ts-node/src/index.ts:1059:12)

My motivation is to use esbuild's tree-shaking. Right now my function is very large and appears to contain all of @aws-cdk and typeorm and others, so I am very sure tree shaking is not working correctly.
The esbuild docs say:

Note that esbuild's tree shaking implementation relies on the use of ECMAScript module import and export statements. It does not work with CommonJS modules.

But I can see that aws-lambda-nodejs is not possibly doing this because of the above error and the require stack here is using the cjs not ejs version of the @jetkit/cdk module:

Require stack:
- /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/cjs/cdk/lambda/node14func.js
- /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/cjs/cdk/api/api.js
- /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/cjs/cdk/api/crud.js
- /Users/cyber/dev/jb/jkv2-demo/cdk/node_modules/@jetkit/cdk/build/cjs/index.js
- /Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/lib/cdk-stack.ts
- /Users/cyber/dev/jb/jkv2-demo/cdk/packages/infra/bin/cdk.ts
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)

I believe if esbuild was configured to support ES-modules then we could get much much smaller function bundles. As it is now it appears to bundle entire packages:

jogold · 2021-04-26T13:10:44Z

Right now my function is very large and appears to contain all of @aws-cdk and typeorm and others,

This is really strange... can you share a repro? Are those dependencies imported in your handler? If yes, why? If not, they shouldn't be bundled.

revmischa · 2021-04-26T13:14:06Z

Right now my function is very large and appears to contain all of @aws-cdk and typeorm and others,

This is really strange... can you share a repro? Are those dependencies imported in your handler? If yes, why? If not, they shouldn't be bundled.

My handler includes @jetkit/cdk which is what I'm trying to build right now. The construct I'm using is https://github.com/jetbridge/jetkit-cdk/blob/implement-view-routing/src/cdk/generator.ts and the app code is https://github.com/jetbridge/jetkit-cdk/blob/implement-view-routing/src/test/sampleApp.ts

jogold · 2021-04-26T18:51:20Z

My handler includes @jetkit/cdk which is what I'm trying to build right now

Can you show your handler? Is it a .ts file?

revmischa · 2021-04-26T19:01:52Z

Yes, https://github.com/jetbridge/jetkit-cdk/blob/implement-view-routing/src/test/sampleApp.ts is the handler file. It's the handler export

export const handler = apiViewHandler(__filename, AlbumApi)

apiViewHandler is defined here: https://github.com/jetbridge/jetkit-cdk/blob/5d3adb8dbf732fdc1d968b5d5486ad2ec433a0fd/src/api/base.ts#L219

I'm trying to make some decorators that a CDK construct can make use of to generate NodeJsLambda functions and APIGW routes automatically

Sparticuz · 2021-05-11T14:23:41Z

According to this SO question, the Node on Lambda doesn't support ecmascript modules yet. I would love to see this happen ASAP. All of Sindre Sorhus' packages are currently moving to ESM only! https://blog.sindresorhus.com/hello-modules-d1010b4e777b

Mystogab · 2021-08-25T17:03:10Z

Any updates on this? I'm on a mayor refactor on a curtial step of a pure JS project ( I wanted to switch to TS but was too agressive to suggest all changes at once). So if I use nodejs 14.x runtime, then export const handler wont work?

groner · 2021-09-22T18:53:17Z

Here's a possible approach to running ES modules in lambda today (without the benefit of NodejsFunction)

The following stackoverflow answer claims success running ES modules with a shim using dynamic import().

From https://stackoverflow.com/a/69136438/91927

This worked for me on Lambda Node 14.x -

in app.js

exports.lambdaHandler = async (event, context) => {
  const { App } = await import('./lib/app.mjs');
  return new App(event, context);
}

And then in lib/app.mjs -

class App {

  constructor(event, context) {
    return {
      'statusCode': 200,
      'body': JSON.stringify({
        'message': 'hello world'
      })
    }
  }
} 

export {App}

groner · 2021-09-22T19:08:50Z

The following code in aws/aws-lambda-nodejs-runtime-interface-client looks like it could be adapted to use import() and import.meta.resolve() without much trouble.

From https://github.com/aws/aws-lambda-nodejs-runtime-interface-client/blob/f9373b3e7cb153fe2895db446e21f3241eb156c6/src/utils/UserFunction.ts#L70-L111

/**
 * Attempt to load the user's module.
 * Attempts to directly resolve the module relative to the application root,
 * then falls back to the more general require().
 */
function _tryRequire(appRoot: string, moduleRoot: string, module: string): any {
  const lambdaStylePath = path.resolve(appRoot, moduleRoot, module);
  if (_canLoadAsFile(lambdaStylePath)) {
    return require(lambdaStylePath);
  } else {
    // Why not just require(module)?
    // Because require() is relative to __dirname, not process.cwd()
    const nodeStylePath = require.resolve(module, {
      paths: [appRoot, moduleRoot],
    });
    return require(nodeStylePath);
  }
}


/**
 * Load the user's application or throw a descriptive error.
 * @throws Runtime errors in two cases
 *   1 - UserCodeSyntaxError if there's a syntax error while loading the module
 *   2 - ImportModuleError if the module cannot be found
 */
function _loadUserApp(
  appRoot: string,
  moduleRoot: string,
  module: string
): any {
  try {
    return _tryRequire(appRoot, moduleRoot, module);
  } catch (e) {
    if (e instanceof SyntaxError) {
      throw new UserCodeSyntaxError(<any>e);
    } else if (e.code !== undefined && e.code === "MODULE_NOT_FOUND") {
      throw new ImportModuleError(e);
    } else {
      throw e;
    }
  }
}

intptr-t · 2021-11-09T12:41:30Z

Since there seems to be no progress, I will give my personal opinion.
There are two issues, which I think should be resolved independently of each other.

1. .mjs is not supported.

It should be legitimate for esbuild to output *.mjs and *.cjs as format=cjs.
I think it is reasonable that this be resolved by adding a feature (label=feature-request) to aws-lambda-nodejs in the CDK.

add a regular expression to \. (c|m)?jsx?|(c|m)?tsx?$
Allow passing in the format option to esbuild:, as shown by blake-regalia.

2. import.meta is not supported

The CDK is not concerned about the AWS Lambda runtime fully supporting ES2020 features (e.g. import.meta).
So it's better to wait until AWS Lambda supports it, or build it yourself using a custom runtime.

jogold · 2022-01-07T09:31:36Z

https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/

https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/

adamdottv · 2022-01-08T16:56:55Z

https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/

https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/

Came here to see if this was in the works; @jogold are you planning to add support for native esm?

jogold · 2022-01-09T10:20:30Z

ame here to see if this was in the works; @jogold are you planning to add support for native esm?

I will try to have a look at it this week or next week.

Add a `format` option to choose the output format (CommonJS or ECMAScript module). Generate a `index.mjs` file when the ECMAScript module output format is chosen so that AWS Lambda treats it correctly. Closes aws#13274

jogold · 2022-01-10T16:27:39Z

See #18346

Add a `format` option to choose the output format (CommonJS or ECMAScript module). Generate a `index.mjs` file when the ECMAScript module output format is chosen so that AWS Lambda treats it correctly. See https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/ See https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/ Closes #13274 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

github-actions · 2022-01-11T21:58:13Z

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

pfried · 2022-01-27T06:41:41Z

Not an aws-cdk Issue, but for everyone who ends up here: Using modules form layers is currently not supported in lambda when using ESM

Add a `format` option to choose the output format (CommonJS or ECMAScript module). Generate a `index.mjs` file when the ECMAScript module output format is chosen so that AWS Lambda treats it correctly. See https://aws.amazon.com/about-aws/whats-new/2022/01/aws-lambda-es-modules-top-level-await-node-js-14/ See https://aws.amazon.com/blogs/compute/using-node-js-es-modules-and-top-level-await-in-aws-lambda/ Closes aws#13274 ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

paul-uz · 2022-03-31T09:40:25Z

@pfried damn, thats exactly the issue I hit. aws/aws-sdk-js-v3#3386

pfried · 2022-12-14T10:42:51Z

@paul-uz I opened an issue here: #23333

paul-uz · 2022-12-14T10:51:21Z

@paul-uz I opened an issue here: #23333

I believe this now works in node 18 on lambda

blake-regalia added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Feb 25, 2021

github-actions bot added the @aws-cdk/aws-lambda-nodejs label Feb 25, 2021

github-actions bot assigned eladb Feb 25, 2021

eladb added effort/small Small work item – less than a day of effort p2 labels Feb 25, 2021

eladb removed their assignment Feb 25, 2021

oscard0m mentioned this issue May 26, 2021

Move away from Vercel squash-commit-app/squash-commit-app#8

Closed

revmischa mentioned this issue Jul 2, 2021

Using captureAWSClient over directly referenced dynamodb client fails aws/aws-xray-sdk-node#23

Open

NGL321 removed the needs-triage This issue or PR still needs to be triaged. label Jul 23, 2021

revmischa mentioned this issue Oct 1, 2021

Testing out not generating/bundling prisma client, only engines jetbridge/jetkit-cdk#33

Closed

Sparticuz mentioned this issue Jan 7, 2022

Node v16 Released 🎉 aws/aws-lambda-base-images#14

Closed

jogold mentioned this issue Jan 10, 2022

feat(lambda-nodejs): ES modules #18346

Merged

mergify bot closed this as completed in #18346 Jan 11, 2022

twelch mentioned this issue Apr 15, 2024

Migrate to ES Module structure/runtime - Node 20, Vite, Vitest, Storybook 8 seasketch/geoprocessing#272

Merged

39 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(aws-lambda-nodejs): `.mjs` file extension and `import.meta` not supported #13274

(aws-lambda-nodejs): `.mjs` file extension and `import.meta` not supported #13274

blake-regalia commented Feb 25, 2021

jogold commented Feb 25, 2021

blake-regalia commented Feb 25, 2021

jogold commented Feb 25, 2021

blake-regalia commented Feb 25, 2021 •

edited

Loading

jogold commented Feb 26, 2021

ronaldocpontes commented Mar 9, 2021

ronaldocpontes commented Mar 9, 2021

ronaldocpontes commented Mar 9, 2021

revmischa commented Apr 24, 2021 •

edited

Loading

jogold commented Apr 26, 2021

revmischa commented Apr 26, 2021

jogold commented Apr 26, 2021

revmischa commented Apr 26, 2021

Sparticuz commented May 11, 2021

Mystogab commented Aug 25, 2021

groner commented Sep 22, 2021 •

edited

Loading

groner commented Sep 22, 2021

intptr-t commented Nov 9, 2021

jogold commented Jan 7, 2022

adamdottv commented Jan 8, 2022

jogold commented Jan 9, 2022

jogold commented Jan 10, 2022

github-actions bot commented Jan 11, 2022

pfried commented Jan 27, 2022 •

edited

Loading

paul-uz commented Mar 31, 2022

pfried commented Dec 14, 2022

paul-uz commented Dec 14, 2022

(aws-lambda-nodejs): .mjs file extension and import.meta not supported #13274

(aws-lambda-nodejs): .mjs file extension and import.meta not supported #13274

Comments

blake-regalia commented Feb 25, 2021

Reproduction Steps

What did you expect to happen?

What actually happened?

Environment

Other

jogold commented Feb 25, 2021

blake-regalia commented Feb 25, 2021

jogold commented Feb 25, 2021

blake-regalia commented Feb 25, 2021 • edited Loading

jogold commented Feb 26, 2021

ronaldocpontes commented Mar 9, 2021

ronaldocpontes commented Mar 9, 2021

ronaldocpontes commented Mar 9, 2021

revmischa commented Apr 24, 2021 • edited Loading

jogold commented Apr 26, 2021

revmischa commented Apr 26, 2021

jogold commented Apr 26, 2021

revmischa commented Apr 26, 2021

Sparticuz commented May 11, 2021

Mystogab commented Aug 25, 2021

groner commented Sep 22, 2021 • edited Loading

groner commented Sep 22, 2021

intptr-t commented Nov 9, 2021

1. .mjs is not supported.

2. import.meta is not supported

jogold commented Jan 7, 2022

adamdottv commented Jan 8, 2022

jogold commented Jan 9, 2022

jogold commented Jan 10, 2022

github-actions bot commented Jan 11, 2022

⚠️COMMENT VISIBILITY WARNING⚠️

pfried commented Jan 27, 2022 • edited Loading

paul-uz commented Mar 31, 2022

pfried commented Dec 14, 2022

paul-uz commented Dec 14, 2022

(aws-lambda-nodejs): `.mjs` file extension and `import.meta` not supported #13274

(aws-lambda-nodejs): `.mjs` file extension and `import.meta` not supported #13274

blake-regalia commented Feb 25, 2021 •

edited

Loading

revmischa commented Apr 24, 2021 •

edited

Loading

groner commented Sep 22, 2021 •

edited

Loading

pfried commented Jan 27, 2022 •

edited

Loading