Skip to content

Commit

Permalink
deps: upgrade to cjs-module-lexer@1.0.0
Browse files Browse the repository at this point in the history
PR-URL: #35928
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Bradley Farias <bradley.meck@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
  • Loading branch information
guybedford authored and targos committed Nov 3, 2020
1 parent c09fdba commit 5cb77f2
Show file tree
Hide file tree
Showing 8 changed files with 85 additions and 39 deletions.
7 changes: 7 additions & 0 deletions deps/cjs-module-lexer/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,10 @@
1.0.0
- Unsafe getter tracking (https://github.com/guybedford/cjs-module-lexer/pull/29)

0.6.0
- API-only breaking change: Unify JS and Wasm interfaces (https://github.com/guybedford/cjs-module-lexer/pull/27)
- Add type definitions (https://github.com/guybedford/cjs-module-lexer/pull/28)

0.5.2
- Support named getter functions (https://github.com/guybedford/cjs-module-lexer/pull/26)

Expand Down
72 changes: 49 additions & 23 deletions deps/cjs-module-lexer/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,9 @@ npm install cjs-module-lexer
For use in CommonJS:

```js
const parse = require('cjs-module-lexer');
const { parse } = require('cjs-module-lexer');

// `init` return a promise for parity with the ESM API, but you do not have to call it

const { exports, reexports } = parse(`
// named exports detection
Expand Down Expand Up @@ -84,7 +86,9 @@ EXPORTS_SPREAD: `...` (IDENTIFIER | REQUIRE)
EXPORTS_MEMBER: EXPORTS_DOT_ASSIGN | EXPORTS_LITERAL_COMPUTED_ASSIGN
EXPORTS_DEFINE: `Object` `.` `defineProperty `(` IDENTIFIER_STRING `, {`
EXPORTS_DEFINE: `Object` `.` `defineProperty `(` EXPORTS_IDENFITIER `,` IDENTIFIER_STRING
EXPORTS_DEFINE_VALUE: EXPORTS_DEFINE `, {`
(`enumerable: true,`)?
(
`value:` |
Expand Down Expand Up @@ -119,7 +123,9 @@ EXPORT_STAR_LIB: `Object.keys(` IDENTIFIER$1 `).forEach(function (` IDENTIFIER$2

Spacing between tokens is taken to be any ECMA-262 whitespace, ECMA-262 block comment or ECMA-262 line comment.

* The returned export names are taken to be the combination of the `IDENTIFIER` and `IDENTIFIER_STRING` slots for all `EXPORTS_MEMBER`, `EXPORTS_LITERAL` and `EXPORTS_DEFINE` matches.
* The returned export names are taken to be the combination of:
1. All `IDENTIFIER` and `IDENTIFIER_STRING` slots for `EXPORTS_MEMBER` and `EXPORTS_LITERAL` matches.
2. The first `IDENTIFIER_STRING` slot for all `EXPORTS_DEFINE_VALUE` matches where that same string is not an `EXPORTS_DEFINE` match that is not also an `EXPORTS_DEFINE_VALUE` match.
* The reexport specifiers are taken to be the the combination of:
1. The `REQUIRE` matches of the last matched of either `MODULE_EXPORTS_ASSIGN` or `EXPORTS_LITERAL`.
2. All _top-level_ `EXPORT_STAR` `REQUIRE` matches and `EXPORTS_ASSIGN` matches whose `IDENTIFIER` also matches the first `IDENTIFIER` in `EXPORT_STAR_LIB`.
Expand Down Expand Up @@ -160,6 +166,8 @@ It will in turn underclassify in cases where the identifiers are renamed:
})(exports);
```

#### Getter Exports Parsing

`Object.defineProperty` is detected for specifically value and getter forms returning an identifier or member expression:

```js
Expand All @@ -186,6 +194,24 @@ Object.defineProperty(exports, 'd', { value: 'd' });
Object.defineProperty(exports, '__esModule', { value: true });
```

To avoid matching getters that have side effects, any getter for an export name that does not support the forms above will
opt-out of the getter matching:

```js
// DETECTS: NO EXPORTS
Object.defineProperty(exports, 'a', {
value: 'no problem'
});

if (false) {
Object.defineProperty(module.exports, 'a', {
get () {
return dynamic();
}
})
}
```

Alternative object definition structures or getter function bodies are not detected:

```js
Expand Down Expand Up @@ -335,63 +361,63 @@ JS Build:

```
Module load time
> 5ms
> 4ms
Cold Run, All Samples
test/samples/*.js (3635 KiB)
> 323ms
> 299ms
Warm Runs (average of 25 runs)
test/samples/angular.js (1410 KiB)
> 14.84ms
> 13.96ms
test/samples/angular.min.js (303 KiB)
> 4.8ms
> 4.72ms
test/samples/d3.js (553 KiB)
> 7.84ms
> 6.76ms
test/samples/d3.min.js (250 KiB)
> 4ms
test/samples/magic-string.js (34 KiB)
> 0.72ms
> 0.64ms
test/samples/magic-string.min.js (20 KiB)
> 0.4ms
> 0ms
test/samples/rollup.js (698 KiB)
> 9.32ms
> 8.48ms
test/samples/rollup.min.js (367 KiB)
> 6.52ms
> 5.36ms
Warm Runs, All Samples (average of 25 runs)
test/samples/*.js (3635 KiB)
> 44ms
> 40.28ms
```

Wasm Build:
```
Module load time
> 11ms
> 10ms
Cold Run, All Samples
test/samples/*.js (3635 KiB)
> 42ms
> 43ms
Warm Runs (average of 25 runs)
test/samples/angular.js (1410 KiB)
> 9.92ms
> 9.32ms
test/samples/angular.min.js (303 KiB)
> 3.2ms
> 3.16ms
test/samples/d3.js (553 KiB)
> 5.2ms
> 5ms
test/samples/d3.min.js (250 KiB)
> 2.52ms
> 2.32ms
test/samples/magic-string.js (34 KiB)
> 0.16ms
test/samples/magic-string.min.js (20 KiB)
> 0.04ms
> 0ms
test/samples/rollup.js (698 KiB)
> 6.44ms
> 6.28ms
test/samples/rollup.min.js (367 KiB)
> 3.96ms
> 3.6ms
Warm Runs, All Samples (average of 25 runs)
test/samples/*.js (3635 KiB)
> 30.48ms
> 27.76ms
```

### Wasm Build Steps
Expand Down
2 changes: 1 addition & 1 deletion deps/cjs-module-lexer/dist/lexer.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions deps/cjs-module-lexer/dist/lexer.mjs

Large diffs are not rendered by default.

29 changes: 20 additions & 9 deletions deps/cjs-module-lexer/lexer.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ let openTokenDepth,
starExportMap,
lastStarExportSpecifier,
_exports,
unsafeGetters,
reexports;

function resetState () {
Expand All @@ -27,6 +28,7 @@ function resetState () {
lastStarExportSpecifier = null;

_exports = new Set();
unsafeGetters = new Set();
reexports = new Set();
}

Expand All @@ -37,7 +39,7 @@ const ExportStar = 2;

const strictReserved = new Set(['implements', 'interface', 'let', 'package', 'private', 'protected', 'public', 'static', 'yield', 'enum']);

module.exports = function parseCJS (source, name = '@') {
function parseCJS (source, name = '@') {
resetState();
try {
parseSource(source);
Expand All @@ -47,7 +49,7 @@ module.exports = function parseCJS (source, name = '@') {
e.loc = pos;
throw e;
}
const result = { exports: [..._exports], reexports: [...reexports] };
const result = { exports: [..._exports].filter(expt => !unsafeGetters.has(expt)), reexports: [...reexports] };
resetState();
return result;
}
Expand Down Expand Up @@ -260,6 +262,7 @@ function tryParseObjectDefineOrKeys (keys) {
pos++;
ch = commentWhitespace();
if (ch === 100/*d*/ && source.startsWith('efineProperty', pos + 1)) {
let expt;
while (true) {
pos += 14;
revertPos = pos - 1;
Expand All @@ -276,7 +279,7 @@ function tryParseObjectDefineOrKeys (keys) {
let quot = ch;
const exportPos = ++pos;
if (!identifier() || source.charCodeAt(pos) !== quot) break;
const expt = source.slice(exportPos, pos);
expt = source.slice(exportPos, pos);
pos++;
ch = commentWhitespace();
if (ch !== 44/*,*/) break;
Expand Down Expand Up @@ -304,9 +307,9 @@ function tryParseObjectDefineOrKeys (keys) {
pos += 5;
ch = commentWhitespace();
if (ch !== 58/*:*/) break;
pos++;
addExport(expt);
break;
pos = revertPos;
return;
}
else if (ch === 103/*g*/) {
if (!source.startsWith('et', pos + 1)) break;
Expand Down Expand Up @@ -372,6 +375,9 @@ function tryParseObjectDefineOrKeys (keys) {
}
break;
}
if (expt) {
unsafeGetters.add(expt);
}
}
else if (keys && ch === 107/*k*/ && source.startsWith('eys', pos + 1)) {
while (true) {
Expand Down Expand Up @@ -899,7 +905,7 @@ function tryParseLiteralExports () {

// --- Extracted from AcornJS ---
//(https://github.com/acornjs/acorn/blob/master/acorn/src/identifier.js#L23
//
//
// MIT License

// Copyright (C) 2012-2018 by various contributors (see AUTHORS)
Expand Down Expand Up @@ -1034,7 +1040,7 @@ function throwIfImportStatement () {
case 46/*.*/:
throw new Error('Unexpected import.meta in CJS module.');
return;

default:
// no space after "import" -> not an import keyword
if (pos === startPos + 6)
Expand Down Expand Up @@ -1203,7 +1209,7 @@ function readPrecedingKeyword (pos, match) {
}

function readPrecedingKeyword1 (pos, ch) {
return source.charCodeAt(pos) === ch && (pos === 0 || isBrOrWsOrPunctuatorNotDot(source.charCodeAt(pos - 1)));
return source.charCodeAt(pos) === ch && (pos === 0 || isBrOrWsOrPunctuatorNotDot(source.charCodeAt(pos - 1)));
}

// Detects one of case, debugger, delete, do, else, in, instanceof, new,
Expand Down Expand Up @@ -1274,7 +1280,7 @@ function isExpressionKeyword (pos) {
// throw
return readPrecedingKeyword(pos - 2, 'thr');
default:
return false;
return false;
}
}
return false;
Expand Down Expand Up @@ -1320,3 +1326,8 @@ function isExpressionTerminator (curPos) {
}
return false;
}

const initPromise = Promise.resolve();

module.exports.init = () => initPromise;
module.exports.parse = parseCJS;
6 changes: 4 additions & 2 deletions deps/cjs-module-lexer/package.json
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
{
"name": "cjs-module-lexer",
"version": "0.5.2",
"version": "1.0.0",
"description": "Lexes CommonJS modules, returning their named exports metadata",
"main": "lexer.js",
"exports": {
"import": "./dist/lexer.mjs",
"default": "./lexer.js"
},
"types": "lexer.d.ts",
"scripts": {
"test-js": "mocha -b -u tdd test/*.js",
"test-wasm": "WASM=1 mocha -b -u tdd test/*.js",
Expand All @@ -28,7 +29,8 @@
"terser": "^4.1.4"
},
"files": [
"dist"
"dist",
"lexer.d.ts"
],
"repository": {
"type": "git",
Expand Down
2 changes: 1 addition & 1 deletion doc/api/esm.md
Original file line number Diff line number Diff line change
Expand Up @@ -1287,7 +1287,7 @@ success!
[`transformSource` hook]: #esm_transformsource_source_context_defaulttransformsource
[`string`]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String
[`util.TextDecoder`]: util.md#util_class_util_textdecoder
[cjs-module-lexer]: https://github.com/guybedford/cjs-module-lexer/tree/0.5.2
[cjs-module-lexer]: https://github.com/guybedford/cjs-module-lexer/tree/1.0.0
[special scheme]: https://url.spec.whatwg.org/#special-scheme
[the official standard format]: https://tc39.github.io/ecma262/#sec-modules
[transpiler loader example]: #esm_transpiler_loader
2 changes: 1 addition & 1 deletion lib/internal/modules/esm/translators.js
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ const { emitWarningSync } = require('internal/process/warning');
let cjsParse;
async function initCJSParse() {
if (typeof WebAssembly === 'undefined') {
cjsParse = require('internal/deps/cjs-module-lexer/lexer');
cjsParse = require('internal/deps/cjs-module-lexer/lexer').parse;
} else {
const { parse, init } =
require('internal/deps/cjs-module-lexer/dist/lexer');
Expand Down

0 comments on commit 5cb77f2

Please sign in to comment.