lib: fix code cache generation, add more tests and enable them by default #23855

joyeecheung · 2018-10-24T16:08:34Z

lib: fix code cache generation

e7f710c broke the code cache generation since internalBinding
is now passed in through the wrapper and cannot be redeclared.
This patch fixes that.

test: run code cache test by default and test generator

Add the code cache tests to the default test suite, and test
the bookkeeping when the binary is not built with the code cache.
Test the code cache generator to make sure we do not accidentally
break it - until we enable code cache in the CI.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

nodejs-github-bot · 2018-10-24T16:08:36Z

@joyeecheung build started: https://ci.nodejs.org/blue/organizations/jenkins/node-test-pull-request-lite-pipeline/detail/node-test-pull-request-lite-pipeline/1359/pipeline

joyeecheung · 2018-10-24T16:16:22Z

CI: https://ci.nodejs.org/job/node-test-pull-request/18120/

targos · 2018-10-24T16:16:16Z

test/code-cache/test-code-cache.js

-  assert(isUint8Array(codeCache[key]) && codeCache[key].length > 0,
-         `Code cache for "${key}" should've been generated`);
+} else {
+  // The binary is not configured with code cache.


Remove "not" ?

refack

LGTM with nit addressed

refack · 2018-10-24T17:25:17Z

test/code-cache/test-code-cache-generator.js

+assert.ifError(child.error);
+
+const stat = fs.statSync(dest);
+assert(stat.isFile());


Prefer assert.ok
Is there anything else we can test about the file? Maybe that it has the string static uint8_t internal_bootstrap_cache_raw[]?

test/code-cache/test-code-cache-generator.js

joyeecheung · 2018-10-25T10:27:54Z

Addressed nits and added tests for verifying node::DefineCodeCache() and node::DefineCodeCacheHash() are defined in the generated code.

CI: https://ci.nodejs.org/job/node-test-pull-request/18135/

refack · 2018-10-25T13:19:36Z

Addressed nits and added tests for verifying node::DefineCodeCache() and node::DefineCodeCacheHash() are defined in the generated code.

Thank you!

joyeecheung · 2018-10-26T10:35:55Z

Hmm, CI was not happy, I'll investigate

https://ci.nodejs.org/job/node-test-commit-linux-containered/8109/nodes=ubuntu1604_sharedlibs_withoutintl_x64/console

18:36:01   ...
18:36:02 not ok 2161 code-cache/test-code-cache-generator
18:36:02   ---
18:36:02   duration_ms: 0.551
18:36:02   severity: fail
18:36:02   exitcode: 1
18:36:02   stack: |-
18:36:02     events.js:167
18:36:02           throw er; // Unhandled 'error' event
18:36:02           ^
18:36:02     
18:36:02     Error: ENOENT: no such file or directory, open '/home/iojs/node-tmp/.tmp.1/cache.cc'
18:36:02     Emitted 'error' event at:
18:36:02         at fs.open (internal/fs/streams.js:130:12)
18:36:02         at FSReqCallback.oncomplete (fs.js:148:20)

e7f710c broke the code cache generation since internalBinding is now passed in through the wrapper and cannot be redeclared. This patch fixes that.

- Add the code cache tests to the default test suite, and test the bookkeeping when the binary is not built with the code cache. - Test the code cache generator to make sure we do not accidentally break it - until we enable code cache in the CI.

joyeecheung · 2018-10-28T20:24:45Z

Rebased and added logs when the tools/generate_code_cache.js command fails (somehow when the child process fails (child.status !== 0), child.error is still null..is that a child process bug?)

CI: https://ci.nodejs.org/job/node-test-pull-request/18189/

EDIT: --without-intl implying no inspector is intentional: #12758

joyeecheung · 2018-10-29T07:16:35Z

Apparently inspector is not available in the withoutintl job (cc @refack is that intentional?)..skipped the inspector modules when inspector is not available.

CI: https://ci.nodejs.org/job/node-test-pull-request/18196/

joyeecheung · 2018-10-29T11:23:39Z

trace events are also unavailable in withoutintl jobs..fixed the cannotUseCache list again

CI: https://ci.nodejs.org/job/node-test-pull-request/18203/

joyeecheung · 2018-10-29T16:12:50Z

So the resume build is green: https://ci.nodejs.org/job/node-test-pull-request/18204/
Although previously there were two occurrence of SSL routines:ssl3_get_record:decryption failed or bad record mac:../deps/openssl/openssl/ssl/record/ssl3_record.c:469: see #23913

I am leaning towards that's a irrelevant flake.

joyeecheung · 2018-10-29T16:18:41Z

@refack @cjihrig @targos Can you take another look? (not too much has changed since the initial iteration, though, mostly just skipping creating cache for modules that are not available in withoutintl jobs)

jasnell · 2018-10-30T00:07:17Z

lib/internal/bootstrap/cache.js

@@ -42,6 +43,15 @@ const cannotUseCache = [
  'internal/bootstrap/node'
 ].concat(depsModule);

+if (process.config.variables.v8_enable_inspector !== 1) {


May be worthwhile including a quick comment with these that says why they cannot use the cache, just so it's clear to folks coming along later.

joyeecheung · 2018-10-30T06:43:51Z

Landed in b255cd4...010a3f8 with the comment requested by @jasnell added

e7f710c broke the code cache generation since internalBinding is now passed in through the wrapper and cannot be redeclared. This patch fixes that. Refs: #21563 PR-URL: #23855 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

- Add the code cache tests to the default test suite, and test the bookkeeping when the binary is not built with the code cache. - Test the code cache generator to make sure we do not accidentally break it - until we enable code cache in the CI. Refs: #21563 PR-URL: #23855 Reviewed-By: Michaël Zasso <targos@protonmail.com> Reviewed-By: Refael Ackermann <refack@gmail.com> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>