Flaky ELIFECYCLE/3221225477 exit error with win10/nodejs@14 #2660

dr-js · 2020-04-30T04:17:17Z

Version: v14.0.0 and v14.1.0
Platform: Windows 10
Subsystem:

What steps will reproduce the bug?

I got flaky ELIFECYCLE/3221225477 exit error from both:

GitHub Actions (win10/nodejs@14.1.0/npm@6.14.4)
- win32 - 10.0.17763 [x64] from (Log CI info)
local machine (win10/nodejs@14.0.0/npm@6.14.3)
- 10.0.18363 from node -p "os.release()"

The exited process is babel, under bootstrap JS script and multiple npm run. the layer is like:

cmd.exe
- npm test
  - npm run script-pack-test
    - node -r @babel/register ./script verbose pack test
      - npm run build-library
        
        babel ./source --out-dir ./output-gitignore/library

The main error output from GitHub Actions:

> @dr-js/dev@0.3.0-dev.9 build-library D:\a\dr-dev\dr-dev
> babel ./source --out-dir ./output-gitignore/library

npm ERR! code ELIFECYCLE
npm ERR! errno 3221225477
npm ERR! @dr-js/dev@0.3.0-dev.9 build-library: `babel ./source --out-dir ./output-gitignore/library`
npm ERR! Exit status 3221225477
npm ERR! 
npm ERR! Failed at the @dr-js/dev@0.3.0-dev.9 build-library script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\npm\cache\_logs\2020-04-30T03_38_16_411Z-debug.log
Error: Command failed: npm run build-library
    at checkExecSyncError (child_process.js:611:11)

    at execSync (child_process.js:647:15)
## error after 3.71s: Error: Command failed: npm run build-library  [verbose+pack+test|3.71s]
    at execShell (D:\a\dr-dev\dr-dev\script\/index.js:19:32)
    at buildOutput (D:\a\dr-dev\dr-dev\script\/index.js:25:3)
    at D:\a\dr-dev\dr-dev\script\/index.js:85:9 {
  status: 3221225477,
  signal: null,
  output: [ null, null, null ],
  pid: 6504,
  stdout: null,
  stderr: null
}
npm ERR! code ELIFECYCLE
npm ERR! errno 4294967295
npm ERR! @dr-js/dev@0.3.0-dev.9 script-pack-test: `node -r @babel/register ./script verbose pack test`
npm ERR! Exit status 4294967295
npm ERR! 
npm ERR! Failed at the @dr-js/dev@0.3.0-dev.9 script-pack-test script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\npm\cache\_logs\2020-04-30T03_38_16_552Z-debug.log
npm ERR! Test failed.  See above for more details.
##[error]Process completed with exit code 1.

How often does it reproduce? Is there a required condition?

Flaky, same build may pass on second run

What is the expected behavior?

Not exit

What do you see instead?

Related CI runs:

And some npm error log from my local build:

with Exit status 3221225477: 2020-04-29T11_46_56_044Z-debug.log
with Exit status 4294967295 (may be related?): 2020-04-29T11_46_56_095Z-debug.log

Additional information

Possible related issue about status 3221225477: nodejs/node#25512

The text was updated successfully, but these errors were encountered:

Hakerh400 · 2020-04-30T08:44:45Z

Error 3221225477 stands for Access violation on Windows. Does your script (or any of your dependencies) uses any native modules (.node files) that are compiled from C++ source? If you are using only JavaScript files, it may be a bug in V8 engine, or maybe a stack overflow if you have too deep recursion and V8 miscalculated stack size.

I cloned your repo and experienced the same error on first run, but I cannot reproduce it again (tried more than 200 times).

dr-js · 2020-04-30T08:55:41Z

True it's harder to hit locally, on the GitHub CI it's like 1 exit in 4 runs.

I doubt it may be relate to fresh install, babel do keep cache under ./node_modules/.cache,
but deleting it locally or using npm ci don't seem to trigger the exit.

dr-js · 2020-04-30T09:11:03Z

And one wired thing is I only got the exit in dr-js/dr-dev.
My other repo use similar setup do not hit the exit, locally or on GitHub CI, like: dr-js/dr-js and dr-js/dr-node

The difference I can think of may be:

the 1 level deeper nested npm run
the source code is reused to build itself:
- imported from the ./script/index.js with
  "script-pack-test": "node -r @babel/register ./script verbose pack test"
- later build by babel with
  "build-library": "babel ./source --out-dir ./output-gitignore/library"

dr-js · 2020-04-30T09:23:29Z

And I only see exit on win10 + nodejs@14 so far:

all pass: https://github.com/dr-js/dr-dev/actions/runs/91856389
nodejs@14 fail: https://github.com/dr-js/dr-dev/actions/runs/91867942

I'll add more detail if I manage to get the exit again,
and I think to reproduce the CI exit you can fork dr-js/dr-dev and push some change to trigger the CI

I also created a branch for CI to test win10 + nodejs@12|13|14: test-3221225477-exit and got the same nodejs@14 fail on first run: https://github.com/dr-js/dr-dev/actions/runs/91997364

dr-js · 2020-05-01T02:35:06Z

Some update:

Tried reduce the script layer to 1 level of npm, and got 1 exit in 4 CI runs:

1 fail 1 pass: https://github.com/dr-js/dr-dev/actions/runs/92715420
2 pass: https://github.com/dr-js/dr-dev/actions/runs/92721561

Then Run directly with node, and got a powershell error:

https://github.com/dr-js/dr-dev/actions/runs/92731124

> node ./node_modules/@dr-js/dev/bin/index.js --EE BABEL_ENV=module -E -- node ./node_modules/@babel/cli/bin/babel.js ./source --out-dir ./output-gitignore/module
shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
##[error]Process completed with exit code 1.

So change to cmd and got 1 exit in 4 runs:

2 pass: https://github.com/dr-js/dr-dev/actions/runs/92733541

1 fail 1 pass: https://github.com/dr-js/dr-dev/actions/runs/92734367

> node ./node_modules/@babel/cli/bin/babel.js ./source --out-dir ./output-gitignore/library
shell: C:\windows\system32\cmd.EXE /D /E:ON /V:OFF /S /C "CALL "{0}""
##[error]Process completed with exit code -1073741819.

2 error happens on the command with 1 more layers of spawn, but the one with cmd is not

dr-js · 2020-05-01T04:14:01Z

Update:

Currently reduced test case to:
https://github.com/dr-js/dr-dev/actions/runs/92784081

But compared to previous test case with more files, the fail rate seems lower (1 fail vs 2 early fail):
https://github.com/dr-js/dr-dev/actions/runs/92775421

Maybe related to the reduce file count to resolve and parse?
Still not sure if both plugin is need to trigger the exit, will continue reduce test later.

dr-js · 2020-05-01T06:17:27Z

Update:

Enabled more debug log and found the exit happens before the @babel/env plugin gets load:

{TIMESTAMP} babel:config:loading:files:configuration Found configuration 'd:\\a\\dr-dev\\dr-dev\\babel.config.js' from 'd:\\a\\dr-dev\\dr-dev'.
{TIMESTAMP} babel:config:loading:files:plugins Loaded plugin 'minify-replace' from 'd:\\a\\dr-dev\\dr-dev'.
{TIMESTAMP} babel:config:loading:files:plugins Loaded plugin 'module-resolver' from 'd:\\a\\dr-dev\\dr-dev'.
<-- may exit with exit code -1073741819 -->
{TIMESTAMP} babel:config:loading:files:plugins Loaded preset '@babel/env' from 'd:\\a\\dr-dev\\dr-dev'.
source\babel.js -> output-gitignore\library\babel.js
{TIMESTAMP} babel:config:loading:files:configuration Found configuration 'd:\\a\\dr-dev\\dr-dev\\babel.config.js' from 'd:\\a\\dr-dev\\dr-dev'.
{TIMESTAMP} babel:config:loading:files:configuration Found configuration 'd:\\a\\dr-dev\\dr-dev\\babel.config.js' from 'd:\\a\\dr-dev\\dr-dev'.
source\common\dev.js -> output-gitignore\library\common\dev.js

Related CI run: https://github.com/dr-js/dr-dev/actions/runs/92845322

notable change: - break: use `@dr-js/core@0.3.0-dev.11` and `@dr-js/node@0.3.0-dev.7` - ci: mark `windows-latest + 14.x` as unstable. check: nodejs/help#2660 and nodejs/node#33166 - code sort - package update

yperson · 2020-06-04T06:51:52Z

Did you find a workaround or the rootcause?
I'm experiencing the same kind of issue with a simple 'npm test' on a Windows Server 2016 + Node@14 setup when it's scheduled through the Windows Task Scheduler: flaky failures with 3221225477 error code.
Running 'npm test' in a terminal works 100% of the time though.

dr-js · 2020-06-04T07:03:38Z

Not yet, for me the error is hard to trigger on my local win10 laptop (1 in 50), but very often on GitHub Action (1 in 3).

With the above added test, my current guess is the problem maybe win10 plus:

the platform is under virtualization (CI)
not work well with too many CPU/RAM
heavy fs IO on some fs/disk/device

yperson · 2020-06-04T07:13:41Z

I can confirm the platform is also virtualized on my side (VM)

notable change: - break: move `getPackageTgzName` to `@dr-js/node` as `toPackageTgzName` in `module/Software/npm` - break: promote `common/terminalColor`, `node/npm/path`, and `parsePackageNameAndVersion` from `node/npm/npxLazy` to `@dr-js/node` - break: use `getGitBranch|getGitCommitHash` from `@dr-js/node` - break: use `createTest` to init test in `common/test` - break: longer default test timeout - break: use `nodejs@14` instead of `nodejs@13` for CI test - break: use `dr-dev -E` instead of `cross-env` - break: update to `eslint-config-almost-standard-v14` - break: update to `eslint-config-almost-standard-jsx-v8` - break: use `getSourceJsFileListFromPathList` instead of `getScriptFileListFromPathList` from `node/filePreset` - break: use `collectSourceJsRouteMap` instead of `collectSourceRouteMap` from `node/export/parsePreset` - break: pass `pathAutoLicenseFile` instead of `pathLicenseFile` to `initOutput()` from `output` - add: `@dr-js/node` as dependency - add: `@dr-js/dev-eslint` config - add: `clearOutput` to `output` - add: `wrapTestScriptStringToHTML` to `puppeteer` - add: cleaner `isTest` script - add: verify rule - add: `source/node/preset` for keep preset path naming rule in one place - add: support `testUrl` for `testWithPuppeteer` - fix: default `pathInfoFilter` in `collectSourceJsRouteMap` - fix: code lint - fix: build script clean `README.md` file - bin: add: `exec` and `exec-load` mode - ci: mark `windows-latest + 14.x` as unstable. check: nodejs/help#2660 and nodejs/node#33166 - ci: test on all 3 platform - ci: update GitHub CI: https://github.blog/changelog/2020-04-15-github-actions-sets-the-ci-environment-variable-to-true/ - ci: update `INIT#.github#workflows#ci-test.yml` - script sort - package update

dr-js · 2020-07-11T06:33:28Z

Update:

just did a few test with nodejs@14.4.0 and the problem seems gone.
also verify again with nodejs@14.1.0 still error with: exit code -1073741819

Related CI run:

test 14.4.0: https://github.com/dr-js/dr-dev/actions/runs/165398610 (pass, and 2 run before)
verify 14.1.0: https://github.com/dr-js/dr-dev/actions/runs/165399476 (only the 14.1.0 fail)

notable change: - break: use `webpack@5` - break: use `@dr-js/*@0.4.0-dev.*` - ci: unmark `windows-latest + 14.x` as unstable. issue may be fixed, check: nodejs/help#2660 and nodejs/node#33166 - script sort - package update

usersina · 2021-03-26T11:44:31Z

I have the exact same error when executing an oracledb query with nodejs@14.15.3
Here's the github issue.

mscdex transferred this issue from nodejs/node Apr 30, 2020

dr-js added a commit to dr-js/dr-dev that referenced this issue Apr 30, 2020

TEST: CI: nodejs/help#2660

e908ae4

dr-js mentioned this issue May 1, 2020

[Regression in 14.1.0 - Windows] stdout is sometimes empty nodejs/node#33166

Closed

dr-js closed this as completed Jul 11, 2020

thecrypticace mentioned this issue Dec 28, 2020

[DNM] Log segfaults to stderr laravel-mix/laravel-mix#2714

Closed

hardfist mentioned this issue Aug 30, 2023

chore: takedown node14 test on windows web-infra-dev/rspack#4068

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Flaky ELIFECYCLE/3221225477 exit error with win10/nodejs@14 #2660

Flaky ELIFECYCLE/3221225477 exit error with win10/nodejs@14 #2660

dr-js commented Apr 30, 2020

Hakerh400 commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented May 1, 2020

dr-js commented May 1, 2020

dr-js commented May 1, 2020

yperson commented Jun 4, 2020

dr-js commented Jun 4, 2020

yperson commented Jun 4, 2020

dr-js commented Jul 11, 2020

usersina commented Mar 26, 2021

Flaky ELIFECYCLE/3221225477 exit error with win10/nodejs@14 #2660

Flaky ELIFECYCLE/3221225477 exit error with win10/nodejs@14 #2660

Comments

dr-js commented Apr 30, 2020

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior?

What do you see instead?

Additional information

Hakerh400 commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented Apr 30, 2020

dr-js commented May 1, 2020

dr-js commented May 1, 2020

dr-js commented May 1, 2020

yperson commented Jun 4, 2020

dr-js commented Jun 4, 2020

yperson commented Jun 4, 2020

dr-js commented Jul 11, 2020

usersina commented Mar 26, 2021