Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

addons/register-signal-handler/test and abort/test-addon-register-signal-handler failed #34410

Open
richardlau opened this issue Jul 17, 2020 · 17 comments · May be fixed by #34648
Open

addons/register-signal-handler/test and abort/test-addon-register-signal-handler failed #34410

richardlau opened this issue Jul 17, 2020 · 17 comments · May be fixed by #34648
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. ibm i Issues and PRs related to the IBM i platform.

Comments

@richardlau
Copy link
Member

  • Test: addons/register-signal-handler/test and abort/test-addon-register-signal-handler
  • Platform: IBM i
  • Console Output:
09:41:12 not ok 2801 addons/register-signal-handler/test
09:41:12   ---
09:41:12   duration_ms: 8.21
09:41:12   severity: fail
09:41:12   exitcode: 1
09:41:12   stack: |-
09:41:12     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6 reset 1
09:41:12     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  1
09:41:12     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  2
09:41:12     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 11  1
09:41:12     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 11  2
09:41:12     assert.js:103
09:41:12       throw new AssertionError(obj);
09:41:12       ^
09:41:12     
09:41:12     AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
09:41:12     + actual - expected
09:41:12     
09:41:12       [
09:41:12     -   11,
09:41:12         11
09:41:12       ]
09:41:12         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js:54:12)
09:41:12         at Module._compile (internal/modules/cjs/loader.js:1252:30)
09:41:12         at Object.Module._extensions..js (internal/modules/cjs/loader.js:1273:10)
09:41:12         at Module.load (internal/modules/cjs/loader.js:1101:32)
09:41:12         at Function.Module._load (internal/modules/cjs/loader.js:966:14)
09:41:12         at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:60:12)
09:41:12         at internal/main/run_main_module.js:17:47 {
09:41:12       generatedMessage: true,
09:41:12       code: 'ERR_ASSERTION',
09:41:12       actual: [ 11 ],
09:41:12       expected: [ 11, 11 ],
09:41:12       operator: 'deepStrictEqual'
09:41:12     }
09:41:12   ...
...
09:44:59 not ok 2901 abort/test-addon-register-signal-handler
09:44:59   ---
09:44:59   duration_ms: 9.300
09:44:59   severity: fail
09:44:59   exitcode: 1
09:44:59   stack: |-
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6 reset 1
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6 reset 2
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  1
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  2
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 11  1
09:44:59     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 11  2
09:44:59     assert.js:103
09:44:59       throw new AssertionError(obj);
09:44:59       ^
09:44:59     
09:44:59     AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
09:44:59     + actual - expected
09:44:59     
09:44:59       [
09:44:59     -   11,
09:44:59         11
09:44:59       ]
09:44:59         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js:54:12)
09:44:59         at Module._compile (internal/modules/cjs/loader.js:1252:30)
09:44:59         at Object.Module._extensions..js (internal/modules/cjs/loader.js:1273:10)
09:44:59         at Module.load (internal/modules/cjs/loader.js:1101:32)
09:44:59         at Function.Module._load (internal/modules/cjs/loader.js:966:14)
09:44:59         at Module.require (internal/modules/cjs/loader.js:1141:19)
09:44:59         at require (internal/modules/cjs/helpers.js:75:18)
09:44:59         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/abort/test-addon-register-signal-handler.js:7:1)
09:44:59         at Module._compile (internal/modules/cjs/loader.js:1252:30)
09:44:59         at Object.Module._extensions..js (internal/modules/cjs/loader.js:1273:10) {
09:44:59       generatedMessage: true,
09:44:59       code: 'ERR_ASSERTION',
09:44:59       actual: [ 11 ],
09:44:59       expected: [ 11, 11 ],
09:44:59       operator: 'deepStrictEqual'
09:44:59     }
09:44:59   ...

abort/test-addon-register-signal-handler runs the test from addons/register-signal-handler/test with environment variable ALLOW_CRASHES = true.

node/test/abort/test-addon-register-signal-handler.js

Lines 4 to 7 in 08e8997

// This is a sibling test to test/addons/register-signal-handler/
process.env.ALLOW_CRASHES = true;
require('../addons/register-signal-handler/test');

cc @nodejs/platform-ibmi

  • Build Links:
@richardlau richardlau added flaky-test Issues and PRs related to the tests with unstable failures on the CI. ibm i Issues and PRs related to the IBM i platform. labels Jul 17, 2020
@richardlau
Copy link
Member Author

Didn't fail in the subsequent manually requested run of the master branch: https://ci.nodejs.org/job/node-test-commit-ibmi/35/

@richardlau
Copy link
Member Author

Has not reoccurred on any of the subsequent daily builds.

@richardlau
Copy link
Member Author

I've still not seen this fail again. Stress run (100x): https://ci.nodejs.org/job/node-stress-single-test/141/nodes=ibmi72-ppc64/

@richardlau
Copy link
Member Author

The 100x stress run all passed.
Here's another with 1000x: https://ci.nodejs.org/job/node-stress-single-test/143/nodes=ibmi72-ppc64/

If that passes then I'm leaning towards closing this as unrecreatable.

@richardlau
Copy link
Member Author

Those all passed too. At this point I'm going to close as unrecreatable. We can always reopen if it does reoccur.

@richardlau
Copy link
Member Author

ugh, it's back: https://ci.nodejs.org/job/node-test-commit-ibmi/45/nodes=ibmi72-ppc64/

@richardlau richardlau reopened this Jul 24, 2020
@ThePrez
Copy link
Contributor

ThePrez commented Jul 24, 2020

@dmabupt, any ideas?

@richardlau
Copy link
Member Author

Another 1000 runs, no failures: https://ci.nodejs.org/job/node-test-commit-custom-suites-freestyle/15579/

@Trott
Copy link
Member

Trott commented Jul 24, 2020 •
edited
Loading

Here it is failing around 9 hours ago. https://ci.nodejs.org/job/node-test-commit-ibmi/45/

Host: test-iinthecloud-ibmi72-ppc64_be-1

01:35:42 not ok 2906 abort/test-addon-register-signal-handler
01:35:42   ---
01:35:42   duration_ms: 6.608
01:35:42   severity: fail
01:35:42   exitcode: 1
01:35:42   stack: |-
01:35:42     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6 reset 1
01:35:42     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6 reset 2
01:35:42     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  1
01:35:42     Running: node /home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js child 6  2
01:35:42     assert.js:103
01:35:42       throw new AssertionError(obj);
01:35:42       ^
01:35:42     
01:35:42     AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
01:35:42     + actual - expected
01:35:42     
01:35:42       [
01:35:42     -   6,
01:35:42         6
01:35:42       ]
01:35:42         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/addons/register-signal-handler/test.js:54:12)
01:35:42         at Module._compile (internal/modules/cjs/loader.js:1270:30)
01:35:42         at Object.Module._extensions..js (internal/modules/cjs/loader.js:1291:10)
01:35:42         at Module.load (internal/modules/cjs/loader.js:1119:32)
01:35:42         at Function.Module._load (internal/modules/cjs/loader.js:981:14)
01:35:42         at Module.require (internal/modules/cjs/loader.js:1159:19)
01:35:42         at require (internal/modules/cjs/helpers.js:75:18)
01:35:42         at Object.<anonymous> (/home/iojs/build/workspace/node-test-commit-ibmi/nodes/ibmi72-ppc64/test/abort/test-addon-register-signal-handler.js:7:1)
01:35:42         at Module._compile (internal/modules/cjs/loader.js:1270:30)
01:35:42         at Object.Module._extensions..js (internal/modules/cjs/loader.js:1291:10) {
01:35:42       generatedMessage: true,
01:35:42       code: 'ERR_ASSERTION',
01:35:42       actual: [ 6 ],
01:35:42       expected: [ 6, 6 ],
01:35:42       operator: 'deepStrictEqual'
01:35:42     }
01:35:42   ...

@Trott
Copy link
Member

Trott commented Jul 24, 2020 •
edited
Loading

@richardlau Every failure documented here has been on test-iinthecloud-ibmi72-ppc64_be-1 and every successful stress test has been on test-iinthecloud-ibmi72-ppc64_be-2 so maybe there's a difference between the two machines?

@richardlau

This comment has been minimized.

Sign in to view
@richardlau
Copy link
Member Author

Stress run on test-iinthecloud-ibmi72-ppc64_be-1: https://ci.nodejs.org/job/node-stress-single-test/148/nodes=ibmi72-ppc64/

@richardlau
Copy link
Member Author

@dmabupt It looks like the reproducibility of this is low but based on the stress runs (same test run over and over) below we've now seen this on both of the test CI machines (test-iinthecloud-ibmi72-ppc64_be-1 and test-iinthecloud-ibmi72-ppc64_be-2).

machine fails/repetitions build
test-iinthecloud-ibmi72-ppc64_be-2 0/100 https://ci.nodejs.org/job/node-stress-single-test/141/nodes=ibmi72-ppc64/
test-iinthecloud-ibmi72-ppc64_be-2 0/1000 https://ci.nodejs.org/job/node-stress-single-test/143/nodes=ibmi72-ppc64/
test-iinthecloud-ibmi72-ppc64_be-1 1/1000 https://ci.nodejs.org/job/node-stress-single-test/148/nodes=ibmi72-ppc64/
test-iinthecloud-ibmi72-ppc64_be-2 0/1000 https://ci.nodejs.org/job/node-stress-single-test/149/nodes=ibmi72-ppc64/
test-iinthecloud-ibmi72-ppc64_be-2 1/930 https://ci.nodejs.org/job/node-stress-single-test/150/nodes=ibmi72-ppc64/console
test-iinthecloud-ibmi72-ppc64_be-2 ???/1000 https://ci.nodejs.org/job/node-stress-single-test/153/nodes=ibmi72-ppc64/

@richardlau richardlau linked a pull request Aug 6, 2020 that will close this issue
Open
2 tasks
@richardlau
Copy link
Member Author

Seen again in today's daily: https://ci.nodejs.org/job/node-test-commit-ibmi/58/nodes=ibmi72-ppc64/

@richardlau
Copy link
Member Author

Still occasionally seeing this on the CI. Latest instance: https://ci.nodejs.org/job/node-test-commit-ibmi/221/nodes=ibmi72-ppc64/

@ThePrez
Copy link
Contributor

ThePrez commented Jan 14, 2021

Based on internal discussions with the IBM kernel team, I think this one can be skipped as I believe it is in z/OS, due to the way the platform handles signals.

@richardlau richardlau mentioned this issue Jan 18, 2021
Closed
Trott pushed a commit that referenced this issue Jan 19, 2021
@richardlau @Trott
test: mark flaky tests on IBM i
863ec7d 
PR-URL: #36986
Refs: #34410
Refs: #36925
Refs: #36929
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
ruyadorno pushed a commit that referenced this issue Jan 22, 2021
@richardlau @ruyadorno
test: mark flaky tests on IBM i
3a122aa 
PR-URL: #36986
Refs: #34410
Refs: #36925
Refs: #36929
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
ruyadorno pushed a commit that referenced this issue Jan 25, 2021
@richardlau @ruyadorno
test: mark flaky tests on IBM i
7acea78 
PR-URL: #36986
Refs: #34410
Refs: #36925
Refs: #36929
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
@Trott
Copy link
Member

Trott commented Mar 16, 2021 •
edited
Loading

Based on internal discussions with the IBM kernel team, I think this one can be skipped as I believe it is in z/OS, due to the way the platform handles signals.

@ThePrez Does that mean this is a bug in z/OS and eventually we can stop skipping the test? Or does this mean that z/OS handles signals differently than other platforms by design and is working as intended?

targos pushed a commit that referenced this issue May 1, 2021
@richardlau @targos
test: mark flaky tests on IBM i
5408f51 
PR-URL: #36986
Refs: #34410
Refs: #36925
Refs: #36929
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Michael Dawson <midawson@redhat.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. ibm i Issues and PRs related to the IBM i platform.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

src: set SA_SIGINFO flag when using sa_sigaction richardlau/node-1
3 participants
@Trott @richardlau @ThePrez