CI Reliability 2022-08-01 #334
Comments
kvakil
added a commit
to kvakil/node
that referenced
this issue
Aug 2, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient.
kvakil
added a commit
to kvakil/node
that referenced
this issue
Aug 2, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: nodejs#43499 Refs: nodejs#43084
nodejs-github-bot
pushed a commit
to nodejs/node
that referenced
this issue
Aug 5, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
danielleadams
pushed a commit
to nodejs/node
that referenced
this issue
Aug 16, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
ruyadorno
pushed a commit
to nodejs/node
that referenced
this issue
Aug 23, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
targos
pushed a commit
to nodejs/node
that referenced
this issue
Sep 5, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Fyko
pushed a commit
to Fyko/node
that referenced
this issue
Sep 15, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: nodejs#43499 Refs: nodejs#43084 PR-URL: nodejs#44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
juanarbol
pushed a commit
to nodejs/node
that referenced
this issue
Oct 10, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
juanarbol
pushed a commit
to nodejs/node
that referenced
this issue
Oct 11, 2022
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: #43499 Refs: #43084 PR-URL: #44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
guangwong
pushed a commit
to noslate-project/node
that referenced
this issue
Jan 3, 2023
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: nodejs/node#43499 Refs: nodejs/node#43084 PR-URL: nodejs/node#44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
guangwong
pushed a commit
to noslate-project/node
that referenced
this issue
Jan 3, 2023
These tests seem to timeout quite often. I don't know why, but one possible reason is that they are starting a lot of threads. It seems that tests in `test/parallel` are assumed to only start one thread each, so having 11 threads running at a time feels like a lot. It also seems that these tests fail in a correlated fashion: take a look at [this reliability report][]. The failures all occur on the same build machines on the same PRs. This suggests to me some sort of CPU contention. [this reliability report]: nodejs/reliability#334 On my Linux machine decreasing the parallelism & iterations here reduce the `user` time from ~11.5 seconds to ~2 seconds, depending on the test. I have seen these tests take 30-60 seconds on CI (Alpine in particular). I went back to the diffs that introduced that introduced these changes and verified that they failed at least 90% of the time with the reduced iteration count, which feels sufficient. Refs: nodejs/node#43499 Refs: nodejs/node#43084 PR-URL: nodejs/node#44090 Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com> Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Failures in node-test-pull-request/45660 to node-test-pull-request/45758 that failed more than 2 PRs
(Generated with
ncu-ci walk pr --stats=true --markdown /home/runner/work/reliability/reliability/results.md)Jenkins Failure
Backing channel 'JNLP4-connect connection from ... is disconnected.Example
Build Failure
ERROR: Build step failed with exceptionExample
ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?Example
undefined
UnknownExample
JSTest Failure
parallel/test-worker-heap-snapshotExample
parallel/test-worker-fshandles-error-on-terminationExample
parallel/test-worker-fshandles-open-close-on-terminationExample
parallel/test-net-connect-memleakExample
abort/test-addon-uv-handle-leakExample
parallel/test-vm-break-on-sigintExample
sequential/test-worker-profExample
Progress
Backing channel 'JNLP4-connect connection from ... is disconnected.(10)ERROR: Build step failed with exception(3)ERROR: Step ?Publish JUnit test result report? failed: No test report files were found. Configuration error?(2)Unknown(21)parallel/test-worker-heap-snapshot(6)parallel/test-worker-fshandles-error-on-termination(4)parallel/test-worker-fshandles-open-close-on-termination(4)parallel/test-net-connect-memleak(3)abort/test-addon-uv-handle-leak(2)parallel/test-vm-break-on-sigint(2)sequential/test-worker-prof(2)The text was updated successfully, but these errors were encountered: