Base/CI: Run tests from /usr/Tests in GitHub Actions #6459
Conversation
ADKaster
force-pushed
the
enable-all-tests
branch
from
19a9abe
to
ac8b130
Compare
ADKaster
force-pushed
the
enable-all-tests
branch
5 times, most recently
from
21f5e98
to
a4ae46c
Compare
ADKaster
force-pushed
the
enable-all-tests
branch
3 times, most recently
from
5159e90
to
cc21d56
Compare
ADKaster
force-pushed
the
enable-all-tests
branch
3 times, most recently
from
591751c
to
d9f0424
Compare
ADKaster
force-pushed
the
enable-all-tests
branch
3 times, most recently
from
3141961
to
c22ba0d
Compare
ADKaster
force-pushed
the
enable-all-tests
branch
from
c22ba0d
to
ffe43bd
Compare
| # test-js: We start this one manually with the show progress flag set to false | ||
| exclude_patterns='Kernel|.inc|UserEmulator|stack-smash|TestJSON|.frm|test-web|test-js' | ||
|
|
||
| system_tests=((test-js --show-progress=false) (test-crypto -c -t test)) |
There was a problem hiding this comment.
Why don't the test-js and test-math failures in the run log cause the CI to fail?
There was a problem hiding this comment.
Only returning a non-zero exit code from the script would fail the CI job.
https://docs.github.com/en/actions/creating-actions/setting-exit-codes-for-actions
actions/runner#662
Was talking with CxByte about this on discord earlier, might be worth to write out the "test result" to a file and mount the disk image after ninja run to check out what's up with it? But curiously, when I run locally in Qemu with CPU=max,vmx=off, those particular tests don't fail...
This makes it easier to run tests we know will work in CI, and ignore ones that need some help to be repeatable.
Uncomment the tests that were disabled due to frequent freezes when running without KVM. This also adds a new github actions group for every single test, which makes it easier to browse test boundaries during test runs. Move catting the serial output log back to its own step, so that it has higher visibility. The previous solution was also shown to not actually cat the log in the case of a failed boot and timeout :^(.
ADKaster
force-pushed
the
enable-all-tests
branch
from
ffe43bd
to
d59d56f
Compare
Userland/Shell/Job.h
Outdated
| @@ -31,7 +27,7 @@ class Job : public RefCounted<Job> { | |||
|
|
|||
| ~Job() | |||
| { | |||
| #ifdef JOB_TIME_INFO | |||
| #if SHELL_JOB_DEBUG | |||
There was a problem hiding this comment.
n i t:
| #if SHELL_JOB_DEBUG | |
| if constexpr (SHELL_JOB_DEBUG) { |
There was a problem hiding this comment.
(and replace the #endif below with the good old })
There was a problem hiding this comment.
le fixed, and moved the macro in AK/Debug.in.h to its proper sorted location at the same time
ADKaster
force-pushed
the
enable-all-tests
branch
from
d59d56f
to
123e774
Compare
Uncomment running the tests that were disabled due to frequent freezes when running without KVM.
Fixes #5541
This still skips many tests held in the
exclude_patternsshell variable in the test script.Also, hide the dbg spam from shell jobs dying behind a debug flag. Without it, the console gets absolutely clobbered when running shell tests.