-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Bug][CI] integration tests not currently running in ci_gpu image #12529
Comments
ONNX testing that runs as part of Integration tests is also not running anywhere. One is actually broken, #12532 |
@gigiblender is it possible to leverage #12436 to add another GH check to assert we aren't skipping a couple of critical integration tests in a PR? |
Sure thing. We could change that script to pass a list of critical tests to ensure that some tests are not skipped and call it in a GH check. Let me know if that sounds right. |
In this PR, the skipped tests script will also check if tests in the `required_tests_to_run.json` have not been skipped. If there are skipped tests, they will be added to the returned comment. I am not entirely sure where it's best to place the `required_tests_to_run` file, so I left it in `tvm/ci/scripts/`. I am happy to take suggestions. Aims to prevent situations such as #12529
In this PR, the skipped tests script will also check if tests in the `required_tests_to_run.json` have not been skipped. If there are skipped tests, they will be added to the returned comment. I am not entirely sure where it's best to place the `required_tests_to_run` file, so I left it in `tvm/ci/scripts/`. I am happy to take suggestions. Aims to prevent situations such as apache#12529
When looking at recent CI test run (which I'm enabling torch/ONNX tests on AArch64), I noticed an issue with a missing
transformers
package while executingtest_meta_schedule_integration_extract_from_bert_base
, causing the integration tests to fail on AArch64.Investigating the issue a bit deeper, it seems that integration tests are not running on
ci_GPU
image, which is the one that is supposed to containtorch
and therefore would reproduce the problem.I'm raising this issue just because I won't have the time in the next weeks to enable that package and test the GPU image and test it to solve the problem. So I create this PR #12528 to make the test to be properly skipped if
transformers
is not installed, but leaving this issue so that we can investigate two potential problems:transformers
installation, if that is what we want (pending to be confirmed, I didn't fully investigate)cc @Mousius @areusch @driazati @gigiblender
The text was updated successfully, but these errors were encountered: