Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Run E2E tests on PVC on postcommit #15308

Merged
merged 37 commits into from
Dec 24, 2024
Merged

Conversation

uditagarwal97
Copy link
Contributor

@uditagarwal97 uditagarwal97 commented Sep 5, 2024

This PR enables running E2E tests on PVC in the post-commit.

Currently, the latest PVC driver hangs when a large number of jobs are submitted to it in parallel. Due to this, E2E tests might timeout spuriously on PVC. The PVC bug is being tracked internally via GSD-10241. To prevent spurious test timeouts, we restrict the number of parallel E2E test runs on PVC and to prevent disruption in CI, we are currently enabling PVC in post-commit only. When the bug gets fixed, we can move PVC testing to pre-commit.

@uditagarwal97 uditagarwal97 self-assigned this Sep 5, 2024
@uditagarwal97 uditagarwal97 marked this pull request as ready for review September 5, 2024 22:04
@uditagarwal97 uditagarwal97 requested a review from a team as a code owner September 5, 2024 22:04
@uditagarwal97
Copy link
Contributor Author

Need to update CI container first to test changes in this PR: #15324

@uditagarwal97 uditagarwal97 marked this pull request as draft September 12, 2024 22:28
@uditagarwal97
Copy link
Contributor Author

@intel/llvm-gatekeepers I think we are good to merge this PR. The following test failure on PVC w/ igc dev is unrelated and an existing PR (#16452) is there to mark it XFAIL on PVC.

Failed Tests (1):
  SYCL :: Matrix/SG32/get_coordinate_ops.cpp

@sarnex sarnex merged commit 864038a into sycl Dec 24, 2024
25 of 31 checks passed
@bader bader deleted the sycl-devops-pr/udit/pvc_e2e branch December 24, 2024 18:32
sarnex pushed a commit to sarnex/llvm that referenced this pull request Dec 24, 2024
This PR enables running E2E tests on PVC in the post-commit.

Currently, the latest PVC driver hangs when a large number of jobs are
submitted to it in parallel. Due to this, E2E tests might timeout
spuriously on PVC. The PVC bug is being tracked internally via
GSD-10241. To prevent spurious test timeouts, we restrict the number of
parallel E2E test runs on PVC and to prevent disruption in CI, we are
currently enabling PVC in post-commit only. When the bug gets fixed, we
can move PVC testing to pre-commit.
sarnex pushed a commit that referenced this pull request Jan 3, 2025
#15308 added PVC to post-commit. Since then, we have not seen any
spurious E2E test timeouts on PVC, so testing on PVC seems stable enough
to be moved to pre-commit.
Running E2E tests on PVC takes only about 5 minutes so moving PVC
testing to pre-commit shouldn't cause any noticeable slowdown in pre
commit testing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants