Run scheduled benchmark test on self-hosted runner #43
Conversation
|
This PR also fixed #21, the names of test profile, machines, and runners will not contain |
|
@hershd23 @navendu-pottekkat @leecalcote Can you review and give any comments? Thanks. |
Signed-off-by: Huang Xin <xin1.huang@intel.com>
gyohuangxin
force-pushed
the
scheduled-benchmark
branch
from
49d2178 to
ea44022
Compare
There was a problem hiding this comment.
Great work @gyohuangxin. I have made one comment.
I have one more thing to ask, the manual and scheduled runs reuse almost all the code, what is the reason for having another copy entirely for scheduled? Is there any difference in logic that I am unable to see?
| runs-on: ubuntu-latest | ||
| if: ${{ github.event_name == 'schedule' }} | ||
| strategy: | ||
| fail-fast: false |
There was a problem hiding this comment.
When jobs.<job_id>.strategy.fail-fast is set to true, GitHub cancels all in-progress jobs if any matrix job fails. Default: true
https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstrategyfail-fast
Obviously, all tasks run in parallel and should not affect each other.
Thanks for your comments, the matrixs are different between manual and scheduled runs. And I think it's hard to reuse codes from anyone of them because the event types are also different. |
|
Excellent. 😄 |
Description
This PR fixes #38, implements running scheduled benchmark test on self-hosted runner.
There are two method Scheduled benchmarking test can be triggered:
Manual test: we can trigger the test by giving specific test configurations (load-test, soak-test), then the matrix will be tested.
Scheduled test: the matrix test will run everyday.
There are three steps for scheduled benchmarking test like configurable benchmarking test:
Workflows:
Github self-hosted runners:
CNCF CIL machines:
Notes for Reviewers
Signed commits