{CI} Extension regression test pipeline #25026
Conversation
evelyn-ys
requested review from
kairu-ms,
jiasli,
wangzelin007 and
calvinhzy
as code owners
ghost
added
ghost
requested review from
|
CI |
| displayName: "Rerun tests" | ||
| - bash: | | ||
| publishErrorModules='false' | ||
| if [[ -f '/mnt/vss/.azdev/env_config/mnt/vss/_work/1/s/env/test_results_error_modules_$(Instance_idx).txt' ]]; then |
There was a problem hiding this comment.
We may use variables instead of hard-coding the path /mnt/vss (#24201 (comment)).
There was a problem hiding this comment.
Per @wangzelin007 and my test, $HOME, ~ directs to /home/cloudtest instead of /mnt/vss
Agent.HomeDirectory -> /usr/local/vss-agent/2.214.1
Pipeline.Workspace -> /mnt/vss/_work/1
I can't find the exact variable for /mnt/vss
There was a problem hiding this comment.
Or else we can use $(Pipeline.Workspace)/../..? Looks a liitle bit ugly to me
There was a problem hiding this comment.
We changed the .azdev folder from /home/vsts/.azdev to /mnt/vss/.azdev by mistake in #24201
It should always be ~/.azdev, which is /home/cloudtest/.azdev in new pipeline. I have changed all .azdev path back.
| - task: PublishTestResults@2 | ||
| condition: succeededOrFailed() | ||
| inputs: | ||
| testResultsFiles: '/mnt/vss/.azdev/env_config/mnt/vss/_work/1/s/env/test_results_*.xml' | ||
| testRunTitle: 'CLI Regression test results of instance $(Instance_idx)' |
There was a problem hiding this comment.
It will be great if we can adopt this PublishTestResults task in Azure.azure-cli Full Test pipeline.
| REGRESSION_BRANCHES=$(curl -H "Authorization: token ${GITHUB_TOKEN}" https://api.github.com/repos/azclibot/azure-cli-extensions/git/matching-refs/heads/regression_test_$(Build.BuildId)_ | jq '.[] | .ref') | ||
|
|
||
| for branch in ${REGRESSION_BRANCHES[@]} | ||
| do | ||
| branch=${branch##*/} | ||
| branch=${branch%\"*} |
There was a problem hiding this comment.
Can we add an example to better understand these code?
| if [[ -f '/mnt/vss/.azdev/env_config/mnt/vss/_work/1/s/env/test_results_error_modules_$(Instance_idx).txt' ]]; then | ||
| publishErrorModules='true' | ||
| fi | ||
| echo "##vso[task.setvariable variable=publishErrorModules]$publishErrorModules" |
There was a problem hiding this comment.
This handling is clever and worth a 👍
* init * extension regression test * checkout repo * variable path * checkout extension branch * dependson * typo * extension repo path * script * fix extension command * fix Nonetype * test extension one by one * commit recordings * script path * publish test results * task condition * prepare results * branch name & artifacts path * result summary * fix * create PR * bash array * path and comments
Description
This PR implements CLI Extension regression test pipeline
This checklist is used to make sure that common guidelines for a pull request are followed.
The PR title and description has followed the guideline in Submitting Pull Requests.
I adhere to the Command Guidelines.
I adhere to the Error Handling Guidelines.