Skip to content

Commit

Permalink
[CI] Enable failed test collection on ES-intake triggered serverless …
Browse files Browse the repository at this point in the history
…verification runs (elastic#191937)

## Summary
Currently, failed runs on
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote
that were triggered by the ES Serverless pipeline's downstream, are not
collecting failures. This makes it harder to grasp what set of tests
failed.

I think we originally disabled this reporting not to muddy our stats
based on runs on various ES Serverless versions - however, with this,
it's really hard to get an overview on which tests are failing for a
run, so I'd like to get the failed suites to be displayed on the job.

With this change - if I'm not mistaken - we'll generate the annotation
on the Buildkite job, but we won't report to ci-stats or create github
issues.

### In short
What we have:
<img width="889" alt="Screenshot 2024-09-02 at 18 28 45"
src="https://github.com/user-attachments/assets/d294dc00-4dd0-4b29-9e2f-866d68246a5d">

What I'd like:
<img width="1195" alt="Screenshot 2024-09-02 at 18 29 14"
src="https://github.com/user-attachments/assets/455465ba-f176-4ab5-ba31-9578c167448f">


Tested in:
https://buildkite.com/elastic/kibana-elasticsearch-serverless-verify-and-promote/builds/1962
  • Loading branch information
delanni committed Sep 4, 2024
1 parent a2dbebc commit 243de82
Showing 1 changed file with 9 additions and 6 deletions.
15 changes: 9 additions & 6 deletions .buildkite/scripts/lifecycle/post_command.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,15 @@ if [[ "$IS_TEST_EXECUTION_STEP" == "true" ]]; then
buildkite-agent artifact upload '.es/**/*.hprof'
buildkite-agent artifact upload 'data/es_debug_*.tar.gz'

if [[ $BUILDKITE_COMMAND_EXIT_STATUS -ne 0 ]] && \
# Skip when triggered from elasticsearch's validation pipeline
[[ $BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG != 'elasticsearch-serverless-intake' ]]
then
echo "--- Run Failed Test Reporter"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'
if [[ $BUILDKITE_COMMAND_EXIT_STATUS -ne 0 ]]; then
if [[ $BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG == 'elasticsearch-serverless-intake' ]]; then
echo "--- Run Failed Test Reporter (only junit)"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'\
--no-github-update --no-index-errors
else
echo "--- Run Failed Test Reporter"
node scripts/report_failed_tests --build-url="${BUILDKITE_BUILD_URL}#${BUILDKITE_JOB_ID}" 'target/junit/**/*.xml'
fi
fi

if [[ -d 'target/test_failures' ]]; then
Expand Down

0 comments on commit 243de82

Please sign in to comment.