Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestResult exit code seemingly not surfaced in BEP #25165

Open
aaronsky opened this issue Feb 1, 2025 · 0 comments · May be fixed by #25203
Open

TestResult exit code seemingly not surfaced in BEP #25165

aaronsky opened this issue Feb 1, 2025 · 0 comments · May be fixed by #25203
Labels
category: BEP team-Core Skyframe, bazel query, BEP, options parsing, bazelrc type: bug untriaged

Comments

@aaronsky
Copy link

aaronsky commented Feb 1, 2025

Description of the bug:

I need to perform some additional behavior based on the exit code of the underlying test runner when running with bazel test. rules_apple gives me the capability to do that, based on this line, but Bazel responds with exit code 3 regardless of how the test fails if it built.

This is all standard and reasonable as far as I am aware. I thought to check BEP to see if there was an exit_code field on TestResult. The good news is that there is one, in TestResult.ExecutionInfo! The bad news is that I'm not seeing it in my BEP jsonl output file. My test, which is run standalone, is only providing a timing_breakdown value in its test result.

Does anyone know if this feature is working as intended? I did see in the code for StandaloneTestStrategy that setExitCode is called at some point, but it wasn't clear to me whether it was being called in a location relevant to BEP. Or maybe there's a better way to get at this information?

Which category does this issue belong to?

Core

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

  1. Set a value for --build_event_json_file
  2. Write a test that fails in an unusual way
  3. Run the test with a test runner that returns some non-conventional exit code, such as 65.
  4. Observe the absent exit_code field in the BEP output for the failed testResult

Which operating system are you running Bazel on?

macOS 15.3

What is the output of bazel info release?

release 7.4.1

Have you found anything relevant by searching the web?

https://bazelbuild.slack.com/archives/CA31HN1T3/p1738199682813539

Any other information, logs, or outputs that you want to share?

No response

@aaronsky aaronsky changed the title TestResult exit code seemingly not passed to BEP TestResult exit code seemingly not surfaced in BEP Feb 1, 2025
@sgowroji sgowroji added team-Core Skyframe, bazel query, BEP, options parsing, bazelrc category: BEP labels Feb 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category: BEP team-Core Skyframe, bazel query, BEP, options parsing, bazelrc type: bug untriaged
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants