Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Save test coverage data to bazel-testlogs. #1387

Merged
merged 2 commits into from
Mar 16, 2018

Conversation

jmillikin-stripe
Copy link
Contributor

This lets bazel coverage //some:test produce a Go coverage data file
in bazel-testlogs/some/test/coverage.dat. This file can be processed
by normal Go coverage tooling to produce a coverage report.

The change to go/private/rules/test.bzl needs some explanation --
Bazel only looks for coverage data if the test target has an
InstrumentedFilesProvider, but this provider can currently only be
created using "legacy" provider syntax. Old and new provider syntaxes
can be combined by putting new-style providers in a providers field
of the old-style struct.

If the provider is found and at least one source file is present, Bazel
will set the COVERAGE_OUTPUT_FILE environment variable during tests
and will save that file to the build events + test outputs.

Related to #1150 and #140

@jeb2239
Copy link

jeb2239 commented Mar 16, 2018

Do you know of any coverage tools which work with this dat file or ones that this was tested with?

This lets `bazel coverage //some:test` produce a Go coverage data file
in `bazel-testlogs/some/test/coverage.dat`. This file can be processed
by normal Go coverage tooling to produce a coverage report.

The change to `go/private/rules/test.bzl` needs some explanation --
Bazel only looks for coverage data if the test target has an
`InstrumentedFilesProvider`, but this provider can currently only be
created using "legacy" provider syntax. Old and new provider syntaxes
can be combined by putting new-style providers in a `providers` field
of the old-style struct.

If the provider is found and at least one source file is present, Bazel
will set the `COVERAGE_OUTPUT_FILE` environment variable during tests
and will save that file to the build events + test outputs.
@jmillikin-stripe
Copy link
Contributor Author

This is the plain Go coverage data (same as generated by go test -coverprofile), so any tools based on golang.org/x/tools/cover will be able to process it.

go tool coverage needs a little bit of help because it assumes all your sources are under src/, but this is pretty easy to work around with ln -s . src if you want to use the unmodified upstream SDK.

Copy link
Contributor

@jayconrod jayconrod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I copied the comment from your commit message to test.bzl.

@jayconrod
Copy link
Contributor

The InstrumentedFilesProvider thing is super weird. I wish coverage was getting more attention from the Bazel folks.

I'm guessing we'll need to break this when Bazel has full coverage support. Blaze (Google internal version) expects coverage data in the lcov format, so we may need to write that instead of Go's format.

For now though, this is welcome. Thanks for submitting this.

@jmillikin-stripe
Copy link
Contributor Author

I've been talking with the Bazel devs about coverage as part of the Envoy project, and one of the sticking points is whether coverage should be Blaze-style (unified for all languages), or toolchain-specific.

Being able to gather code coverage in the native format from Go will be useful data for that decision, and I'm hopeful we can expand this approach to other Skylark rulesets.

@jayconrod jayconrod merged commit 6a2a5c5 into bazelbuild:master Mar 16, 2018
@jmillikin-stripe jmillikin-stripe deleted the save-test-coverage-data branch March 16, 2018 19:23
jayconrod pushed a commit that referenced this pull request Mar 29, 2018
This lets `bazel coverage //some:test` produce a Go coverage data file
in `bazel-testlogs/some/test/coverage.dat`. This file can be processed
by normal Go coverage tooling to produce a coverage report.

The change to `go/private/rules/test.bzl` needs some explanation --
Bazel only looks for coverage data if the test target has an
`InstrumentedFilesProvider`, but this provider can currently only be
created using "legacy" provider syntax. Old and new provider syntaxes
can be combined by putting new-style providers in a `providers` field
of the old-style struct.

If the provider is found and at least one source file is present, Bazel
will set the `COVERAGE_OUTPUT_FILE` environment variable during tests
and will save that file to the build events + test outputs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants