Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generating coverage output for Haskell tests #699

Merged
merged 18 commits into from
Feb 14, 2019

Conversation

SebastianKG
Copy link
Contributor

@SebastianKG SebastianKG commented Feb 12, 2019

This is my work towards a code coverage solution for Haskell in Bazel. Running this is done best in this form:

bazel coverage //tests/two-libs:two-libs -s --test_output=all

This will show you the coverage report of the chosen haskell_test rule. This is not the standard Bazel coverage report, but instead a custom-made solution to get around the current restrictions for bazel coverage's report-collection capabilities.

However, the above won't do anything on its own. To get a coverage report for a haskell_test, you must establish a required base level of coverage, like this:

haskell_test(
    name = "two-libs",
    srcs = ["Main.hs"],
    expected_expression_coverage = 80,
    deps = [
        ":two",
        "//tests/hackage:base",
    ],
)

This will require expression coverage >= 80%. Only expression coverage is currently supported, for simplicity.

I have noted that not all of the test rules pass yet with this new functionality, but I don't know if immediately handling complex cases is the best approach. My proposal is that we incrementally improve the flexibility of Haskell code coverage in Bazel. After all, it has worked for 0% of the test cases until now. two-libs is an example of a test case which works.

Copy link
Contributor

@guibou guibou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
@Profpatsch
Copy link
Contributor

Am going to review tomorrow.

mboes
mboes previously requested changes Feb 12, 2019
haskell/haskell.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/shell_utils.bzl Outdated Show resolved Hide resolved
Copy link
Contributor

@Profpatsch Profpatsch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! A few trivial and one quoting/templating improvement.

haskell/haskell.bzl Outdated Show resolved Hide resolved
haskell/private/actions/compile.bzl Outdated Show resolved Hide resolved
haskell/private/actions/compile.bzl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
haskell/private/coverage_wrapper.sh.tpl Outdated Show resolved Hide resolved
haskell/private/haskell_impl.bzl Outdated Show resolved Hide resolved
@SebastianKG SebastianKG dismissed mboes’s stale review February 14, 2019 18:54

All changes have been made.

@SebastianKG SebastianKG merged commit 17c2ff2 into master Feb 14, 2019
@SebastianKG SebastianKG deleted the generating_coverage_output branch February 14, 2019 18:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants