switch to source based coverage

[yaahc]

Motivation

Source-based coverage is more accurate and useful than 'gcov-based' coverage that we currently rely on:

Above: A comparison of the gcov (left) and source-based coverage (right) results. gcov highlights skipped lines, marked with #####, while source-based coverage highlights exact regions of code that were skipped. Note that on line 30, one boolean subexpression is short-circuited. This is surfaced by source-based coverage but not gcov.

https://blog.rust-lang.org/inside-rust/2020/11/12/source-based-code-coverage.html

Solution

This PR switches to using -Zinstrument-coverage to calculate coverage using llvm's source based coverage instrumentation. We then upload that report in two different formats. First, we create an html report using llvm-cov show, which includes the exact coverage report, including details down to the specific regions in a line that are covered, and coverage per expansion of generic types.

A second format is then used to upload the coverage to codecov, this is because codecov doesn't currently support region based coverage results, so we have to fall back to an older format that they do understand. This discards some information and shows lines as covered when they're only partially covered, but should still be useful for tracking statistics and for ease of access.

The code in this pull request has:

• Documentation Comments
• Unit Tests and Property Tests

Review

@dconnolly

Related Issues

Follow Up Work

[yaahc]

@dconnolly okay this is ready to merge probably. I've still got an ongoing ticket in the codecov support forum for getting region based coverage reports into codecov, but for now it seems like they just don't support it yet, so we're probably out of luck until I can nerd snipe one of their engineers into prioritizing it.

[dconnolly]

@yaahc I would merge but for it looks like the coverage reported via this is missing the code executed for some of our tests such as unit/proptests such as in zebra_chain:

#main (tarpaulin): https://codecov.io/gh/ZcashFoundation/zebra/src/main/zebra-chain/src/sprout/keys.rs
this branch (-Zinstrument-coverage): https://app.codecov.io/gh/ZcashFoundation/zebra/compare/1293/tree/zebra-chain/src/sprout/keys.rs

[yaahc]

alright @dconnolly, I think i figured out what was causing it to lose coverage for zebra-chain

new coverage: https://codecov.io/gh/ZcashFoundation/zebra/src/75c202891cb996b8ecc090f5b4360d87ca93aeec/zebra-chain/src/sprout/keys.rs

old coverage: https://codecov.io/gh/ZcashFoundation/zebra/src/main/zebra-chain/src/sprout/keys.rs

It seems to be getting a more accurate report now, except for on the proptest lines, where it doesn't show coverage hits for some reason, tho this same issue doesn't happen in disk_format.rs where I use proptest closures, so I'm guessing it has to do with how the source mappings are done before and after the proptest! macro is applied, and that it doesn't matter.

[dconnolly]

One q: zebra-utils contains mostly tools that are useful to a general zcash/zebra user, vs our coverage script, which is mostly useful only to developers. If it were shorter I would suggest inlining it in the workflow file but I understand why it's not. Is there a better location for it, like a plain scripts or utils maybe?

[yaahc]

not sure, I think we should probably discuss this as a team, I'll put an item on the agenda for our next zebra sync.