Is using "function coverage" metric instead of "line coverage" intended?

[mtjhrc]

While working with @dorindabassey on a gpu in vhost-device, she has noticed the CI seem to be using the "function coverage" metric of llvm-cov. Then I also noticed the comment in the python code of the CI, that seems to suggest "line coverage" is intended to be used.

rust-vmm-ci/integration_tests/test_coverage.py

Lines 101 to 106 in 209c04e

	# Output of llvm-cov is like
	# TOTAL 743 153 79.41% 185 50 72.97% 1531 125 91.84% 0 0 -
	# where the first three numbers are related to region coverage, and next three to line coverage (what we want)
	# and the last three to branch coverage (which is not yet supported). Below grabs the line coverage, and strips
	# off the '%'.
	coverage = float(summary.split()[6][:-1])

The comment here doesn't seem to be correct - at least on my machine the order of the columns in the CLI output of cargo llvm-cov test --summary-only is: Region coverage, Function coverage, Line coverage.

Confusingly the order of the columns is different when using the html output (cargo llvm-cov test --open): Function coverage, Line coverage, Region coverage.

Btw there is also cargo llvm-cov test --json --summary-only which could prevent such confusion.

This means that it seems like the projects using this CI, use the function coverage metric instead of line coverage.
Is this intended or it is it a bug in the parsing of the output? I would think the original intention was to use line coverage.

[roypat]

Hey, thanks for reporting this! Yes, that is indeed supposed to be line coverage! I wonder something changed on llvm-cov's side recently (there was a rust toolchain update that caused weird changes of the coverage numbers all over the place). The json option does indeed sound more robust. I'll have a look into fixing this tomorrow morning (although if you already have a fix, please feel free to open a PR and I'll have a look at that instead :)