Usage in CI

[taiki-e]

codecov claims to support many formats, but it actually happens "There was an error processing coverage reports" errors on the server. It is necessary to understand which formats are actually supported and which ones are preferred to use.

I'm not sure about other code-coverage services as I have never used them.

[jhpratt]

Check out the time-rs/time repo. I just added codecov using json successfully. lcov also worked.

[jhpratt]

Actually I stand corrected. JSON does not work. lcov does. https://app.codecov.io/gh/time-rs/time

[taiki-e]

Thanks @jhpratt. I also checked it in my test repository, it seems to work only with lcov. (Uploading of json and text reports succeeded, but an error happened in the server.)

[jhpratt]

No problem. It looks like codecov only cares about line coverage as well, rather than the more detailed report lcov provides. But I'm pretty sure that's on codecov's end.

[taiki-e]

It looks like codecov only cares about line coverage as well, rather than the more detailed report lcov provides. But I'm pretty sure that's on codecov's end.

Hmm. It seems codecov supports branch coverage, so that might be a rustc's problem (#8, rust-lang/rust#79649).

https://docs.codecov.io/docs/about-code-coverage

partial indicates that the source code was not fully executed by the test suite; there are remaining branches that were not executed.

llvm-cov (that used by cargo-llvm-cov internally) also supports the export of branch coverage.

https://llvm.org/docs/CommandGuide/llvm-cov.html#llvm-cov-export

When exporting an lcov trace file, the line-based coverage, branch coverage, and summaries will be exported.

[taiki-e]

It looks like codecov only cares about line coverage as well, rather than the more detailed report lcov provides. But I'm pretty sure that's on codecov's end.

We can probably support region coverage on codecov by converting llvm-cov's json coverage format to codecov's json coverage format.