-
Notifications
You must be signed in to change notification settings - Fork 105
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve reporting of tests coverage #1893
Comments
Any drawback on this solution? It seems trivial and also I don't think cleanliness is a concerns, we need to add the tests so good thing to add a placeholder |
I dont think there are any drawbacks, it should be straightforward. Probably can be added manually now, and for future if new package is added without any test files, codecov PR check should catch it, so we dont need any additional CI checks for this? |
We should prevent in our code reviews to let someone add a new package without any tests |
Describe the Issue
Currently, for packages without any
_test.go
file,go test
reports[no-test-files]
and coverage is not included at all (eg.common/bitcoin
package, not present incodecov
report, because there are no tests, latest develop report: https://app.codecov.io/gh/zeta-chain/node/tree/develop/common). Reason for this is that packages that are without test files are not included incoverage.out
which is uploaded tocodecov
.Expected Outcome
These type of packages should be reported with 0% coverage, so they are detected and included in overall coverage, otherwise coverage is not correct.
Details
There are a lot of details in this issue: golang/go#24570 and it seems that this works as expected in go1.22 https://tip.golang.org/doc/go1.22#tools where 0% coverage is reported instead. But for now we can do the following (summary from various github issues):
prefered solution: add empty
_test.go
file in packages that are reported with[no test files]
and we want to include in the coverage, for example adding empty_test.go
file incommon/bitcoin
package shows it in the coverage.While this is not super clean, it seems like the most accurate solution (also recommended throughout this issue, some projects even implemented CI to automatically add this).
use
-coverpkg=./...
option withgo test
- from docs:This kinda works, but not quite. If package doesn't contain any test, but test of some other package calls it, it will still be covered. Eg:
common/chain.go
functionIsHeaderSupportedEvmChain
is not covered with unit tests incommon
package, but because it is called inmessage_add_block_header.go
, which has tests, it is also reported as covered when this flag is used. This is not correct in context of unit tests coverage, where each package that is reporting coverage is expected to have its own tests.Also, it doesn't solve the problem completely - if package is not referenced in any test, it still won't be included in coverage.
Furthermore, this flag has multiple bugs after go1.19, which is solved by using
GOEXPERIMENT=nocoverageredesign
(different coverage results are reported with and without this flag: golang/go#58770 and it is confirmed locally).The text was updated successfully, but these errors were encountered: