cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

dev-gto · 2024-02-07T13:08:05Z

Go version

go version go1.22.0 linux/amd64

Output of `go env` in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/hamada/.cache/go-build'
GOENV='/home/hamada/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/hamada/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/hamada/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go1.22.0'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go1.22.0/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/hamada/go/src/gitlab.in.registro.br/dev/br/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3637209596=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Having a directory called, e.g., holiday with some go files and their corresponding test files, I called the go test -cover from the parent directory. Depending on directories arguments, I have different percentage values, which wasn't occurring on go < 1.22.0.

What did you see happen?

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./holiday/
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./h*
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	0.005s	coverage: 47.3% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./...
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 23.5% of statements
...

What did you expect to see?

A consistent percentage value (in this case, 97.6%) for all commands:

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./holiday/
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./h*
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	0.005s	coverage: 97.6% of statements

DEV=dev go test -tags dev -cover -buildvcs=false -ldflags '-s -w' ./...
...
ok  	gitlab.in.registro.br/dev/br/core/holiday	(cached)	coverage: 97.6% of statements
...

The text was updated successfully, but these errors were encountered:

bcmills · 2024-02-07T14:54:19Z

This is almost certainly an effect of recent changes in -coverpkg:

cmd/cover: regression in 1.20, cover tool no longer includes packages without tests #58770 (Go 1.22.0)
cmd/cover: extend coverage testing to include applications #51430 (Go 1.19)

See previously:

cmd/go: -coverpkg=all gives different coverage value when run on a package list vs ./... #23883

(attn @thanm)

thanm · 2024-02-07T15:14:23Z

Thanks for the report. I'll take a look.

thanm · 2024-02-07T15:52:51Z

@dev-gto would you kindly post a pointer to the code that you are working with, so that I can reproduce the problem on my machine? Thanks.

dev-gto · 2024-02-07T17:51:20Z

@thanm , actually my project is a private and large one.

Despite some testing errors (the original project does not emit any errors), you can reproduce with
https://github.com/registrobr/zxcvbn-go

go test -cover ./...
ok  	github.com/registrobr/zxcvbn-go	(cached)	coverage: 60.6% of statements
ok  	github.com/registrobr/zxcvbn-go/adjacency	(cached)	coverage: 40.8% of statements
	github.com/registrobr/zxcvbn-go/data		coverage: 0.0% of statements
	github.com/registrobr/zxcvbn-go/data/ptbr		coverage: 0.0% of statements
ok  	github.com/registrobr/zxcvbn-go/entropy	(cached)	coverage: 35.9% of statements
...

go test -cover ./[ae]*
ok  	github.com/registrobr/zxcvbn-go/adjacency	(cached)	coverage: 83.3% of statements
ok  	github.com/registrobr/zxcvbn-go/entropy	(cached)	coverage: 50.0% of statements

AlexanderYastrebov · 2024-02-08T11:46:30Z

#62212 might be related

aaron42net · 2024-02-08T20:43:27Z

GOEXPERIMENT=nocoverageredesign go test -cover ./... does appear to provide the expected coverage result, matching running go test -cover for each package separately. So this does seem related to the coverage redesign.

I've confirmed this with both @dev-gto's example above and a large private monorepo where I also encountered the issue.

dev-gto · 2024-02-09T09:41:14Z

Indeed, GOEXPERIMENT=nocoverageredesign returns the expected values on my original files

dev-gto · 2024-04-05T11:41:03Z

hi!
Any news on this? go 1.22.2 still has this issue.

jehowell · 2024-04-17T23:05:07Z

Code coverage has been broken for about a year now... Please can someone make this a priority to fix.

sfc-gh-jlangefeld · 2024-04-20T02:21:40Z

I came here because I just ran into this as well on 1.22.2

# Description This is to upgrade to 1.22.2. However we might need to wait until the fix for coverage drop issue is ready - golang/go#65570 ## Type of change - This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Radius (issue link optional).  Fixes: #issue_number Signed-off-by: Young Bu Park <youngp@microsoft.com>

gopherbot · 2024-06-13T11:45:46Z

Change https://go.dev/cl/592204 mentions this issue: internal/coverage: refactor EmitPercent in preparation for bugfix

gopherbot · 2024-06-13T11:45:47Z

Change https://go.dev/cl/592205 mentions this issue: cmd/go: fix problems with coverage percentage reporting w/ -coverpkg

Refactor cformat.EmitPercent to accept a package filter (list of packages to report). This is a no-op in terms of exposed coverage functionality, but we will need this feature in a subsequent patch. Updates #65570. Change-Id: I04ddc624a634837ea31c12ec395aa1295a0ea1f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/592204 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: David Chase <drchase@google.com>

dev-gto · 2024-09-05T16:33:24Z

1.23.1 and 1.22.7 still have this issue, although it was closed.

dev-gto · 2024-09-05T18:16:59Z

1.23.1 and 1.22.7 still have this issue, although it was closed.

sorry about that, actually it is working

pepehandsjpg · 2024-09-11T13:04:43Z

1.23.1 and 1.22.7 still have this issue, although it was closed.

sorry about that, actually it is working

is it? Can't see the difference

dev-gto changed the title ~~go 1.22.0: inconsistent and variable code coverage output for the same folder~~ cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder Feb 7, 2024

bcmills added the GoCommand cmd/go label Feb 7, 2024

bcmills added this to the Backlog milestone Feb 7, 2024

thanm self-assigned this Feb 7, 2024

thanm added the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 7, 2024

alainstoffels mentioned this issue Feb 9, 2024

cmd/go: duplicate lines in coverage file after update to 1.22 #65636

Open

jhendrixMSFT mentioned this issue Feb 16, 2024

Disable code coverage redesign in CI Azure/azure-sdk-for-go#22402

Merged

jasonphi mentioned this issue Feb 16, 2024

testing: flags from different packages interact with GOEXPERIMENT=nocoverageredesign #65763

Closed

seankhliao removed the WaitingForInfo Issue is not actionable because of missing required information, which needs to be provided. label Feb 16, 2024

AlexanderYastrebov mentioned this issue Feb 19, 2024

Update client-go/kubectl to Kubernetes v1.25 zalando-incubator/cluster-lifecycle-manager#748

Merged

xuezhaojun mentioned this issue Mar 4, 2024

🌱 Update dependencies to k8s 1.29 client libs open-cluster-management-io/managed-serviceaccount#96

Merged

c-pius mentioned this issue Apr 18, 2024

Create a unit test coverage report for all packages kyma-project/lifecycle-manager#1439

Closed

3 tasks

This was referenced Apr 27, 2024

Upgrade go to 1.22.2 radius-project/radius#7533

Closed

Upgrade go to 1.22.2 radius-project/radius#7559

Merged

ChrisHines mentioned this issue May 31, 2024

internal/goexperiment: remove CoverageRedesign some time in 1.23 #55953

Open

singholt mentioned this issue Jun 5, 2024

update go to 1.22.3 aws/amazon-ecs-agent#4184

Merged

gopherbot closed this as completed in a85b881 Jun 14, 2024

gabyhelp mentioned this issue Jul 29, 2024

Go package coverage is calculated inconsistently in 1.22.5 #68627

Closed

gkampitakis mentioned this issue Sep 6, 2024

Coverage not showing correct value gkampitakis/coverage#2

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

dev-gto commented Feb 7, 2024

bcmills commented Feb 7, 2024

thanm commented Feb 7, 2024

thanm commented Feb 7, 2024

dev-gto commented Feb 7, 2024 •

edited

Loading

AlexanderYastrebov commented Feb 8, 2024

aaron42net commented Feb 8, 2024

dev-gto commented Feb 9, 2024

dev-gto commented Apr 5, 2024

jehowell commented Apr 17, 2024

sfc-gh-jlangefeld commented Apr 20, 2024

gopherbot commented Jun 13, 2024

gopherbot commented Jun 13, 2024

dev-gto commented Sep 5, 2024

dev-gto commented Sep 5, 2024

pepehandsjpg commented Sep 11, 2024

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

cmd/go: go 1.22.0 inconsistent and variable code coverage output for the same folder #65570

Comments

dev-gto commented Feb 7, 2024

Go version

Output of go env in your module/workspace:

What did you do?

What did you see happen?

What did you expect to see?

bcmills commented Feb 7, 2024

thanm commented Feb 7, 2024

thanm commented Feb 7, 2024

dev-gto commented Feb 7, 2024 • edited Loading

AlexanderYastrebov commented Feb 8, 2024

aaron42net commented Feb 8, 2024

dev-gto commented Feb 9, 2024

dev-gto commented Apr 5, 2024

jehowell commented Apr 17, 2024

sfc-gh-jlangefeld commented Apr 20, 2024

gopherbot commented Jun 13, 2024

gopherbot commented Jun 13, 2024

dev-gto commented Sep 5, 2024

dev-gto commented Sep 5, 2024

pepehandsjpg commented Sep 11, 2024

Output of `go env` in your module/workspace:

dev-gto commented Feb 7, 2024 •

edited

Loading