cmd/go: go module does not honor a local git override to manage multiple git identities

[Ravenwater]

What version of Go are you using (go version)?

$ go version
go version go1.12 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
set GOARCH=amd64
set GOBIN=C:\Users\tomtz\Documents\dev\Golang\bin
set GOCACHE=C:\Users\tomtz\AppData\Local\go-build
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\tomtz\Documents\dev\Golang
set GOPROXY=
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=C:\Users\tomtz\Documents\dev\ibm\kms\sst\tools\cli\protect\go.mod
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\tomtz\AppData\Local\Temp\go-build179743270=/tmp/go-build -gno-record-gcc-switches

What did you do?

I have two git identities. One for open source contributions, and one for private repo work.

I have a private repo tree, and I use a local .git/config override to work as my private repo identity in that tree. Git operations function as expected in this tree. This private repo has go code that refers to modules that are also hosted on this private domain.

However, go build/mod tidy, etc. use the global git identity, and thus fail to download the dependencies that are in the private repo.

What did you expect to see?

I expected go build to pick up my local .git/config override and take on the private repo git identity with the proper redirects to access the private repo.

What did you see instead?

Instead, the go build tools pick up the global .gitconfig and use that to access the dependencies. When I replace the global .gitconfig with my local override that has the right url adapter, the go tools succeed.

[andybons]

@bcmills @jayconrod

[bcmills]

go build etc. download modules to a GOPATH-wide module cache, so it is certainly not feasible to segregate module dependencies based on the .git/config in the current module.

We could perhaps do some sort of GOPATH-wide configuration, but we don't currently have such a thing. (See #30411 for a related proposal, but that's still per-user rather than per-GOPATH.)

[Ravenwater]

My thinking was that since the module cache contains digital artifacts that are orthogonal to git identity, reusing the same cache under GOPATH would not create any strange behavior. It certainly assumes this property how I use multiple git identities mixing open source and private repos alternatingly.

[bcmills]

@Ravenwater The problem with using the same cache is that it mixes together modules fetched using different credentials, so your builds using the public identity could accidentally depend on (and vendor in the contents of) modules fetched using the private one.

On the other hand, if you're ok with mixing those credentials together, why not put them all in your global .git/config? (I had assumed that you were intentionally keeping them separate to avoid that sort of mixing.)

[bcmills]

Note that you could always implement your own module proxy server that understands git configurations (see https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol). For example, such a proxy could change HOME to point to a different directory (containing the alternate .git/config credentials) and then shell out to go mod download to complete the request.

(For all I know, Athens may already do something like that. CC @marwan-at-work for private-repo-proxy ideas.)

[Ravenwater]

@bcmills on mixing together modules in the same cache creating the condition that the public identity might become dependent on a private module: good point, did not synthesize that inconsistency. That would be a failure mode for sure.

So proxy it is. Thank you for your feedback and insights.

[bcmills]

proxy it is.

That being the case, I'm going to close out this issue, but please do file new issues if you run into trouble with the proxy approach!

[marwan-at-work]

You can certainly configure Athens to authenticate with your private code host, see here and here.

Out of curiosity, if go build and go mod tidy set the CWD of the git commands it internally ran to the CWD of where go build got run, shouldn't it pick up on the local git configuration? I could be missing something obvious here and I don't know if that's wanted behavior, but I just thought I'd point it out.

[Ravenwater]

@marwan-at-work I don't know enough about the internal workings of git. Git appears to be able to find the root of the repo, as you can issue git commands inside directories in the repo tree and still get the local git identity that is stored at the root. If cmd/go delegates to git the repo operations, your suggestion that running in the CWD context might indeed all that is needed.

However, that would still leave @bcmills observation that cross contamination between a private repo and a public program would become possible when both share the same module cache.