cmd/go: support @tip as an alias to "the latest development version"

[mvdan]

Problem statement

Historically, one would use go get -d github.com/foo/bar and the repository would be checked out inside GOPATH. What was nice about it is that the default branch would be used, usually meaning master on git, for example.

Now that we're in the world of modules, we can use go get -d github.com/foo/bar@latest to grab that module's latest released version, regardless of the VCS usptream is using.

However, note that "the latest released version" is not quite the same. If one wants to use the latest available code ignoring releases, the general advice is to use @master. This is equivalent to using the version at the commit pointed to by the branch master.

While git is massively popular and master is also a very popular default branch name for it, this method has significant downsides. For example:

• It likely won't work with other VCS systems, since they often don't have a master branch/tag/ref
• A git repository could easily have a master branch but have a different default branch, leading to confusing behavior
• Since many git repositories are replacing master with main (as will git, eventually!), it will be hard to remember what modules should be used with @master versus @main

In other words, we have no way to say "the latest development version", usually meaning the latest commit from the default branch.

Proposed solution

Just like @latest is a special version, I think we should add another to have the meaning above: the latest development version.

Naming this special version might be hard. I can see arguments in favor of a number of options:

1. @master, since it already has this meaning for many people, and many users and scripts already use it. However, it goes against the effort to replace master with main in git.
2. @main, as an alternative to @master following git.
3. @HEAD, which means "default branch" in git. Very weird to have it be all-caps though, so maybe @head instead.
4. @tip, borrowing from the hg/mercurial terminology
5. @dev, @devel, @unstable, or some other English word meaning "latest version ignoring stable releases"

I personally am in favor of @tip because it's short to type, and because it's very unlikely to cause problems with existing branch/tag names in git, given how such a name would be very confusing to anyone familiar with hg. I'm not sure if it would cause any problems with people currently using @tip with hg repositories.

Related issues

• x/pkgsite: support viewing a package @master #36811 - pkgsite's support for "the latest development version" would likely improve with this change, from the user's perspective and for VCS portability

cc @myitcv @rogpeppe @ianthehat @bcmills @jayconrod

[bcmills]

I'm not sure I understand the use-case.

I would generally expect a user explicitly requesting an unreleased version to have some reason for using that unreleased version, in which case they probably also know what branch or commit they're trying to get — for example, they may have filed a bug, and the maintainer of the module may have replied with a request that the consumer try out the fix before it is tagged. But then the maintainer can also tell them which branch to request...

Could you give some more detail on when this comes up? pkgsite is one example, but that one is a bit distant from the cmd/go command-line API.

[mvdan]

I agree that sometimes one is already familiar with upstream development, but I disagree that that's always/often the case. In fact, having to check the upstream's site/repository before I need to feels like busy work.

For example, my most common use-case is when I think I've found a bug in a third party module, and I want to quickly re-run my tests with their latest development version before I file a bug. I shouldn't need to know what VCS they use to perform this go get, let alone what name they chose for their default branch. I likely wouldn't even need to look up where their issue tracker is, if the latest development version fixes my issue.

Another similar use case is when I use a third party Go project, and I want to help test their development version before final releases. Historically, that would be a plain GOPATH-mode go get or git clone [...] && go build; at no point would I need to know or specify what their default branch is called. With the old go get mode, I wouldn't even have to know where their git repository is.

Also, I should emphasize the point about portability. Using @master has worked for most people for a while, but it's going to break more and more as time goes on:

• many (but not all!) git repositories are moving to main, so the best case scenario is I have to remember (or repeatedly check) which projects have moved and which haven't
• Go does (or could?) support swapping one VCS with another for a module, with the glaring exception that is @master

[rogpeppe]

Like @mvdan, this is a common thing for me to want to do. "Have they fixed this bug on tip?"

[rogpeppe]

I'd support @tip as a name FWIW. If someone has an actual branch that's named "tip" but doesn't represent tip, they probably deserve what they get :)

[bcmills]

Those are interesting use-cases, but I see several very different queries there:

I've found a bug in a third party module, and I want to quickly re-run my tests with their latest development version before I file a bug.

That sounds like “latest commit on the default branch” — but what leads you to believe that their latest development version is on the default branch at all? (They could be actively refactoring on a non-default branch, or fixing bugs on a release branch and backporting the fixes to development.)

Or maybe “latest commit reachable from any branch”? But then the query would potentially hop back and forth between multiple very different branches.

Or maybe “latest commit on the same branch as the latest tagged release”? But that would give you a stale commit for projects that use release branches.

In other words: I think this use case fundamentally requires at least a cursory understanding of how the project organizes their development.

when I use a third party Go project, and I want to help test their development version before final releases.

For projects that issue pre-releases, that sounds to me like “highest tagged version, even if it's a pre-release”, which is certainly an interesting use case but semantically very different from “latest commit on ”.

In contrast, the assumption of “latest commit” would imply a Go-style freeze/thaw cycle or development continuing on a development branch, rather than (say) development continuing on the default branch and release stabilization occurring on a release branch. Again I think that fundamentally requires some understanding of the project.

[bcmills]

I think we could do the “highest tagged version, even if it's a pre-release” behavior within cmd/go today, without any change to the proxy protocol, and it would generalize pretty well to basically any project that uses pre-release tags.

The only thing we could do in terms of “latest commit on the default branch” that generalizes to the existing proxy protocol would be “@latest or the latest version in @v/list, whichever is higher”. Unfortunately, the clearest name I can think of for that is latest, which is already taken. 😅

[seankhliao]

I believe go does this for repos without any tags, it seems strange to lose this capability once you have a tag, no matter how old it may be. Not having to remember which branch most projects are going to use for latest stable-ish development is a big plus, we can save the brainpower for the few that use a different development model.

[mvdan]

That sounds like “latest commit on the default branch” — but what leads you to believe that their latest development version is on the default branch at all? (They could be actively refactoring on a non-default branch, or fixing bugs on a release branch and backporting the fixes to development.)

Yes, at least in git terms, I do mean latest commit on the default branch. I do generally expect a VCS repository to have the default branch be the latest development, at least as new as the last tag. If a project doesn't follow that pattern, I could always check their docs/site and use @branch instead. It doesn't exclude being able to use @tip as a sane first route.

For projects that issue pre-releases, that sounds to me like “highest tagged version, even if it's a pre-release”, which is certainly an interesting use case but semantically very different from “latest commit on ”.

If someone wants to propose @latest-unstable, I think that would be reasonable. I'm not proposing that, though :)

I generally disagree that Go projects developing their latest code in a non-default branch will be a problem. If that were true, then it would have been a problem for the old GOPATH-mode go get too. I really want the same semantics we want before, which we lost. I believe those are the same semantics we lose when any tags exist in a module, like @seankhliao mentions.

[mvdan]

For what it's worth, I'm happy to restrict the wording from "the latest development version" to "the latest commit of the default branch". I only went for the former because the latter felt very git-centric.

[hyangah]

#29761 :-)
I don't know what is the implication to the module proxy protocol though (what's the freshness requirement, etc). If this ever gets implemented, my hope is the go command switches in the VCS mode to get the module source code (while checksum db check isn't skipped).

[mvdan]

@hyangah thanks for the link, I hadn't seen that issue. It seems like the general resolution there was "we don't want to encourage this behavior" - I disagree with that point today, because plenty of how-tos and docs already tell people to use @master.

I wouldn't mind if this mode skipped module proxies, to be honest. Or support in the proxy protocol could be done at a later time if that would be easier.

[jayconrod]

What would need to change in the proxy protocol to get "the latest commit of the default branch"?

I would think that if the go command is using a proxy, it would send a request to $proxy/$modpath/@v/tip.info, and that would return the same JSON that the go command would return (the proxy would run GOPROXY=direct go mod download $modpath@tip). That's the same thing we do when resolving master and other branch names to pseudo-versions.

The TTL should be short, but probably no different than other branches.

[bcmills]

Hmm, that's a good point. We don't need to change the proxy protocol if we use @v/something.info instead of @latest!

[hyangah]

@jayconrod Yes, it's the same as master or other branch names. What I meant is - because there is no freshness guarantee, if the user's intention of using @tip is to get fresh data that's committed now, serving through the proxy isn't a good idea. It can be a false illusion. If what users expect from @tip is a reasonably recent (?) commit that was once in the default branch, maybe it's ok.

I don't think proxy.golang.org serving @v/master.info was particularly a good idea either.

On the other hand, I think proxies can choose to just return 404/410 on @v/master.info and @v/tip.info to force the go command to fallback to direct mode. Users can also specify GOPRIVATE to avoid proxies causing surprises.