BzlMod external Dependency instructions are not working

[bdleitner]

What version of rules_go are you using?

0.50.1

What version of gazelle are you using?

0.38.0

What version of Bazel are you using?

7.3.1

Does this issue reproduce with the latest releases of all the above?

Yes.

What operating system and processor architecture are you using?

Windows 10 x64

Any other potentially useful information about your toolchain?

No

What did you do?

I tried to use the rules_go bzlmod docs to set up a very simple dependency between local modules and then later with an external dependency (com_github_golang_protobuf) just to test that I knew what I was doing, but even those minimal examples failed.

I created a repo for my minimal examples at https://github.com/bdleitner/golang_bzlmod_demo

What did you expect to see?

bazel run @rules_go/go -- mod tidy -v should update the go.mod file and not show errors, and should update the use_repo section of MODULE.bazel as well.

Once the use_repo section is manually updated, bazel build should succeed.

What did you see instead?

bazel run @rules_go/go -- mod tidy -v reports errors some errors and does not update the go.mod file or the use_repo section of MODULE.bazel. bazel mod tidy removes entries from use_repo that I need.

Once the use_repo section is manually updated, bazel build should succeed - this seems to work for the github.com/golang/protobuf dependency if I fix everything manually, but the tooling at best doesn't help and at worst tries to remove needed entries. Nothing seems to work for "local" external dependencies.

[fmeum]

Thanks for the very detailed description of the challenges you faced.

We don't currently support absolute paths in replace directives and I'm not sure we will any time soon: Bazel is very much designed to not reference absolute paths in a build.

You might want to look into go.work support if you really want a multi-module project, but I would heavily recommend against that unless you already have such a build. With Bazel, there isn't much of a point to this as you can create arbitrary package structures and enforce relations between them via visibility. A monorepo is definitely closer to the happy path with Bazel.

[fmeum]

Regarding protobuf, you are running into an issue ultimately fixed by this commit 2f062f8 a few hours ago. go get doesn't work well with non-go build build systems as it never marks dependencies as direct. Unfortunately upstream has decided to leave this as is, but with the commit above, at least @rules_go//go should give you a better experience.

You could also try to run go mod tidy with the -e flag to continue through errors - definitely recommended if you have generated code.

We should definitely improve the docs, so if you identify anything that's missing, it would be highly appreciated if you could send a PR.

[bdleitner]

We don't currently support absolute paths in replace directives and I'm not sure we will any time soon: Bazel is very much designed to not reference absolute paths in a build.

I'm working on consolidating into a monorepo, but for curiosity: the issues with trying to get "local" remote repos to work also manifest in the same way if I used relative paths (e.g. "../otherworkspace") in the go.mod file. Would that fall under the same "not supported" issue?

[fmeum]

Relative paths should work since bazel-contrib/bazel-gazelle#1776. But I haven't tried this myself yet, so if you find something that needs to be fixed, please let me know.

[bdleitner]

I tried to revisit this today since I have plans for a little project that I want to keep out of my main "monolith" repo but still be ablw to depend on some of my common library code from it.

I went back to my demo from golang_bzlmod_demo to look at the dependency. I didn't make changes, but it's been long enough that bazelisk downloaded bazel version 7.4.1.

I tried removing the references to the greeting package from go.mod and MODULE.bazel and rerunning bazel run @rules_go//go -- mod tidy but that got the error:

go: finding module for package bdl.com/greetings
go: bdl.com/hello imports
        bdl.com/greetings: cannot find module providing package bdl.com/greetings: unrecognized import path "bdl.com/greetings": reading https://bdl.com/greetings?go-get=1: 404 Not Found

Ok, that makes sense, it doesn't know (anymore/yet) to look locally.

So I added replace bdl.com/greetings => c:/projects/go/demos/greetings

To the go.mod file and reran it. This time:

go: found bdl.com/greetings in bdl.com/greetings v0.0.0-00010101000000-000000000000

rules_go: Running 'bazel mod tidy' since go.mod changed...
WARNING: C:/projects/go/demos/hello/MODULE.bazel:10:24: The module extension go_deps defined in @gazelle//:extensions.bzl reported incorrect imports of repositories via use_repo():                                                
                                                                                                                                                                                                                                    
Not imported, but reported as direct dependencies by the extension (may cause the build to fail):
    com_bdl_greetings

Fix the use_repo calls by running 'bazel mod tidy'.
INFO: Updated use_repo calls for @gazelle//:extensions.bzl%go_deps

So it complained at first and said the the imports for use_repo were incorrect and I should fix them with go mod tidy (the command I was running) but then it proceeded to re-add that line to MODULE.bazel in addition to adding require bdl.com/greetings v0.0.0-00010101000000-000000000000 in go.mod.

A subsequent run of bazel run @rules_go//go -- mod tidy just runs ok with no warnings.

At this point, the go.mod and MODULE.bazel files again look like they do in the hello directory of my combined github demo.

But trying to bazel build :hello still fails:

PS C:\projects\go\demos\hello> bazel build :hello
INFO: Repository gazelle~~go_deps~com_bdl_greetings instantiated at:
  <builtin>: in <toplevel>
Repository rule go_repository defined at:
  D:/_bazel_out/5y5g3qvt/external/gazelle~/internal/go_repository.bzl:363:32: in <toplevel>
ERROR: D:/_bazel_out/5y5g3qvt/external/gazelle~/internal/go_repository.bzl:282:13: An error occurred during the fetch of repository 'gazelle~~go_deps~com_bdl_greetings':                                                           
   Traceback (most recent call last):                                                                                                                                                                                               
        File "D:/_bazel_out/5y5g3qvt/external/gazelle~/internal/go_repository.bzl", line 282, column 13, in _go_repository_impl
                fail("%s: %s" % (ctx.name, result.stderr))
Error in fail: gazelle~~go_deps~com_bdl_greetings: fetch_repo: read c:\projects\go\demos\greetings\bazel-bin: Incorrect function.
ERROR: no such package '@@gazelle~~go_deps~com_bdl_greetings//': gazelle~~go_deps~com_bdl_greetings: fetch_repo: read c:\projects\go\demos\greetings\bazel-bin: Incorrect function.                                                 
ERROR: C:/projects/go/demos/hello/BUILD:6:11: //:hello_lib depends on @@gazelle~~go_deps~com_bdl_greetings//:greetings in repository @@gazelle~~go_deps~com_bdl_greetings which failed to fetch. no such package '@@gazelle~~go_deps~com_bdl_greetings//': gazelle~~go_deps~com_bdl_greetings: fetch_repo: read c:\projects\go\demos\greetings\bazel-bin: Incorrect function.                                                                                           
ERROR: Analysis of target '//:hello' failed; build aborted: Analysis failed                                                                                                                                                         
INFO: Elapsed time: 1.625s, Critical Path: 0.00s                                                                                                                                                                                    
INFO: 1 process: 1 internal.                                                                                                                                                                                                        
ERROR: Build did NOT complete successfully

I don't understand the error message or what "incorrect function" means here. It seems like there is some error in trying to actually read the other repo's bazel-bin dir (which I've confirmed exists, and even a bazel build ... in the other repo doesn't help.

So it looks like things are still stuck. The absolute vs. relative path doesn't seem to make a difference. I can change the replace path from the absolute one to ../greetings and bazel run @rules_go//go -- mod tidy works just as well as before to refill in MODULE.bazel and go.mod but I get the same "incorrect function" error when I try to build.