Provide module extensions with a way to reference extension repos

[fmeum]

What underlying problem are you trying to solve with this feature?

The repository mapping that applies to a module extension does not include the repositories it generates. This makes it very tricky to pass a reference to some file in such a repo to another repo rule instantiated by the same extension, e.g.:

def _ext_impl(module_ctx):
    my_lang_toolchain_repo(
        name = "lang_sdk",
    )
    my_toolchain_repo(
        name = "lang_toolchain",
        # This does *not* work as `lang_sdk` is not in the current file's repo mapping.
        sdk_root = "@lang_sdk//:root",
    )

This is a very common pattern, but requires major hacks to pull of without additional API:

1. Observe that the canonical name of an extension repo is <common prefix>~<apparent name> and rely on that implementation detail to construct the canonical name of lang_sdk in lang_toolchain as ctx.attr.name.removesuffix("lang_toolchain") + "lang_sdk".
2. Make use of the fact that lang_sdk is in the repo mapping of the repositories generated by the extension. By making sdk_root an attr.string instead of an attr.label and generating a .bzl file in some other ("hub") repo in the the same extension that calls Label on a label string, lang_toolchain can pass that literal through the correct repo mapping. This requires the module to use_repo the hub repo and is overall pretty complex.

Description of the feature request:

Provide a function on module_ctx that converts the name of a repository generated by the extension and a repo-relative label into an instance of Label referencing the intended target.

This could look as follows:

# In rules_foo v1.2.3, extensions.bzl
def _ext_impl(module_ctx):
    repo(name = "foo", ...)
    repo(name = "bar", ...)

    # Returns `Label("@rules_foo~1.2.3~ext~foo//dir:name")`
    module_ctx.extension_repo_label("@foo//dir:name")
    # Returns `Label("@rules_foo~1.2.3~ext~bar//:name")`
    module_ctx.extension_repo_label("@bar//:name")

    # Fails immediately, argument has to be absolute
    module_ctx.extension_repo_label("//:name")
    # Fails immediately, label must not be canonical
    module_ctx.extension_repo_label("@@rules_foo~1.2.3~ext~foo//dir:name")
    # Fails immediately, repo name must be a valid "user-specified" name
    module_ctx.extension_repo_label("@rules_foo~1.2.3~ext~foo//dir:name")
    # Fails after the module extension returns as `not_foo` is not generated by the extension.
    module_ctx.extension_repo_label("@not_foo//dir:name")

Alternative ideas for names:

• label_in_repo

Alternative signatures:

• Separate arguments for apparent repository name and repo-relative label.

Alternative approaches:

• Hook into the repo rule creation and defer the conversion of label strings into Label instances until after the module extension has fully evaluated, at which point the names of all the repos it generates are known. Then use this information to convert label strings implicitly. This has the advantage that the "obvious" approach just works and the downside that its implementation is probably more involved.

Which operating system are you running Bazel on?

N/A

What is the output of bazel info release?

6.3.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

CC @Wyverald @meteorcloudy, I am especially interested in your opinion on the alternative approach. If it is feasible, I could see this being better than introducing new API.

[Wyverald]

The alternative approach might not be possible since you could do this:

def _extension_impl(mctx):
  repo_rule(name="foo")
  print(str(Label("@foo//:abc")))
  # or with the API
  print(mctx.extension_repo_label("@foo//:abc"))

To clarify, we can't really "smartly" handle the first print. It's true that we may be able to smartly handle something like repo_rule(name="bar", some_label_attr="@foo//:abc"), but I'd say that if we can't smartly handle everything, we shouldn't smartly handle anything.

[fmeum]

Do we really want users to construct Labels into extension repos though? If they feed them to module_ctx.path, that would probably always result in cycles or other kinds of failures. Supporting label attributes of repo rules is what this is really about and I have a hard time coming up with any other usage that would make sense. But I agree that if we can think of just one then we should just expose a function that is guaranteed to handle all cases.

[Wyverald]

I'm undecided. The decision probably needs more input data. Design doc time :)

[meteorcloudy]

It's interesting to see how rules_java works around this:
https://github.com/bazelbuild/rules_java/blob/0878192f194292421788d35bea53533b0c730aec/toolchains/remote_java_repository.bzl#L86

It basically passes the label ("@{repo}//:jdk".format(repo = name)) as a plain string to generate the BUILD file, so that the label actually got resolved during build time. This works if the label isn't needed in the repo rule implementation function of my_toolchain_repo, which I don't know is often true or false.

[Wyverald]

@bazel-io fork 7.4.0