Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bzlmod: Labels pointing to external repos in default value of command line flags #14279

Closed
meteorcloudy opened this issue Nov 15, 2021 · 2 comments
Assignees
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request

Comments

@meteorcloudy
Copy link
Member

meteorcloudy commented Nov 15, 2021

Problem

Some Bazel native rules depend on targets that are specified via command line options, the default value of those options may point to targets in certain external repositories.
See https://cs.opensource.google/search?q=%22defaultValue%20%3D%20%5C%22@%22&ss=bazel

For example:

  @Option(
      name = "crosstool_top",
      defaultValue = "@bazel_tools//tools/cpp:toolchain",
      converter = LabelConverter.class,
      ...
      help = "The label of the crosstool package to be used for compiling C++ code.")
  public Label crosstoolTop;

@bazel_tools//tools/cpp:toolchain is essentially an alias to @local_config_cc//:toolchain.
With WORKSPACE, repository @local_config_cc is ensured to be available because we load cc_configure() in WORKSPACE suffix.

With Bzlmod, this is going to be migrated to to rules_cc, cc_autoconf(name = "local_config_cc") will still be loaded via module extension, but its canonical repo name will be @rules_cc.<version>.<extension name>.local_config_cc.

Therefore, @bazel_tools//tools/cpp:toolchain will no longer be able to locate @local_config_cc//:toolchain because @bazel_tools can only see module extension repos via canonical repo names (no repo mappings is applied). We can manually override the target by --crosstool_top=@rules_cc.<version>.<extension name>.local_config_cc//:toolchain as a workaround, but obviously we need a better solution for this.

Proposed solution

  • Make bazel_tools a default Bazel module, which can depends on other normal Bazel modules (eg. rules_cc) that provides those targets required by native rules.
  • @bazel_tools//tools/cpp:toolchain is still an alias to @local_config_cc//:toolchain, but this time because bazel_tools is a Bazel module, we'll apply proper repo mappings for @local_config_cc//:toolchain, then the alias could work.
  • Other flags that point to repos other than @bazel_tools will have to use @bazel_tools as a bridge. For example, xcode_version_config
@meteorcloudy meteorcloudy added type: feature request P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. labels Nov 15, 2021
@gregestren gregestren added the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label Nov 18, 2021
@philwo philwo removed the team-OSS Issues for the Bazel OSS team: installation, release processBazel packaging, website label Nov 29, 2021
@meteorcloudy meteorcloudy added the area-Bzlmod Bzlmod-specific PRs, issues, and feature requests label Mar 23, 2022
bazel-io pushed a commit that referenced this issue May 11, 2022
- no longer need override --crosstool_top and --xcode_version_config since #14279 is fixed
- update abseil-cpp version to fix build on latest macOS.

PiperOrigin-RevId: 447976139
Wyverald added a commit to bazelbuild/rules_java that referenced this issue May 16, 2022
@meteorcloudy
Copy link
Member Author

@Wyverald With the default @bazel_tools module, we can probably also redirect labels pointing to @com_google_protobuf in Bazel's code so that we don't have to specify it as a well known module?

@Wyverald
Copy link
Member

I was thinking that should be solved by #14852, but @bazel_tools already depends on protobuf, so we can indeed already do the redirect. I'll do the change.

copybara-service bot pushed a commit that referenced this issue Jul 5, 2022
Certain flags have default label values pointing to within @com_google_protobuf. Since the built-in module bazel_tools already has a dependency on @com_google_protobuf, we can simply create aliases for those targets in @bazel_tools, which removes the need for protobuf to be a well-known module.

Related: #14279
PiperOrigin-RevId: 459066103
Change-Id: I583f28ace7afe2bc91997293bb9870f836e393eb
lyqaiym pushed a commit to lyqaiym/bazel that referenced this issue May 14, 2023
3.1 bazelbuild#15168

distdir_deps.bzl 文件修改
"abseil-cpp": {
        "archive": "997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
        "sha256": "35f22ef5cb286f09954b7cc4c85b5a3f6221c9d4df6b8c4a1e9d399555b366ee",
        "urls": [
            "https://mirror.bazel.build/github.com/abseil/abseil-cpp/archive/997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
            "https://github.com/abseil/abseil-cpp/archive/997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
        ],
        "used_in": [
            "additional_distfiles",
            "test_WORKSPACE_files",
        ],
    },
"com_google_absl": {
        "archive": "20211102.0.tar.gz",
        "sha256": "dcf71b9cba8dc0ca9940c4b316a0c796be8fab42b070bb6b7cab62b48f0e66c4",
        "urls": [
            "https://mirror.bazel.build/github.com/abseil/abseil-cpp/archive/refs/tags/20211102.0.tar.gz",
            "https://github.com/abseil/abseil-cpp/archive/refs/tags/20211102.0.tar.gz",
        ],
        "used_in": [
            "additional_distfiles",
            "test_WORKSPACE_files",
        ],
        "strip_prefix": "abseil-cpp-20211102.0",
    },

WORKSPACE 文件增加
# Override the abseil-cpp version defined in grpc_deps(), which doesn't work on latest macOS
# Fixes bazelbuild#15168
dist_http_archive(
    name = "com_google_absl",
)

  3.2 src/main/java/net/starlark/java/eval/EvalUtils.java:399: 错误: 找不到符号
      return s.repeat(n);
              ^
  符号:   方法 repeat(int)
  位置: 类型为String的变量 s

    .bazelrc
     # Enable Bzlmod
     build:bzlmod --experimental_enable_bzlmod
     # TODO(pcloudy): The following should be removed after fixing bazelbuild#14279
     build:bzlmod --crosstool_top=@rules_cc.0.0.1.cc_configure.local_config_cc//:toolchain
     build:bzlmod --xcode_version_config=@rules_cc.0.0.1.cc_configure.local_config_xcode//:host_xcodes

     # Enable Java 11 language features (bazelbuild#14592)
     # Toolchain resolution configuration for Bazel >= 5
     build --java_language_version=11
     build --tool_java_language_version=11
     # Legacy configuration for Bazel <= 4
     build --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
     build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
lyqaiym pushed a commit to lyqaiym/bazel that referenced this issue May 14, 2023
3.1 bazelbuild#15168

distdir_deps.bzl 文件修改
"abseil-cpp": {
        "archive": "997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
        "sha256": "35f22ef5cb286f09954b7cc4c85b5a3f6221c9d4df6b8c4a1e9d399555b366ee",
        "urls": [
            "https://mirror.bazel.build/github.com/abseil/abseil-cpp/archive/997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
            "https://github.com/abseil/abseil-cpp/archive/997aaf3a28308eba1b9156aa35ab7bca9688e9f6.tar.gz",
        ],
        "used_in": [
            "additional_distfiles",
            "test_WORKSPACE_files",
        ],
    },
"com_google_absl": {
        "archive": "20211102.0.tar.gz",
        "sha256": "dcf71b9cba8dc0ca9940c4b316a0c796be8fab42b070bb6b7cab62b48f0e66c4",
        "urls": [
            "https://mirror.bazel.build/github.com/abseil/abseil-cpp/archive/refs/tags/20211102.0.tar.gz",
            "https://github.com/abseil/abseil-cpp/archive/refs/tags/20211102.0.tar.gz",
        ],
        "used_in": [
            "additional_distfiles",
            "test_WORKSPACE_files",
        ],
        "strip_prefix": "abseil-cpp-20211102.0",
    },

WORKSPACE 文件增加
# Override the abseil-cpp version defined in grpc_deps(), which doesn't work on latest macOS
# Fixes bazelbuild#15168
dist_http_archive(
    name = "com_google_absl",
)

  3.2 src/main/java/net/starlark/java/eval/EvalUtils.java:399: 错误: 找不到符号
      return s.repeat(n);
              ^
  符号:   方法 repeat(int)
  位置: 类型为String的变量 s

    .bazelrc
     # Enable Bzlmod
     build:bzlmod --experimental_enable_bzlmod
     # TODO(pcloudy): The following should be removed after fixing bazelbuild#14279
     build:bzlmod --crosstool_top=@rules_cc.0.0.1.cc_configure.local_config_cc//:toolchain
     build:bzlmod --xcode_version_config=@rules_cc.0.0.1.cc_configure.local_config_xcode//:host_xcodes

     # Enable Java 11 language features (bazelbuild#14592)
     # Toolchain resolution configuration for Bazel >= 5
     build --java_language_version=11
     build --tool_java_language_version=11
     # Legacy configuration for Bazel <= 4
     build --java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
     build --host_java_toolchain=@bazel_tools//tools/jdk:toolchain_java11
     -------------
     def use_cpp_toolchain(mandatory = True):
         """
         Helper to depend on the c++ toolchain.

         Usage:
         ```
         my_rule = rule(
             toolchains = [other toolchain types] + use_cpp_toolchain(),
         )
         ```

         Args:
           mandatory: Whether or not it should be an error if the toolchain cannot be resolved.
             Currently ignored, this will be enabled when optional toolchain types are added.

         Returns:
           A list that can be used as the value for `rule.toolchains`.
         """
         return [CPP_TOOLCHAIN_TYPE]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-Bzlmod Bzlmod-specific PRs, issues, and feature requests P2 We'll consider working on this in future. (Assignee optional) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: feature request
Projects
None yet
Development

No branches or pull requests

4 participants