Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New repository_ctx.original_name produces "canonical reproducible" warnings #25192

Closed
mbland opened this issue Feb 4, 2025 · 4 comments
Closed
Assignees
Labels
P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug

Comments

@mbland
Copy link

mbland commented Feb 4, 2025

Description of the bug:

Since commit 8bcfb06 introduced repository_ctx.original_name, closing #24467 and #25121, I've noticed the following "canonical reproducible" warnings (formatted for readability) when using the last_green build (at 4c3863e61136a10564f01d920e4f29e142256e7c as of the timestamp below, in Eastern Standard Time):

2025/02/04 10:28:44 Using unreleased version at commit 4c3863e61136a10564f01d920e4f29e142256e7c 

DEBUG: Rule 'rules_java++toolchains+remote_java_tools'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

DEBUG: Rule 'rules_java++toolchains+remote_java_tools_darwin_arm64'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

DEBUG: Rule 'rules_python++python+python_3_11_aarch64-apple-darwin'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

DEBUG: Rule 'rules_java++toolchains+remotejdk11_macos_aarch64'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

DEBUG: Rule 'rules_java++toolchains+remotejdk21_macos_aarch64'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

DEBUG: Rule 'rules_scala++scala_deps+scala_compiler_source_2_12_14'
  indicated that a canonical reproducible form can be obtained
  by dropping arguments ["_original_name"]

All of these except the last are followed by stanzas similar to:

DEBUG: Repository rules_java++toolchains+remote_java_tools instantiated at:
  <builtin>: in <toplevel>
Repository rule http_archive defined at:
  .../external/bazel_tools/tools/build_defs/repo/http.bzl:392:31: in <toplevel>

None of the following Bazel versions produce this error:

Which category does this issue belong to?

External Dependency

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Note that, as I write this, mbland/rules_scala@21e4be9 is the HEAD of the bzlmod-bazel-8 branch of my rules_scala fork. This branch contains my work on bazelbuild/rules_scala#1482 (Bzlmod compatibility) and bazelbuild/rules_scala#1687 (Bazel 8 compatibility).

git clone https://github.com/mbland/rules_scala.git
cd rules_scala
git checkout 21e4be9557645885ba2aabab7922c457ffdf03ee
cd dt_patches/test_dt_patches_user_srcjar
USE_BAZEL_VERSION=4c3863e61136a10564f01d920e4f29e142256e7c bazel clean --expunge  # if not the first build
USE_BAZEL_VERSION=4c3863e61136a10564f01d920e4f29e142256e7c bazel build --repo_env=SCALA_VERSION=2.12.14 //...

Which operating system are you running Bazel on?

macOS Sequoia 15.3 (24D60)

What is the output of bazel info release?

development version

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

Using the last_green build from Bazelisk

What's the output of git remote get-url origin; git rev-parse HEAD ?

4c3863e61136a10564f01d920e4f29e142256e7c

If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

8bcfb06

Have you found anything relevant by searching the web?

It seems a similar problem surfaced in 2020:

Based on the above, it seems the fix may be to add original_name to RepositoryResolvedEvent.IGNORED_ATTRIBUTE_NAMES at:

/**
* Attributes that may be defined on a repository rule without affecting its canonical
* representation. These may be created implicitly by Bazel.
*/
private static final ImmutableSet<String> IGNORED_ATTRIBUTE_NAMES =
ImmutableSet.of("generator_name", "generator_function", "generator_location");

Any other information, logs, or outputs that you want to share?

In the bzlmod-bazel-8 branch of my rules_scala fork, which contains my work on bazelbuild/rules_scala#1482 and bazelbuild/rules_scala#1687, this causes several dt_patches/dt_patch_test.sh test cases to fail.

The failing test_compiler_srcjar cases, which check for ! grep "canonical reproducible", appear towards the end (formatting slightly to split DEBUG: messages on separate lines):

# Assuming `$PWD` is still `dt_patches/test_dt_patches_user_srcjar`
# after the repro steps above:
$ cd ..

# `$PWD` should now be `dt_patches/`
$ USE_BAZEL_VERSION=4c3863e61136a10564f01d920e4f29e142256e7c ./dt_patch_test.sh

[ ...snip... ]

Starting local Bazel server (no_version) and connecting to it...
INFO: Starting clean (this may take a while). Use --async if the clean takes more than several minutes.
running test test_compiler_srcjar 2.12.14
DEBUG: Rule 'rules_java++toolchains+remote_java_tools_darwin_arm64' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
DEBUG: Rule 'rules_java++toolchains+remote_java_tools' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
DEBUG: Rule 'rules_python++python+python_3_11_aarch64-apple-darwin' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
DEBUG: Rule 'rules_java++toolchains+remotejdk11_macos_aarch64' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
DEBUG: Rule 'rules_java++toolchains+remotejdk21_macos_aarch64' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
DEBUG: Rule 'rules_scala++scala_deps+scala_compiler_source_2_12_14' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 2.12.14" failed  (43 sec)
running test test_compiler_srcjar 2.12.15
DEBUG: Rule 'rules_scala++scala_deps+scala_compiler_source_2_12_15' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 2.12.15" failed  (8 sec)
running test test_compiler_srcjar 2.12.16
DEBUG: Rule '+compiler_user_srcjar_repos+scala_compiler_srcjar' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 2.12.16" failed  (7 sec)
running test test_compiler_srcjar_nonhermetic 2.12.17
 Test "test_compiler_srcjar_nonhermetic 2.12.17" successful (8 sec)
running test test_compiler_srcjar_nonhermetic 2.12.18
 Test "test_compiler_srcjar_nonhermetic 2.12.18" successful (8 sec)
running test test_compiler_srcjar_nonhermetic 2.12.19
 Test "test_compiler_srcjar_nonhermetic 2.12.19" successful (8 sec)
running test test_compiler_srcjar_nonhermetic 2.12.20
 Test "test_compiler_srcjar_nonhermetic 2.12.20" successful (10 sec)
running test test_compiler_srcjar_nonhermetic 2.13.11
 Test "test_compiler_srcjar_nonhermetic 2.13.11" successful (7 sec)
running test test_compiler_srcjar_nonhermetic 2.13.12
 Test "test_compiler_srcjar_nonhermetic 2.13.12" successful (8 sec)
running test test_compiler_srcjar_nonhermetic 2.13.14
 Test "test_compiler_srcjar_nonhermetic 2.13.14" successful (9 sec)
running test test_compiler_srcjar_nonhermetic 2.13.15
 Test "test_compiler_srcjar_nonhermetic 2.13.15" successful (8 sec)
running test test_compiler_srcjar 3.1.3
DEBUG: Rule 'rules_scala++scala_deps+scala_compiler_source_3_1_3' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 3.1.3" failed  (8 sec)
running test test_compiler_srcjar 3.2.2
DEBUG: Rule 'rules_scala++scala_deps+scala_compiler_source_3_2_2' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 3.2.2" failed  (8 sec)
running test test_compiler_srcjar_nonhermetic 3.3.5
 Test "test_compiler_srcjar_nonhermetic 3.3.5" successful (8 sec)
running test test_compiler_srcjar 3.4.3
DEBUG: Rule '+compiler_user_srcjar_repos+scala3_compiler_srcjar' indicated that a canonical reproducible form can be obtained by dropping arguments ["_original_name"]
 Test "test_compiler_srcjar 3.4.3" failed  (9 sec)
running test test_compiler_srcjar_nonhermetic 3.5.2
 Test "test_compiler_srcjar_nonhermetic 3.5.2" successful (9 sec)
running test test_compiler_srcjar_nonhermetic 3.6.2
 Test "test_compiler_srcjar_nonhermetic 3.6.2" successful (10 sec)
@fmeum
Copy link
Collaborator

fmeum commented Feb 4, 2025

@bazel-io fork 8.1.0

@fmeum fmeum added P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. and removed untriaged labels Feb 4, 2025
@fmeum
Copy link
Collaborator

fmeum commented Feb 4, 2025

FYI @Wyverald

fmeum added a commit to fmeum/bazel that referenced this issue Feb 4, 2025
bazel-io pushed a commit to bazel-io/bazel that referenced this issue Feb 5, 2025
Fixes bazelbuild#25192

Closes bazelbuild#25194.

PiperOrigin-RevId: 723594158
Change-Id: I12aaced1db4f167c4a9691623128fe0d3eb15686
bazel-io pushed a commit to bazel-io/bazel that referenced this issue Feb 5, 2025
Fixes bazelbuild#25192

Closes bazelbuild#25194.

PiperOrigin-RevId: 723594158
Change-Id: I12aaced1db4f167c4a9691623128fe0d3eb15686
github-merge-queue bot pushed a commit that referenced this issue Feb 5, 2025
Fixes #25192

Closes #25194.

PiperOrigin-RevId: 723594158
Change-Id: I12aaced1db4f167c4a9691623128fe0d3eb15686

Commit
5e2dfba

Co-authored-by: Fabian Meumertzheim <fabian@meumertzhe.im>
@iancha1992
Copy link
Member

A fix for this issue has been included in Bazel 8.1.0 RC2. Please test out the release candidate and report any issues as soon as possible.
If you're using Bazelisk, you can point to the latest RC by setting USE_BAZEL_VERSION=8.1.0rc2. Thanks!

@mbland
Copy link
Author

mbland commented Feb 6, 2025

Can confirm this issue is resolved by Bazel 8.1.0 RC2, as well as the current last_green release from 5e2dfba. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. type: bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants