Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RBE pre- and postsubmit broken due to broken bazel-toolchains version check #13502

Closed
philwo opened this issue May 21, 2021 · 4 comments
Closed

RBE pre- and postsubmit broken due to broken bazel-toolchains version check #13502

philwo opened this issue May 21, 2021 · 4 comments
Labels
P0 This is an emergency and more important than other current work. (Assignee required) type: bug

Comments

@philwo
Copy link
Member

philwo commented May 21, 2021

Details to follow.
@philwo philwo added type: bug P0 This is an emergency and more important than other current work. (Assignee required) labels May 21, 2021
@philwo
Copy link
Member Author

philwo commented May 21, 2021

A version check in the rbe_repository rule from bazel-toolchains is wrong and assumes that Bazel >= 4.1.0 will have switched to Java platformization. However, this is not actually happening until Bazel 5.0.0.

A fix for bazel-toolchains was made, but due to many projects (Bazel included) not actively tracking the bazel-toolchains releases we didn't pick up this fix: bazelbuild/bazel-toolchains@19884d3

It doesn't help that bazel-toolchains was completely rewritten in the meantime and thus upgrading to it isn't a simple version bump in our WORKSPACE file, but probably more involved.

I will deactivate our RBE-based pre- and postsubmit tests for now.

@philwo
Copy link
Member Author

philwo commented May 21, 2021

For affected other users, please note that this is not a Bazel bug in 4.1.0, the bug is in bazel-toolchains and we have no way to fix or workaround it. You will have to upgrade to the latest bazel-toolchains version.

@philwo philwo mentioned this issue May 21, 2021
Closed
9 tasks
bazel-io pushed a commit that referenced this issue May 21, 2021
@philwo @copybara-github
Disable RBE pre- and postsubmit due to breakage.
5cee034 
Due to a broken version check in bazel-toolchains, all Bazel versions starting from 4.1.0 fail to build with it and this error message is printed:

ERROR: Traceback (most recent call last):
	File "/var/lib/buildkite-agent/.cache/bazel/_bazel_buildkite-agent/ec321eb2cc2d0f8f91b676b6d4c66c29/external/rbe_ubuntu1604_java8/java/BUILD", line 19, column 60, in <toplevel>
		load("@bazel_tools//tools/jdk:local_java_repository.bzl", "local_java_runtime")
Error: file '@bazel_tools//tools/jdk:local_java_repository.bzl' does not contain symbol 'local_java_runtime'
ERROR: Analysis of target '//src:bazel_jdk_minimal' failed; build aborted: error loading package '@rbe_ubuntu1604_java8//java': Package 'java' contains errors

Bazel tracking issue #13502.

RELNOTES: None.
PiperOrigin-RevId: 375096580
@philwo
Copy link
Member Author

philwo commented May 21, 2021

Fixed via https://github.com/bazelbuild/bazel-toolchains/releases/tag/4.1.0. 🥳

@philwo philwo closed this as completed May 21, 2021
@philwo philwo mentioned this issue May 22, 2021
Closed
@meisterT meisterT mentioned this issue Jun 1, 2021
Open
@philwo philwo mentioned this issue Jun 10, 2021
Closed
@philwo
Copy link
Member Author

philwo commented Jul 1, 2021

In case someone stumbles upon this issue in the future, I'll quote the story and solution from the Bazel 4.1.0 release bug here:

We noticed that bazel-toolchains has a broken version check and Bazel 4.1.0 triggers a code path in it that will cause all builds using bazel-toolchains to fail. This is not a bug in Bazel and there's nothing we could have done to prevent this, apart from never bumping our version to 4.1.0 in the first place. :/

The fix is to upgrade to either bazel-toolchains 4.1.0 or bazel-toolchains 5.1.0, which have a fixed version checks.

Because bazel-toolchains has changed in a major way with version 5.0.0, upgrading to it might not be straightforward if you're still on an older version (but still recommended). As a temporary workaround, we have created a bazel-toolchains 4.1.0 release that you can use as a drop-in replacement for pre-5.0.0 versions: https://github.com/bazelbuild/bazel-toolchains/releases/tag/4.1.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
P0 This is an emergency and more important than other current work. (Assignee required) type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@philwo