Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade rules_java to 7.10.0 #1226

Merged
merged 4 commits into from
Sep 5, 2024
Merged

Conversation

comius
Copy link
Contributor

@comius comius commented Sep 3, 2024

rules_java 7.10.0 work with Bazel 7 down to Bazel 6.2.0. This makes it possible to load symbols like JavaInfo from rules_java also on older Bazel versions (previous Java releases didn't have Java/common/java_info.bzl file). A single rules_java version for multiple Bazel releases should also simplify things.

Remove the flag --release, because it breaks on the combination of Bazel 6.x and rules_java 7.10.0, most likely because of incompatible changes in the turbine. The flag is not necessary, because the release is already controlled --java_release_version flag.

@comius comius marked this pull request as ready for review September 4, 2024 07:18
@jin
Copy link
Collaborator

jin commented Sep 4, 2024

Does this also retain compatibility down to Bazel 5.4.0? The tests seem to suggest it, but not your description.

What were the breakages you saw with --release?

@comius
Copy link
Contributor Author

comius commented Sep 4, 2024

Does this also retain compatibility down to Bazel 5.4.0? The tests seem to suggest it, but not your description.

As WORKSPACE file suggests, the compatibility with Bazel 5.4.0 is retained by depending on a different version of rules_java. The blocker to use a single rules_java version is the use of module_name, which wasn't introduced until Bazel 6.2.0.

What were the breakages you saw with --release?

ERROR: /workdir/private/tools/java/com/github/bazelbuild/rules_jvm_external/BUILD:1:13: Compiling Java headers private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external-hjar.jar (3 source files) [for tool] failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //private/tools/java/com/github/bazelbuild/rules_jvm_external:rules_jvm_external) ../remote_java_tools_linux/java_tools/turbine_direct_graal --output bazel-out/k8-opt-exec-2B5CBBC6/bin/private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external-hjar.jar ... (remaining 31 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set
	at java.base@21.0.2/java.util.Objects.requireNonNull(Objects.java:259)
	at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55)
	at com.google.turbine.main.Main.release(Main.java:318)
	at com.google.turbine.main.Main.bootclasspath(Main.java:304)
	at com.google.turbine.main.Main.compile(Main.java:142)
	at com.google.turbine.main.Main.compile(Main.java:133)
	at com.google.turbine.main.Main.main(Main.java:89)
	at java.base@21.0.2/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

https://buildkite.com/bazel/rules-jvm-external/builds/4484#0191b7b6-2483-4346-8d95-0976f9832b80

@@ -1,11 +1,6 @@
java_library(
name = "rules_jvm_external",
srcs = glob(["*.java"]),
javacopts = [
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use this to keep ourselves honest. Our users should still be able to target and use Java 11 in their builds. Without this, it'd be easy for us to accidentally rely on features that aren't available until later versions of the JDK.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe that produced .jar as well as source code limitations are already controlled with --java_language_version and --java_release_version.

Demo for a source version: #1231

Manual check for class version:

$ bazel build //private/tools/java/com/github/bazelbuild/rules_jvm_external
$ unzip bazel-bin/private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external.jar
$ javap -verbose com/github/bazelbuild/rules_jvm_external/ByteStreams.class | grep "major"
  major version: 55

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But the bootclasspath differs: without bazelbuild/rules_java#182, it's determined by --java_runtime_version, not --java_language_version. This makes it harder to e.g. test on JDK 21, but at the same time have compile-time assurance that the output is compatible with Java 11.

@comius
Copy link
Contributor Author

comius commented Sep 4, 2024

What were the breakages you saw with --release?

ERROR: /workdir/private/tools/java/com/github/bazelbuild/rules_jvm_external/BUILD:1:13: Compiling Java headers private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external-hjar.jar (3 source files) [for tool] failed: (Exit 1): turbine_direct_graal failed: error executing command (from target //private/tools/java/com/github/bazelbuild/rules_jvm_external:rules_jvm_external) ../remote_java_tools_linux/java_tools/turbine_direct_graal --output bazel-out/k8-opt-exec-2B5CBBC6/bin/private/tools/java/com/github/bazelbuild/rules_jvm_external/librules_jvm_external-hjar.jar ... (remaining 31 arguments skipped)
Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
java.lang.NullPointerException: attempted to use --release, but JAVA_HOME is not set
	at java.base@21.0.2/java.util.Objects.requireNonNull(Objects.java:259)
	at com.google.turbine.binder.CtSymClassBinder.bind(CtSymClassBinder.java:55)
	at com.google.turbine.main.Main.release(Main.java:318)
	at com.google.turbine.main.Main.bootclasspath(Main.java:304)
	at com.google.turbine.main.Main.compile(Main.java:142)
	at com.google.turbine.main.Main.compile(Main.java:133)
	at com.google.turbine.main.Main.main(Main.java:89)
	at java.base@21.0.2/java.lang.invoke.LambdaForm$DMH/sa346b79c.invokeStaticInit(LambdaForm$DMH)

https://buildkite.com/bazel/rules-jvm-external/builds/4484#0191b7b6-2483-4346-8d95-0976f9832b80

I did further investigation, what's going on.

  • (ok) on Bazel 6.5.0 with old rules_java ./remote_java_tools/java_tools/turbine_direct_binary_deploy.jar is used
  • (fails) on Bazel 6.5.0 with rules_java 7.10.0 ../remote_java_tools_linux/java_tools/turbine_direct_graal is used
  • (ok) on Bazel 7.0.0 with rules_java 7.10.0 ../remote_java_tools_linux/java_tools/turbine_direct_graal is used passing in -Dturbine.ctSymPath=

For --release javacopts to work, turbine_direct_graal needs Dturbine.ctSymPath=1. That was fixed for Bazel 7. in bazelbuild/bazel#20294, but not for Bazel 6.

cc @fmeum

@comius
Copy link
Contributor Author

comius commented Sep 4, 2024

Techincally I might be able to make --release work again if I move java_toolchain and java_runtime implementation into rules_java. @hvadehra wdyt?

@hvadehra
Copy link

hvadehra commented Sep 4, 2024

As it happens, I'm planning to extract both of those rules out very shortly.

java_runtime will likely be fine with already released Bazel versions, but java_toolchain will need some cherrypicks to old Bazel versions.

@shs96c shs96c mentioned this pull request Sep 4, 2024
@shs96c
Copy link
Collaborator

shs96c commented Sep 5, 2024

Alright. I don't want to block this, because it's a good upgrade to have. I'm going to assume that rules_jvm_external using a hermetic java version will be enough to avoid us calling more recent JDK APIs.

I would note that loading things like JavaInfo from rules_java will mean we'll need to drop Bazel 5 support, but I think we'll likely do that once Bazel 8 ships (that is, we retain our current LTS-2 support policy), so we shouldn't start using that until then.

Copy link
Collaborator

@shs96c shs96c left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks adequate to me. I'm a little uneasy, but let's see what happens once this is landed.

javacopts = [
"--release",
"11",
"-Xlint:-options",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This flag was added in #950, so this looks safe to remove.

@comius comius merged commit 5faa640 into bazel-contrib:master Sep 5, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants