Build with annotation processor fails with higher tool Java language version

[fmeum]

Description of the bug:

When a Java target uses a java_plugin that runs an annotation processor and the --tool_java_language_version is higher than the --java_language_version, the build fails even if --java_runtime_version specifies a JDK version that is at least as high as --tool_java_language_version.

Given that this behavior is already triggered by depending on @bazel_tools//tools/java/runfiles, we experienced this as a regression when migrating to Bazel 6.

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

With this WORKSPACE:

# WORKSPACE

# BUILD
java_binary(
    name = "Main",
    srcs = ["Main.java"],
    main_class = "Main",
    deps = ["@bazel_tools//tools/java/runfiles"],
)

# Main.java
public class Main {
    public static void main(String[] args) {}
}

1. bazel build //:Main passes
2. bazel build //:Main --tool_java_language_version=17 --tool_java_runtime_version=remotejdk_17 fails with Caused by: java.lang.UnsupportedClassVersionError: com/google/devtools/build/runfiles/AutoBazelRepositoryProcessor has been compiled by a more recent version of the Java Runtime (class file version 61.0), this version of the Java Runtime only recognizes class file versions up to 55.0
3. bazel build //:Main --tool_java_language_version=17 --tool_java_runtime_version=remotejdk_17 --java_runtime_version=remotejdk_17 fails with the same error
4. bazel build //:Main --tool_java_language_version=17 --tool_java_runtime_version=remotejdk_17 --java_runtime_version=remotejdk_17 --java_language_version=11 fails with the same error
5. bazel build //:Main --tool_java_language_version=17 --tool_java_runtime_version=remotejdk_17 --java_runtime_version=remotejdk_17 --java_language_version=17 passes

I would argue that the failure in 2. is already unexpected: If the Java language levels in tools can't be higher than that for the target configuration, then what is the purpose of the flag? I always assumed that it exists to allow tooling to use modern Java features while limiting the rest of the codebase to something more compatible (e.g. Java 11).

The failures in 3. and 4. are definitely unexpected: Based on https://bazel.build/docs/bazel-and-java#hermetic-testing, I would have expected JDK 17 to be used for compilation with a -target value of 11 (3.) or 17 (4.). In both cases, the annotation processor compiled for a Java 17 runtime should be runnable. Instead, using --verbose_failures, it looks like remotejdk_11 is still used for compilation.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

6.0.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

[fmeum]

@comius @cushon Could you take a look? It may end up just being a doc issue, but it does seem likely to break users of the Java runfiles library updating to Bazel 6.

[cushon]

I think the issue might be that the JDK version that's used to run JavaBuilder isn't controlled by --tool_java_runtime= version, it's configured with the java_toolchain.java_runtime attribute.

Part of the motivation for that is that it lets us ensure JavaBuilder is running on a relative recent JDK version, even if a build wants to use an older --tool_java_runtime=. But it also makes it harder to use a newer --tool_java_runtime=.

Internally we're using the latest available JDK as the java_toolchain.java_runtime. One option is to just update the default to a newer version, but the latest javac versions have dropped support for -source 7 -target 7, so that's probably an aggressive default for Bazel.

Another option here might just be to document how to configure a custom java_toolchain that sets a matching java_runtime.

[fmeum]

What I find surprising is that setting --java_runtime_version=remotejdk_17 --tool_java_runtime_version=remotejdk_17 doesn't fix this, only --java_language_version=17 does. Is that expected? I would have assume that --java_language_version only affects the -source and -target values, but not the runtime used to run JavaBuilder.

Is there a way to set up the toolchain so that a recent JDK is used for compilation (both target and tool) but target compilation outputs remain compatible with old JDKs and are also tested against these old JDKs?

[cushon]

Is there a way to set up the toolchain so that a recent JDK is used for compilation (both target and tool) but target compilation outputs remain compatible with old JDKs and are also tested against these old JDKs?

I remembered that this is eventually going to be improved by 75e0f7d making it into a java_tools release, which will update the default java_toolchain.java_runtime to JDK 17

The issue you encountered in this bug could still come up for --tool_java_language_versions newer than 17, though. It would be nice if a bleeding-edge --tool_java_language_version just worked without having to define a custom java_toolchain.

I would have assume that --java_language_version only affects the -source and -target values, but not the runtime used to run JavaBuilder.

I agree this is having surprising results here, it's the result of all three of those being configured with java_toolchain attributes. The default java_toolchain for the target platform is still @bazel_tools//tools/jdk:toolchain_java8, which with Bazel 6.0.0 is setting a JDK 11 java_runtime:

$ bazel shutdown; bazel build --toolchain_resolution_debug=//tools/jdk:toolchain_type //:Main --tool_java_language_version=17 --tool_java_runtime_version=remotejdk_17 |& grep 'Selected toolchain'
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @bazel_tools//tools/jdk:toolchain_java8
INFO: ToolchainResolution:   Type @bazel_tools//tools/jdk:toolchain_type: target platform @local_config_platform//:host: execution @local_config_platform//:host: Selected toolchain @bazel_tools//tools/jdk:toolchain_jdk_17

$ bazel query --output build @bazel_tools//tools/jdk:toolchain_java8
...
  java_runtime = "@bazel_tools//tools/jdk:remote_jdk11",

[fmeum]

Thanks for the explanations. Does that mean that the language level of annotation processors is limited by the Java runtime used in the target configuration? If that's the case, would it help if processors were compiled for the target rather than the exec configuration?

[cushon]

The language level of annotations processors is limited by the java_toolchain.java_runtime.

Ideally we'd want to ensure that the provided configurations satisfy the following constraints.

--java_runtime_version=	>=	--java_language_level=
--tool_java_runtime_version=	>=	--tool_java_language_level=
java_toolchain.java_runtime	>=	--tool_java_language_level=
java_toolchain.java_runtime	>=	--java_language_level=

i.e. to build code at the Java N language level, you need a JavaBuilder running on JDK >= N to compile it, and a JDK >= N to execute it

[github-actions]

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 30 days. It will be closed in the next 7 days unless any other activity occurs or one of the following labels is added: "not stale", "awaiting-bazeler".

[davido]

Possibly related: bazel-contrib/rules_jvm_external#895

[cushon]

One more note on that is that another way it satisfy the java_toolchain.java_runtime >= constraint is to keep that version upgraded to the latest available JDK. We've been doing that at Google, but it might be too aggressive for Bazel when there are breaking changes like JDK-8173605

[comius]

I also ran into this bug building bazel project on Mac. Can we bump the priority?

[fmeum]

Warnings have been added in Bazel 7 or anything using a recent rules_java release.