Sync java_tools, remote JDKs, and java toolchains in rules_java with Bazel HEAD #110
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
meteorcloudy
force-pushed
the
update
branch
3 times, most recently
from
ca20daf to
fe9bd07
Compare
|
FYI @keertk |
|
bazelbuild/bazel#18423 is now blocking on this PR |
hvadehra
approved these changes
May 23, 2023
…Bazel HEAD Update rules_java to match Bazel at HEAD Sync toolchain changes BUILD file Fix local_java_repository More fixes Export WORKSPACE file Expose //toolchains:srcs fix determine_java_home Fix remote JDK definitions Update remoteJDK 17 versions to match Bazel sources Make buildifier happy Fix extensions.bzl More fixes Fix Bzlmod build More fixes
copybara-service bot
pushed a commit
to bazelbuild/bazel
that referenced
this pull request
May 31, 2023
Context: #18373 Currently the definitions of java tools, remote JDKs, and Java toolchains are duplicated in both Bazel sources and rules_java, and they go out of sync quite often. By default, the default toolchains shipped with the Bazel binary uses the one in Bazel sources except when Bzlmod is enabled. bazelbuild/rules_java#110 syncs java toolchain related files from Bazel sources to rules_java and this PR removes most of the definitions in Bazel source and uses rules_java as the source of truth. Significant changes in this PR: - Removed all java tools definitions from distdir_deps.bzl - Removed all remote JDK definitions from distdir_deps.bzl except the ones for to be embedded within Bazel binary. - Changed `jdk.WORKSPACE.tmpl` to load java tools and JDKs from rules_java as WORKSPACE suffix. - Removed the `jdk_http_archives` hack for some shell tests. That dates back to #8361, I don't think it's needed since we have the WORKSPACE suffix by default. - When building Bazel with Bazel, we were using the java toolchains shipped in `@bazel_tools` of the Bazel binary, now we use the java toolchains from the `rules_java` defined in distdir_deps.bzl. This enables us to build Bazel itself with the latest java toolchain, therefore can detect issues earlier. - Removed `_for_testing` suffixes for the shared repo hack for Bazel CI since the java tools and remote JDKs versions we load to build Bazel are the same as the ones shipped in the Bazel binary we built (via `jdk.WORKSPACE.tmpl`) and used in the integration tests. They all come from the rules_java version in distdir_deps.bzl. - Removed the `test_WORKSPACE_files` hack for `workspace_resolved_test.sh`. - Mocked rules_java in some integration tests to avoid needing to download it or its transitive dependencies. Breaking changes: - Loading `.bzl` files under `@bazel_tools//tools/jdk` in WORKSPACE now requires you to define `rules_java` in advance, the `rules_java` definition in WORKSPACE suffix cannot guarantee `@rules_java` exist because it's appended after the content of the user WORKSPACE file. Closes #18423. RELNOTES[INC]: Loading `.bzl` files under `@bazel_tools//tools/jdk` in WORKSPACE now requires `rules_java` to be defined in advance. PiperOrigin-RevId: 536682532 Change-Id: Ia7475bfe134729ab1592b0ef0a31d28eb839b5ac
|
OK, this is causing many downstream breakages: https://buildkite.com/bazel/bazel-at-head-plus-downstream/builds/3066#_ The reason is that the default rules_java is defined in WORKSPACE suffix, which will be overridden by the rules_java version introduced in the user's WORKSPACE file. If that rules_java version is too old, loading JDK repos and toolchains will fail. I think we'll need to figure out a way to make sure the latest version of rules_java is used in WORKSPACE. And this is not a problem with Bzlmod enabled because we use MVS to resolve Bazel module dependencies. |
copybara-service bot
pushed a commit
to bazelbuild/bazel
that referenced
this pull request
Jun 7, 2023
… try) Rolling forward 975866a (which was rollbacked at d51c75f) with fixes. - Introduce a `rules_java_builtin` repo in WORKSPACE prefix to avoid conflict with user defined rules_java. - `@bazel_tools//tools/jdk/*.bzl` loads from `rules_java_builtin` through repo-mappings. - `@local_jdk` was overridden in `jdk.WORKSPACE` to add repo_mapping for `rules_java`. - `jdk.WORKSPACE` explicitly loads from `rules_java_builtin` for JDK definitions and java toolchain definitions. - Allow using `__SKIP_WORKSPACE_PREFIX__` and `__SKIP_WORKSPACE_SUFFIX__` in WORKSPACE comment. - Fixed many tests by adjusting the WORKSPACE file content. - Re-export more symbols from `@bazel_tools` to be backward-compatible. Fixes #18551 Related: - #18373 - bazelbuild/rules_java#110 - #18423 Closes #18558. PiperOrigin-RevId: 538483417 Change-Id: I5223eec2c4b10131fc8c5b342237385ff2f56413
amishra-u
pushed a commit
to amishra-u/bazel
that referenced
this pull request
Jun 7, 2023
… try) Rolling forward bazelbuild@975866a (which was rollbacked at bazelbuild@d51c75f) with fixes. - Introduce a `rules_java_builtin` repo in WORKSPACE prefix to avoid conflict with user defined rules_java. - `@bazel_tools//tools/jdk/*.bzl` loads from `rules_java_builtin` through repo-mappings. - `@local_jdk` was overridden in `jdk.WORKSPACE` to add repo_mapping for `rules_java`. - `jdk.WORKSPACE` explicitly loads from `rules_java_builtin` for JDK definitions and java toolchain definitions. - Allow using `__SKIP_WORKSPACE_PREFIX__` and `__SKIP_WORKSPACE_SUFFIX__` in WORKSPACE comment. - Fixed many tests by adjusting the WORKSPACE file content. - Re-export more symbols from `@bazel_tools` to be backward-compatible. Fixes bazelbuild#18551 Related: - bazelbuild#18373 - bazelbuild/rules_java#110 - bazelbuild#18423 Closes bazelbuild#18558. PiperOrigin-RevId: 538483417 Change-Id: I5223eec2c4b10131fc8c5b342237385ff2f56413
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Context: bazelbuild/bazel#18373
Preparing for using rules_java as the source of truth for all java toolchain related definitions (bazelbuild/bazel#18423)