load_arbitrary_tool uses tool_suburl to look up sha256 #1695
Conversation
In load_arbitrary_tool, use the suburl (which contains the a
subdirectory containing the date, if the version is beta or nightly) to
look up the sha256 to to download a specific archive. This allows Bazel
to cache the downloaded archive by hash, and supports the correct
behavior for dated nightly builds, as provided in the FILE_KEY_TO_SHA
dict in //rust:known_shas.bzl.
The most recent nightly entry is given in FILE_KEY_TO_SHA as:
```
"2022-11-02/cargo-nightly-x86_64-unknown-linux-gnu.tar.gz": "d32e0a9f78ece567627b9b572912b000c53099c0dd9c9f5cea54848de02c6486",
```
This change fixes downloaded archive caching when using the nightly
toolchains, including the default nightly toolchain.
There was a problem hiding this comment.
Thanks for opening this PR! Could you show a before and after of what values get passed to ctx.download_and_extract?
Also, would you be willing to add a test for this? My thoughts are the url and sha256 value calculations in load_arbitrary_tool should be moved into a helper function so it can be called in a starlark unit test without downloading things. Does that make sense?
Move the sha256 code out of load_arbitrary_tool() into a new function, lookup_tool_sha256(). Implement a new unit test to confirm that the sha256 lookup code behaves as expected, and returns the expected sha256 hash from the static list of known tool hashes.
Adding a debug print immediately before calling Without my change applied, for the rustc download the sha256 is empty because it's trying to look up With my change here applied, at the same point I see a valid hash for the same file: The current stable release is unchanged; both runs give the same hash:
I've moved the sha256 code out of |
In load_arbitrary_tool, use the suburl (which contains the a subdirectory containing the date, if the version is beta or nightly) to look up the sha256 to to download a specific archive. This allows Bazel to cache the downloaded archive by hash, and supports the correct behavior for dated nightly builds, as provided in the FILE_KEY_TO_SHA dict in //rust:known_shas.bzl.
The most recent nightly entry is given in FILE_KEY_TO_SHA as:
This change fixes downloaded archive caching when using the nightly toolchains, including the default nightly toolchain.