Continual recompliation of blake3 #324
Comments
|
I'm also having this issue. |
|
I'd be curious if removing those lines fixes the issue for you. That wouldn't tell us the whole story, but it would be a big part. Also, can you reproduce this issue in any open source project that I could also build for myself. It would help to see it happen on my own machine. |
|
Yeah, it's almost certainly to do with those rerun directives. I was messing around with it earlier and I did notice that it only ever happens in my VS Code terminal, which is quite weird. There are a lot of moving parts w.r.t. the terminal in my particular project because I'm using a Nix shell w/ Direnv, so it's going to be a bit difficult to track down the issue, but I'll set aside some time tomorrow to work on a minimal reproduction. |
|
I have similar problem on Voidlinux. But with small difference:
|
|
I have the same problem on Windows when using Bevy game engine. Any change triggers blake3 recompilation - is this expected? Edit: it seems like clippy may be triggering that. It happens on code change when I use RustRover, because I configured it to run clippy in the background. I'm still not sure if this is expected or not. |
|
Folks who can repro this, can you help me clarify: Are the values of these env vars actually changing every time? If so what sort of values are they taking? (If not, could this be a |
|
Just a note that I no longer work for this company and don't have access to that laptop or code base. Hopefully Max will be of more help to you. |
|
On windows with bevy this is happening for me because between clippy and my manual cargo run, the PATH variable is different, probably because vscode runs clippy in either a cmd or powershell environment, while i manually trigger it in a nushell terminal. Wondering why it's even triggering a rebuild when PATH of all things change. |
|
@AviiNL do you have time to bisect the rerun lines and figure out which one(s) we'd need to delete to fix your setup? I'm open to ideas about whether these reruns are causing more harm than good. On the one hand I don't do as much benchmarking as I used to, and it probably wouldn't matter to me if I deleted these. On the other hand, if your compilation environment really is changing, then using stale artifacts might cause weird build issues, and it can be awfully confusing to debug that stuff. |
|
(Disclaimer: while I was affected by this issue, I've fixed my environment since so I'm not anymore. What follows is my 2cts on this problem) @oconnor663, compilation is much slower because recompiling Blake3 has a cascading effect and lots of other crates get recompiled as well. And to add insult to injuries, in my case, the "path changed" was only duplicating an existing path. The path to As for the PATH changing being a possible problem, I would think it's no different from a new version of the tools in the same path. Or changes of other stuff in the environment not related to paths. |
|
Yeah pretty much what @Nahor wrote is my experience as well, entries in PATH begin duplicated by the IDE. I've put some ducttape on it in the form of extra environment arguments in settings.json for rust analyzers usage of cargo. I did a quick check in build.rs, but there doesn't appear to be anything explicitly checking for the PATH variable, so no idea why it's triggering on it. |
Yeah exactly. I agree with @Nahor that rebuilding on any
Those last five are custom settings that are specific to this project, so I don't expect any other tools to ever set them. However, if you do happen to set them yourself in your regular build environment, and your clippy/miri/etc environment doesn't set them, then something like that could be what's causing the build thrashing that you guys are seeing. Do you set any of those variables anywhere in your scripts? |
|
It was definitely the PATH for me. That's what cargo was complaining about, removing it from the IDE settings fixed my issue, and when I manually update the PATH on the command line it comes back, even between runs from the command line: $ cargo build
Finished `dev` profile [optimized + debuginfo] target(s) in 0.35s
$ export PATH=".:$PATH"
$ cargo build
Compiling blake3 v1.5.1
[...]
Finished `dev` profile [optimized + debuginfo] target(s) in 1m 20s
$ cargo build
Finished `dev` profile [optimized + debuginfo] target(s) in 0.29s
$ export PATH=".:$PATH"
$ cargo build
Compiling blake3 v1.5.1
[...]
Finished `dev` profile [optimized + debuginfo] target(s) in 1m 19sIt might be related to |
|
Yeah I don't know what to tell you, when running with and after forcing PATH in settings.json so rust analyzer uses the same value as my terminal environment, it works as it should {
"rust-analyzer.server.extraEnv": {
"PATH": "..."
}
} |
|
target/debug/.fingerprint/blake3-200e32cadfcbb6ed/run-build-script-build-script-build.json: |
|
I believe I have a fix for this with #413. For anyone else having this issue, can you try my bug fix to see if it works? Simply add this to your [patch.crates-io]
# For testing fix for blake3 env change rebuild thrashing issue
blake3 = { git = "https://github.com/hintron/BLAKE3", branch = "hintron/fix-blake-rebuild-issue"}On Windows, there are still some env changes that will trigger a rebuild, even after my fix:
But the important thing is that changes to |
|
Good find @hintron! It looks like https://github.com/rust-lang/cc-rs/releases/tag/1.0.74 (released a couple years after this crate originally took its |
|
@oconnor663 What additional env vars do you want to rebuild on? Probably |
Changes since 1.5.1: - `build.rs` sets `cc::Build::emit_rerun_if_env_changed(false)` to prevent some unnecessary rebuilds, particularly when the `PATH` changes on Windows. See #324. - Serializing a `Hash` produces a bytestring instead of an array in formats that support bytestrings (like CBOR). Deserialization is backwards-compatible with the array format. - Cleanup and edge case fixes in the C and CMake builds.
Changes since 1.5.1: - `build.rs` sets `cc::Build::emit_rerun_if_env_changed(false)` to prevent some unnecessary rebuilds, particularly when the `PATH` changes on Windows. See BLAKE3-team#324. - Serializing a `Hash` produces a bytestring instead of an array in formats that support bytestrings (like CBOR). Deserialization is backwards-compatible with the array format. - Cleanup and edge case fixes in the C and CMake builds.
|
rust-lang/cc-rs#1215 ( |
Blake3 is in my transitive dependency chain. When I edit my code, it recompiles every time. It seems to believe that the environment variable is changing.. but I'm just editing a file, nothing more.
Most seemingly relevant line below.
Thoughts?
The text was updated successfully, but these errors were encountered: