rustbuild: rustc_codegen_llvm is recompiled when any rustc crate is changed #54008
Labels
I-compiletime
Issue: Problems and improvements with respect to compile times.
T-bootstrap
Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Comments
arielb1
added
I-compiletime
Mark-Simulacrum
added a commit
to Mark-Simulacrum/rust
that referenced
this issue
Aug 14, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Centril
added a commit
to Centril/rust
that referenced
this issue
Aug 15, 2019
…xcrichton Utilize -Zbinary-dep-depinfo in rustbuild The last commit moves us over to using binary-dep-depinfo while the first two permit us to bootstrap from what will become future beta, to be released in the next week (it's the `cfg(bootstrap)` processing). We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes rust-lang#54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rust-lang#54008, fixes rust-lang#50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes rust-lang#53284, and fixes rust-lang#52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes rust-lang#49979 (but no reproduction steps in that issue). Fixes rust-lang#59105. cc rust-lang#63012
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
STR
This also occurs with
keep-stage0etc.Expected result
Changing
librustc_typeckrequires only the reverse dependencies oflibrustc_typeckto be recompiled.Actual result
librustc_codegen_llvmis also recompiled. As you can see, this takes twice as much as compiling the reverse dependencies oflibrustc_typeckalone, leading to compilation being 3x slower.The text was updated successfully, but these errors were encountered: