Utilize -Zbinary-dep-depinfo in rustbuild #63470

Mark-Simulacrum · 2019-08-11T19:05:07Z

We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles.

Both of these are fixable in the longer term but this existing patch gives us the following benefits:

We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes x.py in incremental mode still rebuilds stage0-rustc if stage0-std changed. #54712.
Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes rustbuild: rustc_codegen_llvm is recompiled when any rustc crate is changed #54008, fixes Rustbuild: changing and testing libcore results in "found possibly newer version of crate std which rand depends on" #50481.
We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes Changing rustc_codegen_llvm only will not rerun the tests #53284, and fixes Tests are not rebuilt when codegen is rebuilt #52719.
Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes "found possibly newer version of crate std which cc depends on" while building rustc #49979 (but no reproduction steps in that issue). Fixes Incremental compilation error building rustc (-i --keep-stage 0) #59105.

cc #63012

Mark-Simulacrum · 2019-08-11T19:07:12Z

To be clear this is not intended to land as-is since it won't actually work as-is since the rustc needs to be overridden -- the currently nightly can't bootstrap successfully I believe due to MaybeUninit changes -- I'm using 2b78e10 as the base commit via rustup-toolchain-install-master locally and we shouldn't couple the depdep changes to cfg(bootstrap).

alexcrichton

That's an impressive list of fixed issues!

This may also want to apply https://gist.github.com/alexcrichton/e0840447982f93cb22bbd98878d2c205 to get colors back in error messages and fix the progress bar with pipelining

src/bootstrap/bin/rustc.rs

src/bootstrap/compile.rs

rust-highfive · 2019-08-12T20:08:42Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-12T18:58:37.8552072Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-12T18:58:37.8730136Z ##[command]git config gc.auto 0
2019-08-12T18:58:37.8807199Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-12T18:58:37.8857854Z ##[command]git config --get-all http.proxy
2019-08-12T18:58:37.8988671Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63470/merge:refs/remotes/pull/63470/merge
---
2019-08-12T18:59:13.5557268Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-12T18:59:13.5557320Z 
2019-08-12T18:59:13.5557745Z   git checkout -b <new-branch-name>
2019-08-12T18:59:13.5557775Z 
2019-08-12T18:59:13.5557825Z HEAD is now at d29203c7d Merge 0be63bd405b1a78f07e528b60972a0e05ee9d1f4 into 60960a260f7b5c695fd0717311d72ce62dd4eb43
2019-08-12T18:59:13.5718160Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-12T18:59:13.5721160Z ==============================================================================
2019-08-12T18:59:13.5721206Z Task         : Bash
2019-08-12T18:59:13.5721240Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-12T19:57:52.7030279Z .................................................................................................... 1300/8875
2019-08-12T19:57:59.2473769Z .................................................................................................... 1400/8875
2019-08-12T19:58:05.4539230Z .................................................................................................... 1500/8875
2019-08-12T19:58:16.0749222Z ....................................................................................i............... 1600/8875
2019-08-12T19:58:23.5643833Z i................................................................................................... 1700/8875
2019-08-12T19:58:30.0542706Z ...........................................................................iiiii.................... 1800/8875
2019-08-12T19:58:51.5059152Z .................................................................................................... 2000/8875
2019-08-12T19:58:53.8218187Z .................................................................................................... 2100/8875
2019-08-12T19:58:56.3277882Z .................................................................................................... 2200/8875
2019-08-12T19:59:03.6203121Z .................................................................................................... 2300/8875
---
2019-08-12T20:02:49.0796314Z ......................................................F............................................. 5300/8875
2019-08-12T20:02:56.0246796Z ........i........................................................................................... 5400/8875
2019-08-12T20:03:01.1637736Z .................................................................................................... 5500/8875
2019-08-12T20:03:12.8930664Z .................................................................................................... 5600/8875
2019-08-12T20:03:26.6177343Z ...ii...i..ii...........i........................................................................... 5700/8875
2019-08-12T20:03:41.1958596Z .................................................................................................... 5900/8875
2019-08-12T20:03:45.7230242Z .................................................................................................... 6000/8875
2019-08-12T20:03:45.7230242Z .................................................................................................... 6000/8875
2019-08-12T20:03:59.3523629Z ....i..ii........................................................................................... 6100/8875
2019-08-12T20:04:17.3355668Z ...............................................i.................................................... 6300/8875
2019-08-12T20:04:19.3514211Z .................................................................................................... 6400/8875
2019-08-12T20:04:21.6889439Z ...................i................................................................................ 6500/8875
2019-08-12T20:04:25.9978187Z .................................................................................................... 6600/8875
---
2019-08-12T20:08:16.9345838Z failures:
2019-08-12T20:08:16.9367396Z 
2019-08-12T20:08:16.9368023Z ---- [ui] ui/macros/builtin-std-paths.rs stdout ----
2019-08-12T20:08:16.9368691Z 
2019-08-12T20:08:16.9369270Z error: test compilation failed although it shouldn't!
2019-08-12T20:08:16.9369785Z status: exit code: 1
2019-08-12T20:08:16.9370864Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/macros/builtin-std-paths.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "--emit" "metadata" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/macros/builtin-std-paths" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/macros/builtin-std-paths/auxiliary" "-A" "unused"
2019-08-12T20:08:16.9372149Z ------------------------------------------
2019-08-12T20:08:16.9372400Z 
2019-08-12T20:08:16.9372779Z ------------------------------------------
2019-08-12T20:08:16.9373001Z stderr:
2019-08-12T20:08:16.9373001Z stderr:
2019-08-12T20:08:16.9373718Z ------------------------------------------
2019-08-12T20:08:16.9373985Z error[E0433]: failed to resolve: could not find `Hash` in `hash`
2019-08-12T20:08:16.9374669Z    |
2019-08-12T20:08:16.9374834Z LL |     core::hash::Hash,
2019-08-12T20:08:16.9374834Z LL |     core::hash::Hash,
2019-08-12T20:08:16.9375019Z    |                 ^^^^ could not find `Hash` in `hash`
2019-08-12T20:08:16.9375164Z 
2019-08-12T20:08:16.9375328Z error[E0433]: failed to resolve: could not find `Hash` in `hash`
2019-08-12T20:08:16.9375929Z    |
2019-08-12T20:08:16.9376501Z LL |     std::hash::Hash,
2019-08-12T20:08:16.9376501Z LL |     std::hash::Hash,
2019-08-12T20:08:16.9376681Z    |                ^^^^ could not find `Hash` in `hash`
2019-08-12T20:08:16.9377175Z error: aborting due to 2 previous errors
2019-08-12T20:08:16.9377299Z 
2019-08-12T20:08:16.9377716Z For more information about this error, try `rustc --explain E0433`.
2019-08-12T20:08:16.9377849Z 
---
2019-08-12T20:08:16.9400139Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-08-12T20:08:16.9400428Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-12T20:08:16.9416666Z 
2019-08-12T20:08:16.9418035Z 
2019-08-12T20:08:16.9420739Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-12T20:08:16.9421235Z 
2019-08-12T20:08:16.9421287Z 
2019-08-12T20:08:16.9430791Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-12T20:08:16.9430932Z Build completed unsuccessfully in 1:02:46
2019-08-12T20:08:16.9430932Z Build completed unsuccessfully in 1:02:46
2019-08-12T20:08:18.1170799Z ##[error]Bash exited with code '1'.
2019-08-12T20:08:18.1212444Z ##[section]Starting: Checkout
2019-08-12T20:08:18.1214185Z ==============================================================================
2019-08-12T20:08:18.1214230Z Task         : Get sources
2019-08-12T20:08:18.1214284Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Mark-Simulacrum · 2019-08-12T22:09:30Z

Just force pushed what should hopefully be a fix for the cfg(bootstrap) bit; with stage0.txt modified to the beta scheduled to be released Tuesday we can likely land this PR in one go to avoid wasting two cycles.

rust-highfive · 2019-08-13T00:01:26Z

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-12T23:53:16.4280624Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-12T23:53:16.4479176Z ##[command]git config gc.auto 0
2019-08-12T23:53:16.4540035Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-12T23:53:16.4590193Z ##[command]git config --get-all http.proxy
2019-08-12T23:53:16.4723820Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63470/merge:refs/remotes/pull/63470/merge
---
2019-08-12T23:53:53.4139249Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-12T23:53:53.4139296Z 
2019-08-12T23:53:53.4139496Z   git checkout -b <new-branch-name>
2019-08-12T23:53:53.4139524Z 
2019-08-12T23:53:53.4139569Z HEAD is now at 1a9aa9925 Merge beab7fb5aff44c0d2f79407c3efd489fec64a802 into 60960a260f7b5c695fd0717311d72ce62dd4eb43
2019-08-12T23:53:53.4294082Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-12T23:53:53.4296936Z ==============================================================================
2019-08-12T23:53:53.4296995Z Task         : Bash
2019-08-12T23:53:53.4297044Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-12T23:59:24.4589361Z     Checking backtrace v0.3.34
2019-08-12T23:59:26.4508250Z     Checking rustc-std-workspace-alloc v1.0.0 (/checkout/src/tools/rustc-std-workspace-alloc)
2019-08-12T23:59:26.4512106Z     Checking panic_unwind v0.0.0 (/checkout/src/libpanic_unwind)
2019-08-12T23:59:26.6072567Z     Checking hashbrown v0.4.0
2019-08-12T23:59:28.4944861Z error: use of item 'core::try' that will be deprecated in future version 1.39.0: use the `?` operator instead
2019-08-12T23:59:28.4946216Z     |
2019-08-12T23:59:28.4946216Z     |
2019-08-12T23:59:28.4946854Z 524 |     r#try,
2019-08-12T23:59:28.4947732Z     |
2019-08-12T23:59:28.4948199Z     = note: `-D deprecated-in-future` implied by `-D warnings`
2019-08-12T23:59:28.4948367Z 
2019-08-12T23:59:30.3433869Z error: aborting due to previous error
2019-08-12T23:59:30.3433869Z error: aborting due to previous error
2019-08-12T23:59:30.3434560Z 
2019-08-12T23:59:30.3771646Z error: Could not compile `std`.
2019-08-12T23:59:30.3771725Z 
2019-08-12T23:59:30.3772235Z To learn more, run the command again with --verbose.
2019-08-12T23:59:30.3794089Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0/bin/cargo" "check" "--target" "x86_64-unknown-linux-gnu" "-Zbinary-dep-depinfo" "-j" "2" "--release" "--color" "always" "--features" "panic-unwind backtrace compiler-builtins-c" "--manifest-path" "/checkout/src/libstd/Cargo.toml" "--message-format" "json-render-diagnostics"
2019-08-12T23:59:30.3800655Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap check
2019-08-12T23:59:30.3800939Z Build completed unsuccessfully in 0:02:52
2019-08-12T23:59:30.3800939Z Build completed unsuccessfully in 0:02:52
2019-08-12T23:59:36.9017102Z ##[error]Bash exited with code '1'.
2019-08-12T23:59:36.9058225Z ##[section]Starting: Checkout
2019-08-12T23:59:36.9060323Z ==============================================================================
2019-08-12T23:59:36.9060402Z Task         : Get sources
2019-08-12T23:59:36.9060452Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-13T01:57:49Z

The job x86_64-gnu-llvm-6.0 of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-13T00:49:15.8870884Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-13T00:49:15.9049632Z ##[command]git config gc.auto 0
2019-08-13T00:49:15.9122297Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-13T00:49:15.9171555Z ##[command]git config --get-all http.proxy
2019-08-13T00:49:15.9308213Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63470/merge:refs/remotes/pull/63470/merge
---
2019-08-13T00:49:51.4556209Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-13T00:49:51.4556234Z 
2019-08-13T00:49:51.4556405Z   git checkout -b <new-branch-name>
2019-08-13T00:49:51.4556446Z 
2019-08-13T00:49:51.4556486Z HEAD is now at 0b563536d Merge cc0bdabef81d6a5f66caeefb3744145f22f7fa7f into 60960a260f7b5c695fd0717311d72ce62dd4eb43
2019-08-13T00:49:51.4729816Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-13T00:49:51.4733004Z ==============================================================================
2019-08-13T00:49:51.4733071Z Task         : Bash
2019-08-13T00:49:51.4733109Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-13T01:46:59.6090570Z .................................................................................................... 1300/8875
2019-08-13T01:47:06.2606012Z .................................................................................................... 1400/8875
2019-08-13T01:47:11.8981421Z .................................................................................................... 1500/8875
2019-08-13T01:47:21.9724751Z ....................................................................................i............... 1600/8875
2019-08-13T01:47:29.3827132Z i................................................................................................... 1700/8875
2019-08-13T01:47:35.8508370Z ...........................................................................iiiii.................... 1800/8875
2019-08-13T01:47:57.0335565Z .................................................................................................... 2000/8875
2019-08-13T01:47:59.4164377Z .................................................................................................... 2100/8875
2019-08-13T01:48:02.0424979Z .................................................................................................... 2200/8875
2019-08-13T01:48:09.4670696Z .................................................................................................... 2300/8875
---
2019-08-13T01:51:58.4298800Z .................................................................................................... 5300/8875
2019-08-13T01:52:05.5776011Z ........i........................................................................................... 5400/8875
2019-08-13T01:52:10.8147842Z .................................................................................................... 5500/8875
2019-08-13T01:52:22.8010324Z .................................................................................................... 5600/8875
2019-08-13T01:52:36.4495964Z ...ii...i..ii...........i........................................................................... 5700/8875
2019-08-13T01:52:51.0155298Z .................................................................................................... 5900/8875
2019-08-13T01:52:55.6030007Z .................................................................................................... 6000/8875
2019-08-13T01:52:55.6030007Z .................................................................................................... 6000/8875
2019-08-13T01:53:09.1527375Z ....i..ii........................................................................................... 6100/8875
2019-08-13T01:53:27.2263613Z ...............................................i.................................................... 6300/8875
2019-08-13T01:53:29.3326601Z .................................................................................................... 6400/8875
2019-08-13T01:53:31.7576693Z ...................i................................................................................ 6500/8875
2019-08-13T01:53:36.1665260Z .................................................................................................... 6600/8875
---
2019-08-13T01:57:25.5834481Z 
2019-08-13T01:57:25.5835094Z ---- [ui] ui/lint/lint-qualification.rs stdout ----
2019-08-13T01:57:25.5835363Z diff of stderr:
2019-08-13T01:57:25.5835483Z 
2019-08-13T01:57:25.5835812Z + warning: use of deprecated item 'try': use the `?` operator instead
2019-08-13T01:57:25.5836164Z +   --> $DIR/lint-qualification.rs:13:36
2019-08-13T01:57:25.5836316Z +    |
2019-08-13T01:57:25.5836652Z + LL |     let _ = || -> Result<(), ()> { try!(Ok(())); Ok(()) }; // issue #37345
2019-08-13T01:57:25.5836926Z +    |
2019-08-13T01:57:25.5837059Z +    = note: `#[warn(deprecated)]` on by default
2019-08-13T01:57:25.5837178Z + 
2019-08-13T01:57:25.5837289Z 1 error: unnecessary qualification
2019-08-13T01:57:25.5837289Z 1 error: unnecessary qualification
2019-08-13T01:57:25.5837592Z 2   --> $DIR/lint-qualification.rs:10:5
2019-08-13T01:57:25.5837755Z 3    |
2019-08-13T01:57:25.5837856Z 
2019-08-13T01:57:25.5837970Z 
2019-08-13T01:57:25.5838092Z The actual stderr differed from the expected stderr.
2019-08-13T01:57:25.5838454Z Actual stderr saved to /checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/lint/lint-qualification/lint-qualification.stderr
2019-08-13T01:57:25.5839779Z To update references, rerun the tests and pass the `--bless` flag
2019-08-13T01:57:25.5840264Z To only update this specific test, also pass `--test-args lint/lint-qualification.rs`
2019-08-13T01:57:25.5840600Z error: 1 errors occurred comparing output.
2019-08-13T01:57:25.5840748Z status: exit code: 1
2019-08-13T01:57:25.5840748Z status: exit code: 1
2019-08-13T01:57:25.5841826Z command: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "/checkout/src/test/ui/lint/lint-qualification.rs" "-Zthreads=1" "--target=x86_64-unknown-linux-gnu" "--error-format" "json" "-Zui-testing" "-C" "prefer-dynamic" "--out-dir" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/lint/lint-qualification" "-Crpath" "-O" "-Cdebuginfo=0" "-Zunstable-options" "-Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "-L" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui/lint/lint-qualification/auxiliary" "-A" "unused"
2019-08-13T01:57:25.5842783Z ------------------------------------------
2019-08-13T01:57:25.5842934Z 
2019-08-13T01:57:25.5843234Z ------------------------------------------
2019-08-13T01:57:25.5843405Z stderr:
2019-08-13T01:57:25.5843405Z stderr:
2019-08-13T01:57:25.5843693Z ------------------------------------------
2019-08-13T01:57:25.5844021Z warning: use of deprecated item 'try': use the `?` operator instead
2019-08-13T01:57:25.5844707Z    |
2019-08-13T01:57:25.5844707Z    |
2019-08-13T01:57:25.5845016Z LL |     let _ = || -> Result<(), ()> { try!(Ok(())); Ok(()) }; // issue #37345
2019-08-13T01:57:25.5845326Z    |
2019-08-13T01:57:25.5845456Z    = note: `#[warn(deprecated)]` on by default
2019-08-13T01:57:25.5845563Z 
2019-08-13T01:57:25.5845678Z error: unnecessary qualification
2019-08-13T01:57:25.5845678Z error: unnecessary qualification
2019-08-13T01:57:25.5845988Z   --> /checkout/src/test/ui/lint/lint-qualification.rs:10:5
2019-08-13T01:57:25.5846164Z    |
2019-08-13T01:57:25.5846286Z LL |     foo::bar(); //~ ERROR: unnecessary qualification
2019-08-13T01:57:25.5846534Z    |
2019-08-13T01:57:25.5846667Z note: lint level defined here
2019-08-13T01:57:25.5846988Z   --> /checkout/src/test/ui/lint/lint-qualification.rs:1:9
2019-08-13T01:57:25.5847140Z    |
---
2019-08-13T01:57:25.5872356Z thread 'main' panicked at 'Some tests failed', src/tools/compiletest/src/main.rs:536:22
2019-08-13T01:57:25.5872836Z note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
2019-08-13T01:57:25.5887552Z 
2019-08-13T01:57:25.5887816Z 
2019-08-13T01:57:25.5890275Z command did not execute successfully: "/checkout/obj/build/x86_64-unknown-linux-gnu/stage0-tools-bin/compiletest" "--compile-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib" "--run-lib-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/lib/rustlib/x86_64-unknown-linux-gnu/lib" "--rustc-path" "/checkout/obj/build/x86_64-unknown-linux-gnu/stage2/bin/rustc" "--src-base" "/checkout/src/test/ui" "--build-base" "/checkout/obj/build/x86_64-unknown-linux-gnu/test/ui" "--stage-id" "stage2-x86_64-unknown-linux-gnu" "--mode" "ui" "--target" "x86_64-unknown-linux-gnu" "--host" "x86_64-unknown-linux-gnu" "--llvm-filecheck" "/usr/lib/llvm-6.0/bin/FileCheck" "--host-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--target-rustcflags" "-Crpath -O -Cdebuginfo=0 -Zunstable-options  -Lnative=/checkout/obj/build/x86_64-unknown-linux-gnu/native/rust-test-helpers" "--docck-python" "/usr/bin/python2.7" "--lldb-python" "/usr/bin/python2.7" "--gdb" "/usr/bin/gdb" "--quiet" "--llvm-version" "6.0.0\n" "--system-llvm" "--cc" "" "--cxx" "" "--cflags" "" "--llvm-components" "" "--llvm-cxxflags" "" "--adb-path" "adb" "--adb-test-dir" "/data/tmp/work" "--android-cross-path" "" "--color" "always"
2019-08-13T01:57:25.5891302Z 
2019-08-13T01:57:25.5891556Z 
2019-08-13T01:57:25.5896908Z failed to run: /checkout/obj/build/bootstrap/debug/bootstrap test
2019-08-13T01:57:25.5896995Z Build completed unsuccessfully in 1:01:46
2019-08-13T01:57:25.5896995Z Build completed unsuccessfully in 1:01:46
2019-08-13T01:57:26.2658301Z ##[error]Bash exited with code '1'.
2019-08-13T01:57:26.2716469Z ##[section]Starting: Checkout
2019-08-13T01:57:26.2717898Z ==============================================================================
2019-08-13T01:57:26.2717938Z Task         : Get sources
2019-08-13T01:57:26.2717973Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

rust-highfive · 2019-08-13T21:11:38Z

The job mingw-check of your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-13T21:05:29.6285889Z ##[command]git remote add origin https://github.com/rust-lang/rust
2019-08-13T21:05:29.6299092Z ##[command]git config gc.auto 0
2019-08-13T21:05:30.6272037Z ##[command]git config --get-all http.https://github.com/rust-lang/rust.extraheader
2019-08-13T21:05:30.6277301Z ##[command]git config --get-all http.proxy
2019-08-13T21:05:30.6282982Z ##[command]git -c http.extraheader="AUTHORIZATION: basic ***" fetch --force --tags --prune --progress --no-recurse-submodules --depth=2 origin +refs/heads/*:refs/remotes/origin/* +refs/pull/63470/merge:refs/remotes/pull/63470/merge
---
2019-08-13T21:06:06.3462857Z do so (now or later) by using -b with the checkout command again. Example:
2019-08-13T21:06:06.3462885Z 
2019-08-13T21:06:06.3463225Z   git checkout -b <new-branch-name>
2019-08-13T21:06:06.3463251Z 
2019-08-13T21:06:06.3463317Z HEAD is now at 13221cb90 Merge 417f9ea90cf9bcccd8a1fa569a11a4fc071e3b8c into 60960a260f7b5c695fd0717311d72ce62dd4eb43
2019-08-13T21:06:06.3629640Z ##[section]Starting: Collect CPU-usage statistics in the background
2019-08-13T21:06:06.3632375Z ==============================================================================
2019-08-13T21:06:06.3632642Z Task         : Bash
2019-08-13T21:06:06.3632703Z Description  : Run a Bash script on macOS, Linux, or Windows

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

Mark-Simulacrum · 2019-08-13T21:12:15Z

(This PR has dropped the necessary bootstrap update, by splitting it into #63534).

rust-highfive · 2019-08-15T22:28:30Z

Your PR failed (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.

2019-08-15T22:27:48.4433130Z [command]/bin/bash --noprofile --norc /Users/vsts/agent/2.155.1/work/_temp/1ca1bbff-70cf-4f29-82c4-27dc5c23f5c9.sh
2019-08-15T22:27:48.4912950Z   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
2019-08-15T22:27:48.4913640Z                                  Dload  Upload   Total   Spent    Left  Speed
2019-08-15T22:27:48.4913910Z 
2019-08-15T22:27:48.4927440Z   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: rust-lang-ci-mirrors.s3-us-west-1.amazonaws.com
2019-08-15T22:27:48.5107870Z ##[error]Bash exited with code '6'.
2019-08-15T22:27:48.5386740Z ##[section]Starting: Upload CPU usage statistics
2019-08-15T22:27:48.5391590Z ==============================================================================
2019-08-15T22:27:48.5391690Z Task         : Bash
2019-08-15T22:27:48.5391790Z Description  : Run a Bash script on macOS, Linux, or Windows
---
2019-08-15T22:27:48.7372740Z Script contents:
2019-08-15T22:27:48.7374240Z aws s3 cp --acl public-read cpu-usage.csv s3://$DEPLOY_BUCKET/rustc-builds/$BUILD_SOURCEVERSION/cpu-$SYSTEM_JOBNAME.csv
2019-08-15T22:27:48.7402300Z ========================== Starting Command Output ===========================
2019-08-15T22:27:48.7428490Z [command]/bin/bash --noprofile --norc /Users/vsts/agent/2.155.1/work/_temp/d63caf75-f264-4e34-9cbc-80c2be3d782d.sh
2019-08-15T22:27:48.7602840Z /Users/vsts/agent/2.155.1/work/_temp/d63caf75-f264-4e34-9cbc-80c2be3d782d.sh: line 1: aws: command not found
2019-08-15T22:27:48.7690540Z ##[error]Bash exited with code '127'.
2019-08-15T22:27:48.7735790Z ##[section]Starting: Checkout
2019-08-15T22:27:48.7738930Z ==============================================================================
2019-08-15T22:27:48.7739140Z Task         : Get sources
2019-08-15T22:27:48.7739250Z Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

bors · 2019-08-15T22:28:39Z

💔 Test failed - checks-azure

Centril · 2019-08-15T22:30:55Z

@bors retry spurious

bors · 2019-08-16T00:37:22Z

⌛ Testing commit 417f9ea with merge 445fdfde3204773574bead82363e4e82b2d06a57...

bors · 2019-08-16T04:37:22Z

💥 Test timed out

Mark-Simulacrum · 2019-08-16T12:18:54Z

@bors retry

Pretty sure literally every builder passed and homu just didn't get the notification (cc @pietroalbini - does that look similar to #63513?)

pietroalbini · 2019-08-16T12:25:32Z

No, GitHub was correctly updated. Let me see if there are any errors on homu's side.

pietroalbini · 2019-08-16T12:29:25Z

Oh this is a legit timeout: the build took 4 hours and 6 seconds, which is more than the 4 hour timeout...

Mark-Simulacrum · 2019-08-16T12:44:00Z

Longest builder took just under 4 hours (https://dev.azure.com/rust-lang/rust/_build/results?buildId=6037&view=logs) AFAICT, and it looks like that builder is taking 3h45m or so in other auto builds so presumably this is not unexpected. We should try and move some things around as the partner i686-mingw-1 builder is only taking 3 hours (so we have an hour or so to split across two builders, in theory).

pietroalbini · 2019-08-16T12:49:38Z

Longest builder took just under 4 hours

Yeah, but in the summary the recorded build time was 4 hours and 6 seconds, which is when the final check suite was updated.

We should try and move some things around as the partner i686-mingw-1 builder is only taking 3 hours (so we have an hour or so to split across two builders, in theory).

I'll look into this.

RalfJung · 2019-08-16T13:01:03Z

Some of the text in the OP still sounds like it comes from an alternative universe where this landed before the beta cutoff?

Mark-Simulacrum · 2019-08-16T13:21:37Z

Thanks for the reminder, I had forgotten to remove that text. Should be all down to just this PR now.

bors · 2019-08-16T14:41:20Z

⌛ Testing commit 417f9ea with merge 9a32ad0...

Utilize -Zbinary-dep-depinfo in rustbuild We no longer utilize stamp-file mtimes at all inside rustbuild, and a future PR may be able to entirely eliminate them by eagerly copying to the appropriate sysroot. The only mtime-based dependency tracking left is for documentation because we lie to Cargo about the rustdoc binary, so Cargo does not track changes to the real binary, and codegen-backends because binary-dep-depinfo does not emit that information into the depfiles. Both of these are fixable in the longer term but this existing patch gives us the following benefits: * We no longer delete Cargo target directories manually within a stage. Cross-stage, changes to codegen backends will still clear out target directories. This means that incremental state persists across individual steps (e.g., rebuilding libstd does not clear out librustc incremental state). Fixes #54712. * Dependency tracking across steps within a given stage is now fully precise. We will not clear out all codegen backend dependencies due to changes in librustc_driver, for example, only deleting the final librustc_codegen_llvm crate. Fixes #54008, fixes #50481. * We properly track codegen backends as a dependency (equivalent to rustc) across changes. Fixes #53284, and fixes #52719. * Cross-stage dependency tracking of crates is also much more accurate and reliable. Most likely fixes #49979 (but no reproduction steps in that issue). Fixes #59105. cc #63012

bors · 2019-08-16T18:35:10Z

☀️ Test successful - checks-azure
Approved by: alexcrichton
Pushing 9a32ad0 to master...

LukasKalbertodt · 2019-08-16T18:56:17Z

Thank you a ton!

PRs like these are huge but are not the ones getting all the attention, for obvious reasons. This PR will make the lives of almost all people working on this repo a lot better! While the fixed issues sound like minor inconveniences, they made working on the compiler very annoying at times. I almost gave up on compiler-hacking twice just for annoyances like these! Reducing frustrations with the build-system goes a long way in attracting and keeping more people hacking on this repo.

Please excuse my ramblings, but I just had to express my gratitude for PRs like this. Thanks again!

rust-highfive assigned alexcrichton Aug 11, 2019

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Aug 11, 2019

This comment has been minimized.

Sign in to view

alexcrichton reviewed Aug 12, 2019

View reviewed changes

src/bootstrap/bin/rustc.rs Outdated Show resolved Hide resolved

src/bootstrap/compile.rs Show resolved Hide resolved

Mark-Simulacrum force-pushed the rustc-depdep branch 2 times, most recently from e4f7726 to 0be63bd Compare August 12, 2019 18:57

alexcrichton approved these changes Aug 12, 2019

View reviewed changes

Mark-Simulacrum force-pushed the rustc-depdep branch 2 times, most recently from c034578 to cecdcdb Compare August 12, 2019 22:05

Mark-Simulacrum force-pushed the rustc-depdep branch 2 times, most recently from 5739811 to beab7fb Compare August 12, 2019 23:52

Mark-Simulacrum force-pushed the rustc-depdep branch from beab7fb to cc0bdab Compare August 13, 2019 00:48

Mark-Simulacrum force-pushed the rustc-depdep branch from cc0bdab to 9bec456 Compare August 13, 2019 11:08

Centril mentioned this pull request Aug 13, 2019

[Place 2.0] Convert Place's projection to a boxed slice #63420

Merged

Mark-Simulacrum force-pushed the rustc-depdep branch from 9bec456 to 40da969 Compare August 13, 2019 21:01

Utilize -Zbinary-dep-depinfo for dependency tracking

417f9ea

Mark-Simulacrum force-pushed the rustc-depdep branch from 40da969 to 417f9ea Compare August 13, 2019 21:04

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 15, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 15, 2019

bors added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. labels Aug 16, 2019

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 16, 2019

bors added the merged-by-bors This PR was explicitly merged by bors. label Aug 16, 2019

bors merged commit 417f9ea into rust-lang:master Aug 16, 2019

bors mentioned this pull request Aug 16, 2019

bootstrap: Merge the libtest build step with libstd #63637

Merged

Mark-Simulacrum deleted the rustc-depdep branch August 16, 2019 19:58

Mark-Simulacrum mentioned this pull request Sep 11, 2019

Spurious "found two different crates with name alloc" errors in PR builder #64389

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Utilize -Zbinary-dep-depinfo in rustbuild #63470

Utilize -Zbinary-dep-depinfo in rustbuild #63470

Mark-Simulacrum commented Aug 11, 2019 •

edited

Loading

Mark-Simulacrum commented Aug 11, 2019 •

edited

Loading

This comment has been minimized.

alexcrichton left a comment

rust-highfive commented Aug 12, 2019

Mark-Simulacrum commented Aug 12, 2019

rust-highfive commented Aug 13, 2019

rust-highfive commented Aug 13, 2019

rust-highfive commented Aug 13, 2019

Mark-Simulacrum commented Aug 13, 2019

rust-highfive commented Aug 15, 2019

bors commented Aug 15, 2019

Centril commented Aug 15, 2019

bors commented Aug 16, 2019

bors commented Aug 16, 2019

Mark-Simulacrum commented Aug 16, 2019

pietroalbini commented Aug 16, 2019

pietroalbini commented Aug 16, 2019 •

edited

Loading

Mark-Simulacrum commented Aug 16, 2019

pietroalbini commented Aug 16, 2019

RalfJung commented Aug 16, 2019

Mark-Simulacrum commented Aug 16, 2019

bors commented Aug 16, 2019

bors commented Aug 16, 2019

LukasKalbertodt commented Aug 16, 2019

Utilize -Zbinary-dep-depinfo in rustbuild #63470

Utilize -Zbinary-dep-depinfo in rustbuild #63470

Conversation

Mark-Simulacrum commented Aug 11, 2019 • edited Loading

Mark-Simulacrum commented Aug 11, 2019 • edited Loading

This comment has been minimized.

alexcrichton left a comment

Choose a reason for hiding this comment

rust-highfive commented Aug 12, 2019

Mark-Simulacrum commented Aug 12, 2019

rust-highfive commented Aug 13, 2019

rust-highfive commented Aug 13, 2019

rust-highfive commented Aug 13, 2019

Mark-Simulacrum commented Aug 13, 2019

rust-highfive commented Aug 15, 2019

bors commented Aug 15, 2019

Centril commented Aug 15, 2019

bors commented Aug 16, 2019

bors commented Aug 16, 2019

Mark-Simulacrum commented Aug 16, 2019

pietroalbini commented Aug 16, 2019

pietroalbini commented Aug 16, 2019 • edited Loading

Mark-Simulacrum commented Aug 16, 2019

pietroalbini commented Aug 16, 2019

RalfJung commented Aug 16, 2019

Mark-Simulacrum commented Aug 16, 2019

bors commented Aug 16, 2019

bors commented Aug 16, 2019

LukasKalbertodt commented Aug 16, 2019

Mark-Simulacrum commented Aug 11, 2019 •

edited

Loading

Mark-Simulacrum commented Aug 11, 2019 •

edited

Loading

pietroalbini commented Aug 16, 2019 •

edited

Loading