Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix x build library/std compiler/rustc #99130

Merged
merged 1 commit into from
Jul 11, 2022

Conversation

jyn514
Copy link
Member

@jyn514 jyn514 commented Jul 10, 2022

Previously, this was broken because of improper caching:

  1. StepDescription::maybe_run builds Compile::Std, which only built std and not proc_macro
  2. Std calls builder.ensure(StdLink)
  3. Rustc calls ensure(Std), which builds all crates, including proc_macro
  4. Rustc calls ensure(StdLink). ensure would see that it had already been run and do nothing. <-- bug is here
  5. Cargo gives an error that proc_macro doesn't exist.

This fixes the caching by adding crates to StdLink, so it will get rerun if the crates that are built change.

Fixes #99129.

@jyn514 jyn514 added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Jul 10, 2022
@rust-highfive
Copy link
Collaborator

r? @Mark-Simulacrum

(rust-highfive has picked a reviewer for you, use r? to override)

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 10, 2022
@Mark-Simulacrum
Copy link
Member

RustcLink presumably needs the same treatment?

@jyn514
Copy link
Member Author

jyn514 commented Jul 10, 2022

ah, good call - not sure how you could trigger this for RustcLink, since it's only called from impl Step for Rustc, but it's good to future proof in case we want to build individual crates (e.g. for rustfmt) in the future.

Previously, this was broken because of improper caching:
1. `StepDescription::maybe_run` builds `Compile::Std`, which only built `std` and not `proc_macro`
1. `Std` calls `builder.ensure(StdLink)`
1. `Rustc` calls `ensure(Std)`, which builds all crates, including `proc_macro`
1. `Rustc` calls `ensure(StdLink)`. `ensure` would see that it had already been run and do nothing.  <-- bug is here
1. Cargo gives an error that `proc_macro` doesn't exist.

This fixes the caching by adding `crates` to `StdLink`, so it will get rerun if the crates that are
built change.  This also does the same for `RustcLink`; it doesn't matter in practice currently
because nothing uses it except `impl Step for Rustc`, but it will avoid bugs if we start using it in
the future (e.g. to build individual crates for rustfmt).
@jyn514 jyn514 force-pushed the std-cache-invalidation branch from 587ea8b to 7d4bd54 Compare July 10, 2022 22:51
@Mark-Simulacrum
Copy link
Member

@bors r+ rollup p=5 (breaking builds for folks in practice)

@bors
Copy link
Contributor

bors commented Jul 11, 2022

📌 Commit 7d4bd54 has been approved by Mark-Simulacrum

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 11, 2022
@jyn514
Copy link
Member Author

jyn514 commented Jul 11, 2022

@Mark-Simulacrum I don't think https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/NIghtly.20bustage.3F is actually related - the problem there is that library/std means something different than it used to ("build std" instead of "build all crates in the standard library"). This only affects specifically build library/std compiler/rustc, not build library/std and then rustc +local ....

@Mark-Simulacrum
Copy link
Member

Yes, I agree, but I think this is still something we should fix fairly quickly; maybe @bors p=1 though.

@bors
Copy link
Contributor

bors commented Jul 11, 2022

⌛ Testing commit 7d4bd54 with merge a51fb2b...

@bors
Copy link
Contributor

bors commented Jul 11, 2022

☀️ Test successful - checks-actions
Approved by: Mark-Simulacrum
Pushing a51fb2b to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 11, 2022
@bors bors merged commit a51fb2b into rust-lang:master Jul 11, 2022
@rustbot rustbot added this to the 1.64.0 milestone Jul 11, 2022
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (a51fb2b): comparison url.

Instruction count

  • Primary benchmarks: no relevant changes found
  • Secondary benchmarks: 🎉 relevant improvements found
mean1 max count2
Regressions 😿
(primary)
N/A N/A 0
Regressions 😿
(secondary)
N/A N/A 0
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
-1.5% -1.8% 6
All 😿🎉 (primary) N/A N/A 0

Max RSS (memory usage)

Results
  • Primary benchmarks: no relevant changes found
  • Secondary benchmarks: mixed results
mean1 max count2
Regressions 😿
(primary)
N/A N/A 0
Regressions 😿
(secondary)
3.6% 3.6% 1
Improvements 🎉
(primary)
N/A N/A 0
Improvements 🎉
(secondary)
-2.4% -3.0% 3
All 😿🎉 (primary) N/A N/A 0

Cycles

This benchmark run did not return any relevant results for this metric.

If you disagree with this performance assessment, please file an issue in rust-lang/rustc-perf.

@rustbot label: -perf-regression

Footnotes

  1. the arithmetic mean of the percent change 2

  2. number of relevant changes 2

@jyn514 jyn514 deleted the std-cache-invalidation branch July 11, 2022 13:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

x build library/std compiler/rustc gives an error that proc_macro isn't built
6 participants