Support building with "cargo" directly instead of "x.py" for development purposes #70999
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
(rust_highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
luca-barbieri
force-pushed
the
cargo_build
branch
2 times, most recently
from
340740b to
bbdc4bf
Compare
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
luca-barbieri
force-pushed
the
cargo_build
branch
3 times, most recently
from
c4fed25 to
98f6786
Compare
|
The job Click to expand the log.I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact |
luca-barbieri
force-pushed
the
cargo_build
branch
from
98f6786 to
0187ae3
Compare
rustc_session exports it for other crates to avoid mismatching crate versions.
Otherwise, the compiler can't actually compile anything.
The code was broken because it printed "llvm-config" instead of the absolute path to the llvm-config executable, causing Cargo to always rebuild librustc_llvm if using system LLVM. Also, it's not the build system's job to rebuild when a system library changes, so we simply don't emit "rerun-if-changed" if a path to LLVM was not explicitly provided.
x.py sets it unconditionally, so want it for plain "cargo build"
…ion_env!() from multiple crates
librustc_session already has them, and the crates depend on it, so they will be rebuilt anyway
If the sysroot does not exist, use the one from the toolchain used to build rustc.
… on beta This provides helpful messages when using `cargo build` and sets --cfg=bootstrap when using a beta toolchain.
luca-barbieri
force-pushed
the
cargo_build
branch
from
0187ae3 to
8c1303b
Compare
|
This is amazing! Thank you so much for getting this off the ground :) I've pulled some of the commits here into a separate PR (#71029) and approved that, just to reduce the amount of commits in this PR. Those I'm prepared to see land immediately, I think they're required for progress here anyway. I'm going to need to think a bit about the overall design, though. I'm not quite sure yet whether I want to split and somewhat duplicate code that's currently in bootstrap (or just doesn't really exist). One thought I had is that we should probable come up with some basic "workflows" -- e.g., I'm not quite sure what users would expect with a cargo-native build system. I also think that to start we should just focus on supporting building and testing std, not rustc; rustc is more complicated (needs a bunch of environment variables set, has sysroot management to deal with, can't easily run without a std built by it)... With that in mind, I think most of this PR I'm not prepared to land immediately, given that it introduces a lot of code that is either duplicating other code or adding code to manage some process, and I suspect that may be avoidable, so would like to try to avoid it first. I'm primarily concerned that we don't end up with divergent code between the environment variables that bootstrap sets and what the "cargo build" workflow wants, in an effort to avoid situations where we end up with that leading to e.g. CI failing but locally thing seeming to work. I think the first thing we should get working is the following, in order:
I believe that the first already mostly works on master and with the panic-unwind feature entirely works; certainly after #71029 it works fine. We should probably document that as soon as that lands in rustc-dev-guide, as it seems a major improvement for folks hacking on std with just the need for check/build builds, even if tests don't work quite yet. The second will be more difficult. tests for core/alloc/std I believe all depend on rand, which has not been prepared for a in-tree world with rustc-dep-of-std. That means that they cannot build successfully unless you do what you did with -Zbuild-std. That's a wall of text for sure, but hopefully I'm not too unclear. Feel free to ask if I should clarify my thoughts or if you disagree. |
|
In terms of altering rustc -- I think one thought I've had is that we can likely move things from bootstrap's build parts into build.rs in many cases, especially with a helper crate that things can depend on which will parse config.toml to the extent needed (basically build_helper I think). Another thought I had is that we might be able to build sysroots via a build.rs in a crate that has a build-dependency on src/rustc/Cargo.toml, which can the install the rustc and relevant libraries into the sysroot. |
Dylan-DPC-zz
pushed a commit
to Dylan-DPC-zz/rust
that referenced
this pull request
Apr 12, 2020
…-Simulacrum Partial work on building with Cargo This cherry picks the commits I'm directly approving from rust-lang#70999, I want to land them so that that PR is smaller.
|
☔ The latest upstream changes (presumably #71059) made this pull request unmergeable. Please resolve the merge conflicts. |
joelpalmer
added
S-waiting-on-review
Dylan-DPC-zz
added
S-waiting-on-author
|
@luca-barbieri if you can rebase this, we can get it merged |
joelpalmer
added
S-waiting-on-author
|
Ping from Triage: Closing due to inactivity. @luca-barbieri please reopen with conflicts resolved. Thanks for the PR. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request allows to just use
cargoin addition tox.pyfor rustc and libstd development.This makes it much simpler for new contributors to contribute to rust since there is no need to learn how
x.pyand bootstrapping works, and makes incremental development faster because this new infrastructure does no bootstrapping at all.To make it work, the user is supposed to install the latest nightly with rustup, and then make changes based on a checkout of the same git commit hash of that nightly: this allows to build a new rustc that works with the standard library from the nightly, and to build a new non-cfg(bootstrap) standard library with the compiler from the nightly.
To support this workflow, we check and warn in build.rs if the compiler and repository versions don't match, and we set the compiler release to the same one of the compiler used to build so that artifacts can be shared between them.
Summary of changes:
x.pyWith these changes, this is currently working:
cargo checkfor single cratescargo runin src/rustc and having a working compiler without any bootstrappingcargo buildin src/libstd, src/libcore or src/libtest__CARGO_TESTS_ONLY_SRC_ROOT=$(realpath $PWD/../..) cargo test -Z build-std --target x86_64-unknown-linux-gnuin src/libcoreWhat is not working and missing:
x.pyFuture work would be to document this, and modify cargo's std-aware code so that libstd can be tested and the special incantation is no longer needed to run libstd tests, and so that you can just build the whole workspace with "cargo build" and have it work as if the pieces were built independently.