-
Notifications
You must be signed in to change notification settings - Fork 12.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
creader: Host crate loaded twice produces different CrateNum
s if host != target
#56935
Comments
If I'm not mistaken, the issue will affect any idiomatic Rust 2018 code trying to use proc macros together with cross-compilation. |
CrateNum
s if host != targetCrateNum
s if host != target
The workaround is to avoid using host crates through extern prelude and use them through |
Sorry for the late delay in checking in on this. This is somewhat expected in the sense that we load proc-macro crates from the compiler's target and the actual target differently as they may be two different crates. That was done a really long time ago and I never really followed all of the macro developments and how they affected cross compilation. We can probably add a function that compares two CrateNum instances though to see if they only differ by target and maybe that'd help here? |
metadata: Some crate loading cleanup So, my goal was to fix caching of loaded crates which is broken and causes ICEs like rust-lang#56935 or rust-lang#64450. While investigating I found that the code is pretty messy and likes to confuse various things that look similar but are actually different. This PR does some initial cleanup in that area, I hope to get to the caching itself a bit later.
metadata: Some crate loading cleanup So, my goal was to fix caching of loaded crates which is broken and causes ICEs like rust-lang#56935 or rust-lang#64450. While investigating I found that the code is pretty messy and likes to confuse various things that look similar but are actually different. This PR does some initial cleanup in that area, I hope to get to the caching itself a bit later.
Posting a typical message and backtrace here to make this issue easier to find:
|
It would be great if someone come up with a short-term fix for this, perhaps hacky. Otherwise I hope to get to this in June and fix it together with a few other issues in crate loading. |
allow wasm target for rustc-ap-rustc_span This fixes rust-lang#71998 by applying the work-a-round. The root cause is probably rust-lang#56935, as @petrochenkov pointed out. I reproduced the bug by: ``` cd ~/.cargo/registry/src/github.com-1ecc6299db9ec823/rustc-ap-rustc_span-657.0.0/ cargo build --target wasm32-unknown-unknown ``` Adding this line fixes it.
Work around ICEs during cross-compilation for target, ast, & attr This applies the fix for rust-lang#72003 to work around rust-lang#56935 to three more libraries. With these additional fixes, I'm able to use rustfmt_lib from wasm (rust-lang/rustfmt#4132 (comment)), which was my goal. To get it working locally and to test, I copied the `.cargo/registry/src` and applied the fix and replaced the reference in my project: ``` toml [replace] "rustc-ap-rustc_span:656.0.0" = { path = "../rustc-ap-rustc_span" } "rustc-ap-rustc_target:656.0.0" = { path = "../rustc-ap-rustc_target" } "rustc-ap-rustc_ast:656.0.0" = { path = "../rustc-ap-rustc_ast" } "rustc-ap-rustc_attr:656.0.0" = { path = "../rustc-ap-rustc_attr" } ```
@alexcrichton do you remember what you meant by this? Why could they be different crates? Proc-macros are always compiled for the host, so they should be the same even when cross-compiling. In general, I'm not sure when this could go wrong - the compiler already checks if the target doesn't match and gives an error:
So I'm not sure why we have to check again when loading the dependency. I don't think it's possible to load two crates compiled for different targets at the same time, even without this check, it will only load the crate that matches |
@jyn514 does it also make sense to add the test case described in #85386 (comment) ? |
@TrueDoctor how is that test case different from https://github.com/rust-lang/rust/blob/6882454bd3fabc36f54cfb2125e6edd26e50ec21/src/test/ui/crate-loading/cross-compiled-proc-macro.rs ? |
@jyn514 I my testing I found that the compiler shows an error when you import one proc macro and panics with an ICE as soon as you import two or more proc macros from the same crate. |
@TrueDoctor I can't replicate that ICE. > cargo --version
cargo 1.55.0-nightly (495297903 2021-07-01)
> cat src/lib.rs
pub use reproduction::{ToDiscriminant, TransitiveChild};
use proc_macro::TokenStream;
> cat reproduction/src/lib.rs
#[proc_macro_derive(ToDiscriminant)]
pub fn derive_discriminant(_: TokenStream) -> TokenStream {
TokenStream::new()
}
#[proc_macro_derive(TransitiveChild)]
pub fn derive_transitive_child(_: TokenStream) -> TokenStream {
TokenStream::new()
}
> cargo check --target=wasm32-unknown-unknown
Compiling reproduction v0.1.0 (/home/joshua/test-rustdoc/creader/reproduction)
Checking creader v0.1.0 (/home/joshua/test-rustdoc/creader)
Finished dev [unoptimized + debuginfo] target(s) in 1.20s |
@jyn514 I can still replicate the issue, both with stable and nightly: graphite/editor on ice-debugging [?] ➜ cat ../proc-macro/src/lib.rs
use proc_macro::TokenStream;
#[proc_macro_derive(ToDiscriminant)]
pub fn derive_discriminant(_: TokenStream) -> TokenStream {
TokenStream::new()
}
#[proc_macro_derive(TransitiveChild)]
pub fn derive_transitive_child(_: TokenStream) -> TokenStream {
TokenStream::new()
}
graphite/editor on ice-debugging [?] ➜ cat src/lib.rs
pub use proc_macros::{ToDiscriminant, TransitiveChild};
graphite/editor on ice-debugging [?] ➜ cargo check --target wasm32-unknown-unknown
Checking graphite-editor-core v0.1.0 (/home/dennis/Projects/rust/Graphite/editor)
thread 'rustc' panicked at 'Failed to get crate data for crate15', compiler/rustc_metadata/src/creader.rs:139:32
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: internal compiler error: unexpected panic
note: the compiler unexpectedly panicked. this is a bug.
note: we would appreciate a bug report: https://github.com/rust-lang/rust/issues/new?labels=C-bug%2C+I-ICE%2C+T-compiler&template=ice.md
note: rustc 1.55.0-nightly (b3d11f95c 2021-07-04) running on x86_64-unknown-linux-gnu
note: compiler flags: -C embed-bitcode=no -C debuginfo=2 -C incremental --crate-type lib
note: some of the compiler flags provided by cargo are hidden
query stack during panic:
end of query stack
error: could not compile `graphite-editor-core`
To learn more, run the command again with --verbose. I'm using this repo for testing: https://github.com/GraphiteEditor/Graphite/tree/ice-debugging |
The only difference I can spot in which file |
Sorry it's been so long I don't really remember most of this code here any more. |
remove unused FIXME rust-lang#56935 seems to be fixed.
remove unused FIXME rust-lang#56935 seems to be fixed.
remove unused FIXME rust-lang#56935 seems to be fixed.
remove unused FIXME rust-lang#56935 seems to be fixed.
rust-lang/rust#56935 was closed via rust-lang/rust#86876. Suggested-by: bjorn3 <bjorn3_gh@protonmail.com> Signed-off-by: Miguel Ojeda <ojeda@kernel.org>
Reproduction (as an UI test):
Name resolution will try to load the crate twice, first in initial pass, then in validation pass.
The results will be different due to different loaded
CrateNum
s, so we get the "inconsistent resolution for a macro" assertion.This happens due to the
locate_ctxt.triple == &self.sess.opts.target_triple
condition inCrateLoader::load
preventing reuse ofCrateNum
s for host crates during cross-compilation.cc @alexcrichton probably
The text was updated successfully, but these errors were encountered: