Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking Issue for per-package-target #9406

Open
3 tasks
Tracked by #84
Ekleog opened this issue Apr 24, 2021 · 22 comments
Open
3 tasks
Tracked by #84

Tracking Issue for per-package-target #9406

Ekleog opened this issue Apr 24, 2021 · 22 comments
Labels
C-tracking-issue Category: A tracking issue for something unstable. S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. S-needs-mentor Status: Issue or feature is accepted, but needs a team member to commit to helping and reviewing. S-waiting-on-feedback Status: An implemented feature is waiting on community feedback for bugs or design concerns. Z-per-package-target Nightly: per-package-target

Comments

@Ekleog
Copy link
Contributor

Ekleog commented Apr 24, 2021

Summary

Feature gate: per_package_target
Discussion: https://internals.rust-lang.org/t/proposal-move-some-cargo-config-settings-to-cargo-toml/13336
Implementation: #9030
Documentation: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#per-package-target
Issues: Z-per-package-target Nightly: per-package-target

Unresolved issues

  • The interaction with proposals like RFC 3028 does not work well. Selection for targets to build happens at the workspace level instead of at a level where the full compile graph is known. This affects selected packages (target-specific dependencies) and feature resolution as well. There's also some possible confusion, as the per-package-target settings are ignored in dependencies.
  • The exact interaction with feature resolution is a bit up in the air. This may not be super well-tested where in theory each target artifact should get an independently resolved feature graph (ish). Related to: per-package-target unifies features which can conflict with other targets #9521
  • Interaction with -Zbuild-std
@Ekleog Ekleog added the C-tracking-issue Category: A tracking issue for something unstable. label Apr 24, 2021
@Ekleog Ekleog changed the title Tracking Issue for XXX Tracking Issue for per-package-target Apr 24, 2021
@MarkSwanson
Copy link

#9521
I'm not sure if I should place this here. Just trying to help with keeping track...

@FrankvdStam
Copy link

See #9537

@Airtz
Copy link

Airtz commented Jul 22, 2021

Another issue is the interaction with -Zmultitarget #8176

IMO we should be able to do the following:

[package]
forced-target = ["x86_64-pc-windows-msvc", "i686-pc-windows-msvc"]

tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
tmplt added a commit to rtic-scope/cargo-rtic-scope that referenced this issue Nov 1, 2021
While we do have a dedicated repo for RTIC examples, it makes more sense
to merge those too with this repo: cargo-rtic-scope, cortem-m-rtic-trace
and relevant examples are all interlinked and must be in phase with
eachother.

However, we can't add it to the workspace yet because
per-package-target[0] is not yet stable.

[0] rust-lang/cargo#9406
@kennystrawnmusic
Copy link

kennystrawnmusic commented Nov 20, 2021

Performing some exhaustive tests when attempting to resolve #9451 revealed that #9521 is likely a duplicate of it, all thanks to a very specific panic message that shows when an attempt to compile a per-package-target crate with -Zbuild-std is made using a local fork of Cargo with the check that returns the error mentioned in #9451 commented out.

@kaimast
Copy link

kaimast commented Jan 1, 2022

I tried to set forced target to wasm32-unknown-unknown for a project and get the following panic:

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "wasm-bindgen-futures", version: "0.4.28", source: "registry `crates-io`" } false

Stack backtrace:
   0: <cargo::core::resolver::features::ResolvedFeatures>::activated_features_int
   1: cargo::core::compiler::unit_dependencies::new_unit_dep_with_profile
   2: cargo::core::compiler::unit_dependencies::compute_deps
   3: cargo::core::compiler::unit_dependencies::deps_of
   4: cargo::core::compiler::unit_dependencies::deps_of
   5: cargo::core::compiler::unit_dependencies::deps_of
   6: cargo::core::compiler::unit_dependencies::deps_of_roots
   7: cargo::core::compiler::unit_dependencies::build_unit_dependencies
   8: cargo::ops::cargo_compile::create_bcx
   9: cargo::ops::cargo_compile::compile_ws
  10: cargo::ops::cargo_compile::compile
  11: cargo::commands::build::exec
  12: cargo::cli::execute_subcommand
  13: cargo::cli::main
  14: cargo::main
  15: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  16: std::rt::lang_start::<()>::{closure#0}
  17: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/core/src/ops/function.rs:259:13
  18: std::panicking::try::do_call
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panicking.rs:406:40
  19: std::panicking::try
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panicking.rs:370:19
  20: std::panic::catch_unwind
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panic.rs:133:14
  21: std::rt::lang_start_internal::{{closure}}
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/rt.rs:128:48
  22: std::panicking::try::do_call
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panicking.rs:406:40
  23: std::panicking::try
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panicking.rs:370:19
  24: std::panic::catch_unwind
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/panic.rs:133:14
  25: std::rt::lang_start_internal
             at /rustc/48a5999fceeea84a8971634355287faa349909d4/library/std/src/rt.rs:128:20
  26: main
  27: __libc_start_main
  28: <unknown>', src/tools/cargo/src/cargo/core/resolver/features.rs:296:14

Any idea what could be causing this?

The project compiles fine with wasm-pack. Not sure if that tool sets any special flags, but either way cargo should not panic.

@Ekleog
Copy link
Contributor Author

Ekleog commented Jan 8, 2022

Hmm this looks bad indeed. Do you have an example workspace to reproduce? (Replacing all the code with empty files should most likely still allow reproducing, I'd guess only keeping the Cargo.toml files should be enough)

@kaimast
Copy link

kaimast commented Jan 8, 2022

I made a test repository here.

You should get something like this when compiling simba-wasm without any flags:

~/dev/wasm-compile-test main ❯ cargo build --package=simba-wasm                                                         ⬢
thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "wasm-bindgen-futures", version: "0.4.28", source: "registry `crates-io`" } false', src/tools/cargo/src/cargo/core/resolver/features.rs:296:14
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Interesting cargo build --target=wasm32-unknown-unknown --package=simba-wasm works fine.

@ghost
Copy link

ghost commented Jul 21, 2022

Also related: #10518.

@maddiemort
Copy link

I'm having the same activated_features for invalid package panic

@maddiemort
Copy link

It looks like it might be related to a [target.'cfg(target_arch = "wasm32")'.dependencies] section in a path dependency's Cargo.toml.

@Qubasa
Copy link

Qubasa commented Sep 29, 2022

cargo-features = ["per-package-target", "profile-rustflags"] don't work together.
By this I mean the target does not get applied when profile-rustflags get used.

cargo-features = ["per-package-target", "profile-rustflags"]

[package]
name = "perf_kernel"
version = "0.1.0"
authors = ["Luis Hebendanz <luis.nixos@gmail.com>"]
edition = "2021"
forced-target = "x86_64-unknown-none"

[profile.dev]
rustflags = [ "-C", "link-args=--image-base=0x200000" ]

Will fail with gcc: error: unrecognized command-line option '--image-base=0x200000' even though target defines ld.lld as linker

@max-cura
Copy link

max-cura commented Mar 18, 2023

Also ran into this:

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "js-sys", version: "0.3.60", source: "registry `crates-io`" } NormalOrDev

when running cargo check.

Had forced-target=wasm32-unknown-unknown in one of my crates (if I remove that line, it stops panicking), and the issue was coming from getrandom = { version="0.2.8", features=["js"] }, which has the following, which seems to be what's being referenced in the panic message:

[target.'cfg(all(any(target_arch = "wasm32", target_arch = "wasm64"), target_os = "unknown"))'.dependencies]
wasm-bindgen = { version = "0.2.62", default-features = false, optional = true }
js-sys = { version = "0.3", optional = true }
Stack backtrace:
   0: std::backtrace::Backtrace::create
   1: ::msg::
   2: ::activated_features_int
   3: cargo::core::compiler::unit_dependencies::new_unit_dep_with_profile
   4: cargo::core::compiler::unit_dependencies::compute_deps
   5: cargo::core::compiler::unit_dependencies::deps_of
   6: cargo::core::compiler::unit_dependencies::deps_of
   7: cargo::core::compiler::unit_dependencies::deps_of
   8: cargo::core::compiler::unit_dependencies::deps_of
   9: cargo::core::compiler::unit_dependencies::deps_of_roots
  10: cargo::core::compiler::unit_dependencies::build_unit_dependencies
  11: cargo::ops::cargo_compile::create_bcx
  12: cargo::ops::cargo_compile::compile_ws
  13: cargo::ops::cargo_compile::compile
  14: cargo::commands::check::exec
  15: cargo::cli::main
  16: cargo::main
  17: std::sys_common::backtrace::__rust_begin_short_backtrace::
  18: std::rt::lang_start::<()>::{closure#0}
  19: std::rt::lang_start_internal
  20: _main', src/tools/cargo/src/cargo/core/resolver/features.rs:321:14
stack backtrace:
   0:        0x1012b2328 - ::fmt::hcc0d0be5d915b423
   1:        0x1012d31c4 - core::fmt::write::hd192c56b84d0d5fc
   2:        0x1012ad4c4 - std::io::Write::write_fmt::hc875b6ab61d117f8
   3:        0x1012b213c - std::sys_common::backtrace::print::hfdc3c075d7f207b2
   4:        0x1012b3ddc - std::panicking::default_hook::{{closure}}::h9a29056d92273910
   5:        0x1012b3b34 - std::panicking::default_hook::h918749fa9e3bbd56
   6:        0x1012b42ec - std::panicking::rust_panic_with_hook::h115b7bf8f2adb68e
   7:        0x1012b4220 - std::panicking::begin_panic_handler::{{closure}}::h610771468070e0bf
   8:        0x1012b2748 - std::sys_common::backtrace::__rust_end_short_backtrace::h84374e7a8d4c9564
   9:        0x1012b3f7c - _rust_begin_unwind
  10:        0x10133cfe4 - core::panicking::panic_fmt::heb926c9af39e8a59
  11:        0x10133d31c - core::result::unwrap_failed::h0edb6ed598db9dd6
  12:        0x100c67424 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::new_unit_dep_with_profile
  13:        0x100c64f00 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::compute_deps
  14:        0x100c63b08 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::deps_of
  15:        0x100c63cb4 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::deps_of
  16:        0x100c63cb4 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::deps_of
  17:        0x100c63cb4 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::deps_of
  18:        0x100c63970 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::deps_of_roots
  19:        0x100c62a70 - cargo[b7a945c812b2d771]::core::compiler::unit_dependencies::build_unit_dependencies
  20:        0x100f60fe0 - cargo[b7a945c812b2d771]::ops::cargo_compile::create_bcx
  21:        0x100f5f4d4 - cargo[b7a945c812b2d771]::ops::cargo_compile::compile_ws
  22:        0x100f5f3bc - cargo[b7a945c812b2d771]::ops::cargo_compile::compile
  23:        0x100b0c22c - cargo[4ef0020e058ba257]::commands::check::exec
  24:        0x100adbc60 - cargo[4ef0020e058ba257]::cli::main
  25:        0x100b1474c - cargo[4ef0020e058ba257]::main
  26:        0x100ab90b0 - std[d7a38f31206b4b19]::sys_common::backtrace::__rust_begin_short_backtrace::
  27:        0x100aa6bf4 - std[d7a38f31206b4b19]::rt::lang_start::<()>::{closure#0}
  28:        0x1012a6824 - std::rt::lang_start_internal::hfc06c7bfe3f9fee2
  29:        0x100b16a9c - _main

@str4d
Copy link

str4d commented Apr 2, 2023

How is this feature intended to interact with target.<target>.runner? My expectation was that if cargo run or cargo test is called on a package that has forced-target = "SOME_TARGET", then if a runner is configured via target.SOME_TARGET.runner it will be used. Instead it appears that the binary is compiled for the forced-target but run using the runner for the host target.

My motivation here is similar to #8112: I want to execute cargo test from inside folder foo/, where a foo/.cargo/config.toml currently sets build.target = "thumbv7em-none-eabihf", and have the test binary be run on an embedded device. Currently I do this by setting target.thumbv7em-none-eabihf.runner to a Bash script that changes directory to ../bar before calling cargo run ... (which compiles a tool on the host and then runs it to deploy the binary to the embedded device). However this isn't cross-platform, and there's no way to configure a different runner based on the host OS (that I can find), so I want to use cargo run --manifest-path ../bar/Cargo.toml ... directly as the runner.

@ehuss ehuss added S-needs-mentor Status: Issue or feature is accepted, but needs a team member to commit to helping and reviewing. S-waiting-on-feedback Status: An implemented feature is waiting on community feedback for bugs or design concerns. labels Apr 25, 2023
@ehuss ehuss added the S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. label May 8, 2023
@epage epage moved this to Unstable, baking in Cargo Roadmap Sep 5, 2023
@simbleau
Copy link

simbleau commented Dec 4, 2023

Please let's get some attention on this!

Qix- added a commit to Qix-/cargo-crash-force-target that referenced this issue Jan 1, 2024
@Qix-
Copy link

Qix- commented Jan 1, 2024

While testing this with a JSON file as the target path and using -Zbuild-std, I get a Cargo panic:

Repro here: https://github.com/Qix-/cargo-crash-force-target

Stacktrace
thread 'main' panicked at src/cargo/core/compiler/unit_dependencies.rs:177:34:
no entry found for key
stack backtrace:
   0:     0x7f9f051eef46 - std::backtrace_rs::backtrace::libunwind::trace::h44f4ca7c442c0986
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
   1:     0x7f9f051eef46 - std::backtrace_rs::backtrace::trace_unsynchronized::h8e72beef8f98799e
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x7f9f051eef46 - std::sys_common::backtrace::_print_fmt::had1f7322c7191a83
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/sys_common/backtrace.rs:68:5
   3:     0x7f9f051eef46 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hc02075f68ac68e83
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x7f9f05223a10 - core::fmt::rt::Argument::fmt::h4fb797bf569c9381
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/fmt/rt.rs:142:9
   5:     0x7f9f05223a10 - core::fmt::write::h6f1ea15fe9368d17
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/fmt/mod.rs:1120:17
   6:     0x7f9f051e820f - std::io::Write::write_fmt::haaa4143b58137630
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/io/mod.rs:1810:15
   7:     0x7f9f051eed24 - std::sys_common::backtrace::_print::h16f1f390d8dff6aa
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x7f9f051eed24 - std::sys_common::backtrace::print::h7a4cfc718891723c
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x7f9f051f0e07 - std::panicking::default_hook::{{closure}}::h66f09592313f4ce8
  10:     0x7f9f051f0b69 - std::panicking::default_hook::he45e0a376ce42f7d
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:292:9
  11:     0x7f9f051f1308 - std::panicking::rust_panic_with_hook::h4f38919015b1c60f
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:779:13
  12:     0x7f9f051f11e2 - std::panicking::begin_panic_handler::{{closure}}::h6e933e94ded3024f
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:657:13
  13:     0x7f9f051ef446 - std::sys_common::backtrace::__rust_end_short_backtrace::h4efef00f0bd07d56
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/sys_common/backtrace.rs:171:18
  14:     0x7f9f051f0f40 - rust_begin_unwind
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:645:5
  15:     0x7f9f05220d75 - core::panicking::panic_fmt::h3f2350d70561f7cb
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/panicking.rs:72:14
  16:     0x7f9f05220b23 - core::panicking::panic_display::hd10cbfc2d262f172
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/panicking.rs:196:5
  17:     0x7f9f05220b23 - core::panicking::panic_str::hbed90c8c1c7b42fd
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/panicking.rs:171:5
  18:     0x7f9f05220b23 - core::option::expect_failed::h814960b2c8b9ead7
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/option.rs:1980:5
  19:     0x7f9f0463af4b - cargo[96fc1eea4767ee95]::core::compiler::unit_dependencies::build_unit_dependencies
  20:     0x7f9f04737cca - cargo[96fc1eea4767ee95]::ops::cargo_compile::create_bcx
  21:     0x7f9f04735d7c - cargo[96fc1eea4767ee95]::ops::cargo_compile::compile_ws
  22:     0x7f9f04735c3b - cargo[96fc1eea4767ee95]::ops::cargo_compile::compile
  23:     0x7f9f0412be2d - cargo[ca2d47bc6fe3a914]::commands::build::exec
  24:     0x7f9f041080f0 - cargo[ca2d47bc6fe3a914]::cli::main
  25:     0x7f9f041447aa - cargo[ca2d47bc6fe3a914]::main
  26:     0x7f9f040c9e63 - std[59bbd359e4a0812d]::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  27:     0x7f9f040d83f9 - std[59bbd359e4a0812d]::rt::lang_start::<()>::{closure#0}
  28:     0x7f9f051dd661 - core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once::h3737ca496f48ba0e
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/core/src/ops/function.rs:284:13
  29:     0x7f9f051dd661 - std::panicking::try::do_call::h0791b1268c7577e7
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:552:40
  30:     0x7f9f051dd661 - std::panicking::try::h01cc49c61c00b2c9
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:516:19
  31:     0x7f9f051dd661 - std::panic::catch_unwind::hada5d8a26ee4d1dc
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panic.rs:142:14
  32:     0x7f9f051dd661 - std::rt::lang_start_internal::{{closure}}::ha2b321c2944045e7
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/rt.rs:148:48
  33:     0x7f9f051dd661 - std::panicking::try::do_call::h9b22a884fda2d38f
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:552:40
  34:     0x7f9f051dd661 - std::panicking::try::h5e537adef10f5106
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panicking.rs:516:19
  35:     0x7f9f051dd661 - std::panic::catch_unwind::h6250cfd4a61ec9d4
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/panic.rs:142:14
  36:     0x7f9f051dd661 - std::rt::lang_start_internal::h892a9bd9313ca5d1
                               at /rustc/e51e98dde6a60637b6a71b8105245b629ac3fe77/library/std/src/rt.rs:148:20
  37:     0x7f9f0414aa35 - main
  38:     0x7f9f036f9d90 - __libc_start_call_main
                               at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
  39:     0x7f9f036f9e40 - __libc_start_main_impl
                               at ./csu/../csu/libc-start.c:392:3
  40:     0x7f9f040aa029 - <unknown>
  41:                0x0 - <unknown>

@Qix-
Copy link

Qix- commented Feb 7, 2024

Did some more investigating.

The problem comes down to the following.

-Zbuild-std requires --target. --target is used to populate std_roots here:

std_roots: &HashMap<CompileKind, Vec<Unit>>,

The failure comes when the given unit.kind is not present in std_roots. I added some debug statements and found that unit.kind is populated with an absolute path to the target JSON file. However, since --target is mandatory in -Zbuild-std, the argument passed to --target (e.g. x86_64-unknown-linux-gnu) is the only one populated in std_roots. Thus, the index fails.

The debug stacktrace with the print statements is as follows (and is better than the one in my previous comment):

unit: Unit {
    pkg: Package {
        id: PackageId {
            name: "arch-x86",
            version: "0.1.0",
            source: "/src/qix-/cargo-crash-repro-2023-3-28/arch-x86",
        },
        ..: "..",
    },
    target: TargetInner {
        name: "arch-x86",
        doc: true,
        ..: with_path("/src/qix-/cargo-crash-repro-2023-3-28/arch-x86/src/main.rs", Edition2015),
    },
    profile: Profile {
        ..: default_dev(),
    },
    kind: Target(
        CompileTarget {
            name: "x86_64-unknown-linux-gnu",
        },
    ),
    mode: Build,
    features: [],
    artifact: false,
    artifact_target_for_features: None,
    is_std: false,
    dep_hash: 0,
}
roots: {
    Target(
        CompileTarget {
            name: "x86_64-unknown-linux-gnu",
        },
    ): [
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "core",
                    version: "0.0.0",
                    source: "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                benched: false,
                ..: lib_target("core", ["lib"], "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/lib.rs", Edition2021),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "compiler_builtins",
                    version: "0.1.105",
                    source: "registry `crates-io`",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                ..: lib_target("compiler_builtins", ["lib"], "/home/anonymous/.cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.105/src/lib.rs", Edition2018),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [
                "compiler-builtins",
                "core",
                "default",
                "mem",
                "rustc-dep-of-std",
            ],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
    ],
}
unit: Unit {
    pkg: Package {
        id: PackageId {
            name: "kernel",
            version: "0.1.0",
            source: "/src/qix-/cargo-crash-repro-2023-3-28/kernel",
        },
        ..: "..",
    },
    target: TargetInner {
        ..: lib_target("kernel", ["lib"], "/src/qix-/cargo-crash-repro-2023-3-28/kernel/src/lib.rs", Edition2015),
    },
    profile: Profile {
        ..: default_dev(),
    },
    kind: Target(
        CompileTarget {
            name: "x86_64-unknown-linux-gnu",
        },
    ),
    mode: Build,
    features: [],
    artifact: false,
    artifact_target_for_features: None,
    is_std: false,
    dep_hash: 0,
}
roots: {
    Target(
        CompileTarget {
            name: "x86_64-unknown-linux-gnu",
        },
    ): [
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "core",
                    version: "0.0.0",
                    source: "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                benched: false,
                ..: lib_target("core", ["lib"], "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/lib.rs", Edition2021),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "compiler_builtins",
                    version: "0.1.105",
                    source: "registry `crates-io`",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                ..: lib_target("compiler_builtins", ["lib"], "/home/anonymous/.cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.105/src/lib.rs", Edition2018),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [
                "compiler-builtins",
                "core",
                "default",
                "mem",
                "rustc-dep-of-std",
            ],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
    ],
}
unit: Unit {
    pkg: Package {
        id: PackageId {
            name: "kernel",
            version: "0.1.0",
            source: "/src/qix-/cargo-crash-repro-2023-3-28/kernel",
        },
        ..: "..",
    },
    target: TargetInner {
        ..: lib_target("kernel", ["lib"], "/src/qix-/cargo-crash-repro-2023-3-28/kernel/src/lib.rs", Edition2015),
    },
    profile: Profile {
        ..: default_dev(),
    },
    kind: Target(
        CompileTarget {
            name: "/src/qix-/cargo-crash-repro-2023-3-28/triple/x86_64.json",
        },
    ),
    mode: Build,
    features: [],
    artifact: false,
    artifact_target_for_features: None,
    is_std: false,
    dep_hash: 0,
}
roots: {
    Target(
        CompileTarget {
            name: "x86_64-unknown-linux-gnu",
        },
    ): [
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "core",
                    version: "0.0.0",
                    source: "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                benched: false,
                ..: lib_target("core", ["lib"], "/home/anonymous/.rustup/toolchains/nightly-2024-02-06-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/lib.rs", Edition2021),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
        Unit {
            pkg: Package {
                id: PackageId {
                    name: "compiler_builtins",
                    version: "0.1.105",
                    source: "registry `crates-io`",
                },
                ..: "..",
            },
            target: TargetInner {
                tested: false,
                ..: lib_target("compiler_builtins", ["lib"], "/home/anonymous/.cargo/registry/src/index.crates.io-6f17d22bba15001f/compiler_builtins-0.1.105/src/lib.rs", Edition2018),
            },
            profile: Profile {
                incremental: false,
                ..: default_dev(),
            },
            kind: Target(
                CompileTarget {
                    name: "x86_64-unknown-linux-gnu",
                },
            ),
            mode: Build,
            features: [
                "compiler-builtins",
                "core",
                "default",
                "mem",
                "rustc-dep-of-std",
            ],
            artifact: false,
            artifact_target_for_features: None,
            is_std: true,
            dep_hash: 0,
        },
    ],
}
thread 'main' panicked at src/cargo/core/compiler/unit_dependencies.rs:179:34:
no entry found for key
stack backtrace:
   0: rust_begin_unwind
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/std/src/panicking.rs:647:5
   1: core::panicking::panic_fmt
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/panicking.rs:72:14
   2: core::panicking::panic_display
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/panicking.rs:196:5
   3: core::panicking::panic_str
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/panicking.rs:171:5
   4: core::option::expect_failed
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/option.rs:1988:5
   5: core::option::Option<T>::expect
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/option.rs:894:21
   6: <std::collections::hash::map::HashMap<K,V,S> as core::ops::index::Index<&Q>>::index
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/std/src/collections/hash/map.rs:1341:23
   7: cargo::core::compiler::unit_dependencies::attach_std_deps
             at /src/rust-lang/cargo/src/cargo/core/compiler/unit_dependencies.rs:179:34
   8: cargo::core::compiler::unit_dependencies::build_unit_dependencies
             at /src/rust-lang/cargo/src/cargo/core/compiler/unit_dependencies.rs:133:9
   9: cargo::ops::cargo_compile::create_bcx
             at /src/rust-lang/cargo/src/cargo/ops/cargo_compile/mod.rs:406:26
  10: cargo::ops::cargo_compile::compile_ws
             at /src/rust-lang/cargo/src/cargo/ops/cargo_compile/mod.rs:151:15
  11: cargo::ops::cargo_compile::compile_with_exec
             at /src/rust-lang/cargo/src/cargo/ops/cargo_compile/mod.rs:141:5
  12: cargo::ops::cargo_compile::compile
             at /src/rust-lang/cargo/src/cargo/ops/cargo_compile/mod.rs:128:5
  13: cargo::commands::build::exec
             at /src/rust-lang/cargo/src/bin/cargo/commands/build.rs:68:5
  14: cargo::cli::Exec::exec
             at /src/rust-lang/cargo/src/bin/cargo/cli.rs:448:36
  15: cargo::cli::main
             at /src/rust-lang/cargo/src/bin/cargo/cli.rs:186:5
  16: cargo::main
             at /src/rust-lang/cargo/src/bin/cargo/main.rs:29:9
  17: core::ops::function::FnOnce::call_once
             at /rustc/256b6fb19a2c018eaad4806d2369d1f6a71fc6ec/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

So this new feature isn't properly setting up the std_roots, or disabling them - even when all targets are #![no_std]. I'm not sure if this is also used to attach core or not, but clearly something isn't getting populated correctly.

This is as far as I've gotten, but in the chance I don't get any further I wanted to post this here.

@Ekleog if you have any idea on how this could be fixed and want to give me some pointers (and don't have time yourself) I'm happy to take a whack at a fix. I'm pretty green to the Cargo codebase.


EDIT: I'm pretty sure this feature needs a LOT more attention from someone who knows Cargo intimately as there is a lot happening that doesn't seem right when it comes to building a single crate multiple times for different targets during a single Cargo run. See #11898 for more.

EDIT2: Looks like there's a lot of different work being done surrounding this exact issue, none of which is being tracked together. Would be nice to have some sort of clarity what the Cargo team wants (maybe there already is, but I can't find a single source for it) when it comes to build-std's future and how it's supposed to work across targets. Sorry for the noise, it's just really hard to track what work has been done and where progress has been made.

@mousetail
Copy link

mousetail commented May 30, 2024

In addition to forced-target it would be helpful to have a default-target. I generally want to some of the binaries in a workspace to be built for WASM and others for native but do want to keep the option open for people to override and build with some alternate target.

My main goal is not to have to type --target wasm32-unknown-unknown every time, while still allowing compiling for wasm32-unknown-wasi or wasm32-unknown-emscripten. I can add a config file to .cargo but it will apply to all binaries in a workspace.

@Ekleog
Copy link
Contributor Author

Ekleog commented May 31, 2024

@mousetail There already is a default-target parameter. This being said, the use case you depicted does sound like forced-target would be the better match for you, this way if people want to cross-compile to other platforms they can pass --target aarch64-unknown-linux-gnu but still have the WASM blobs be built as WASM.

Unfortunately both default-target and forced-target have several known bugs, and I don't have any time to look into it, plus this was my first contribution to the cargo codebase so I'm not that proficient with the other features either and fixing them (or even understanding them) would require much more time than I can currently allocate to it :/


@Qix- Sorry to have missed your tag! About tracking the various issues and work being done on it, theoretically it should all be reachable from the top post of this issue, which is editable by any maintainer of the cargo repo. For #11898, it sounds close to #9451 listed in the top comment. Do you want me to add an unresolved issue for it in the top comment, or should it be tracked alongside #9451? And in the first case, can you edit the title of #11898 so that it's obvious how it's different?

As for the fix to the issue you mentioned, unfortunately I don't have much more to add to what you already wrote. IMO the fix would be to change -Zbuild-std to instead be eg. -Zbuild-std=wasm32-unknown-unknown, and to use the pre-built libstd for any target that is not listed in -Zbuild-std. But this discussion should likely happen in #9451 (or #11898), as many people are subscribed to this thread :)

@amnestria
Copy link

I've encountered an issue relating to this feature. #14833

@carloskiki
Copy link

I've been thinking about this feature lately, and I've come to the conclusion that force-target is wrong. I can't find where I've seen this proposition, but I think it would better if one could specify a set of target to include or exclude.

For example:

[target.'cfg(target_arch = "arm")']
allowed = true

Here, only targets with cfg(target_arch = "arm") are allowed to build.

[target.'cfg(target_family = "wasm")']
allowed = false

Here, wasm targets are forbidden.

I don't know the semantics yet, but I think that generally one should only "allow a subset of targets" or "disallow a subset of targets" (not both). There are a multitude of ways we could solve this like failing at compile time, having a unification algorithm, or having another field like allow-all-targets-by-default (terrible name).

A simpler, but more tedious option could be something like this:

disallowed-targets = [
  "wasm32-unknown-emscripten",
  "wasm32-unknown-unknown",
  "wasm32-wasi",
  "wasm32-wasip1",
  "wasm32-wasip1-threads",
  "wasm32-wasip2"
]
allowed-targets = [
  "thumbv8m.base-none-eabi",
  "thumbv8m.main-none-eabi",
  "thumbv8m.main-none-eabihf"
]

The exact names/semantics need to be bikeshedded, but I think it is obvious we need to handle the case where both fields are set. We could do something like the include and exclude entries.

I don't know how any of this should work with custom targets.


Although I prefer the first solution, I think both are better than what we currently have. The functionality of setting a default-target is useful, because specifying targets to build by default in config.toml is a pain (especially in workspaces, because of the recursive search nature). But also now we solve the problem of trying to compile a crate with a dependency that does not support the user's selected target. This gives cryptic errors and is very annoying for new users (this is especially relevant for wasm support). This is also great for hal crates of specific embedded targets, which usually only need to support one or a few targets.

@jwnrt
Copy link

jwnrt commented Dec 3, 2024

I have been thinking about this too and also thought force-target didn't feel right. I was wondering if the allow/deny was more appropriate in a build script which can read TARGET and CARGO_CFG_TARGET_* environment variables and give a nice error message.


My preferred semantics for default-target would be that a crate is compiled for its default-target when the workspace in general is being built (even when --target is given) but the --target flag overrules it when that crate is the "primary" package (i.e. the CWD or the --package flag is used).

The matrix for what a crate would build against would be:

Building workspace Building crate
no --target given default-target default-target
--target given default-target --target

It would also be really useful to set default-target to "host" for situations where one crate in a workspace is executed locally while the others may be retargeted. The most obvious example is where you have an xtask crate that always runs on the host alongside crates building for embedded targets.

I think that would be a change in rustc to recognise --target "host" the same as no flag being set.

@carloskiki
Copy link

Actually, #6179 looks a lot like what we are both talking about. epage's recent comment has a lot of good considerations/inputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-tracking-issue Category: A tracking issue for something unstable. S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. S-needs-mentor Status: Issue or feature is accepted, but needs a team member to commit to helping and reviewing. S-waiting-on-feedback Status: An implemented feature is waiting on community feedback for bugs or design concerns. Z-per-package-target Nightly: per-package-target
Projects
Status: Unstable, baking
Development

No branches or pull requests