`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key #10593

bstrie · 2022-04-22T20:10:28Z

Problem

For convenience I have also uploaded the failing code here: https://github.com/bstrie/bindeperror4

For the following code:

# Cargo.toml
[package]
name = "mycrate"
version = "0.0.0"
edition = "2021"

[dependencies]
mybindep = { path = "mybindep", artifact = "bin", target = "x86_64-unknown-linux-gnu" }

// src/main.rs
fn main() {
    env!("CARGO_BIN_FILE_MYBINDEP");
}

# mybindep/Cargo.toml
[package]
name = "mybindep"
version = "0.0.0"
edition = "2021"

// mybindep/src/main.rs
fn main() {}

Attempting to use cargo tree produces the following:

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "mybindep", version: "0.0.0", source: "/home/ben/code/scrap/bindeperror4/mybindep" } NormalOrDevOrArtifactTarget(None)', src/tools/cargo/src/cargo/core/resolver/features.rs:318:14

Expand for full `cargo tree` backtrace.

thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "mybindep", version: "0.0.0", source: "/home/ben/code/scrap/bindeperror4/mybindep" } NormalOrDevOrArtifactTarget(None)

Stack backtrace:
   0: <anyhow::Error>::msg::<alloc::string::String>
   1: <cargo::core::resolver::features::ResolvedFeatures>::activated_features_int
   2: cargo::ops::tree::graph::add_pkg
   3: cargo::ops::tree::graph::add_pkg
   4: cargo::ops::tree::graph::build
   5: cargo::ops::tree::build_and_print
   6: cargo::commands::tree::exec
   7: cargo::cli::main
   8: cargo::main
   9: std::sys_common::backtrace::__rust_begin_short_backtrace::<fn(), ()>
  10: std::rt::lang_start::<()>::{closure#0}
  11: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/ops/function.rs:280:13
  12: std::panicking::try::do_call
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:492:40
  13: std::panicking::try
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:456:19
  14: std::panic::catch_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panic.rs:137:14
  15: std::rt::lang_start_internal::{{closure}}
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/rt.rs:128:48
  16: std::panicking::try::do_call
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:492:40
  17: std::panicking::try
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:456:19
  18: std::panic::catch_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panic.rs:137:14
  19: std::rt::lang_start_internal
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/rt.rs:128:20
  20: main
  21: <unknown>
  22: __libc_start_main
  23: <unknown>', src/tools/cargo/src/cargo/core/resolver/features.rs:318:14
stack backtrace:
   0: rust_begin_unwind
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/std/src/panicking.rs:584:5
   1: core::panicking::panic_fmt
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/panicking.rs:142:14
   2: core::result::unwrap_failed
             at /rustc/de1bc0008be096cf7ed67b93402250d3b3e480d0/library/core/src/result.rs:1785:5
   3: cargo::ops::tree::graph::add_pkg
   4: cargo::ops::tree::graph::add_pkg
   5: cargo::ops::tree::graph::build
   6: cargo::ops::tree::build_and_print
   7: cargo::commands::tree::exec
   8: cargo::cli::main
   9: cargo::main

Curiously the panic message is very similar to the panic reported in #10431 . Since that issue has already been fixed, it's possible that the solution for this issue may be similar to the solution for that one.

Note that the following diff causes cargo tree to run as expected:

- mybindep = { path = "mybindep", artifact = "bin", target = "x86_64-unknown-linux-gnu" }
+ mybindep = { path = "mybindep", artifact = "bin" }

So the presence of the explicit target key is crucial here. This is similar to the circumstances of #10526 and #10525 , so it further suggests something interesting about how the target key is being handled. It may be that fixing one of these will fix all of these.

cc @Byron

Version

cargo 1.62.0-nightly (edffc4a 2022-04-19)
release: 1.62.0-nightly
commit-hash: edffc4ada3d77799e5a04eeafd9b2f843d29fc23
commit-date: 2022-04-19
host: x86_64-unknown-linux-gnu
libgit2: 1.4.2 (sys:0.14.2 vendored)
libcurl: 7.80.0-DEV (sys:0.4.51+curl-7.80.0 vendored ssl:OpenSSL/1.1.1m)
os: Pop!_OS 21.10 (impish) [64-bit]

The text was updated successfully, but these errors were encountered:

bstrie · 2022-08-05T15:01:08Z

For now, note that the original cargo-tree tool still works and doesn't have this problem: https://github.com/sfackler/cargo-tree

weihanglo · 2022-11-16T22:29:30Z

This issue can be fixed by duplicating the logic of resolver to determine the FeatureFor artifact deps. The following patch demonstrates it:

Click to view the patch

diff --git a/src/cargo/ops/tree/graph.rs b/src/cargo/ops/tree/graph.rs
index 20a9ca0b6..c975e6fcc 100644
--- a/src/cargo/ops/tree/graph.rs
+++ b/src/cargo/ops/tree/graph.rs
@@ -387,10 +387,38 @@ fn add_pkg(
         let dep_pkg = graph.package_map[&dep_id];
 
         for dep in deps {
-            let dep_features_for = if dep.is_build() || dep_pkg.proc_macro() {
-                FeaturesFor::HostDep
-            } else {
-                features_for
+            // This is somehow similar to
+            // <https://github.com/rust-lang/cargo/blob/5181f995d08e72da457bf8a2e6c9c8d0bba09088/src/cargo/core/resolver/features.rs#L800-L852>
+            // without the consideration of `dep = { …, lib = true }`
+            //
+            // TODO: decide how to display artifact deps within cargo-tree,
+            // including handling artifact deps with `{ …, lib = true }`
+            let dep_features_for = match dep
+                .artifact()
+                .and_then(|a| a.target())
+                .and_then(|t| t.to_resolved_compile_target(requested_kind))
+            {
+                // Dependency has a `{ …, target = <triple> }`
+                Some(target) => FeaturesFor::ArtifactDep(target),
+                // Get the information of the dependent crate from `features_for`.
+                // If a dependent crate is
+                //
+                // * specified as an artifact dep with a `target`, or
+                // * a host dep,
+                //
+                // its transitive deps, including build-deps, need to be built on that target.
+                None if features_for != FeaturesFor::default() => features_for,
+                // Dependent crate is a normal dep, then back to old rules:
+                //
+                // * normal deps, dev-deps -> inherited target
+                // * build-deps -> host
+                None => {
+                    if dep.is_build() || dep_pkg.proc_macro() {
+                        FeaturesFor::HostDep
+                    } else {
+                        features_for
+                    }
+                }
             };
             let dep_index = add_pkg(
                 graph,

However, I found another panic in cargo tree when it meets all the followings conditions at once:

a dependency of an artifact dependency is platform specified
artfiact dep itself is { target = <specified> } with the same platform as its own dependency
the platform is not activated.

Here is a reproducer in Cargo's own test dialect:

Click to view the reproducer

#[cargo_test]
fn artifact_dep_target_specified() {
    if cross_compile::disabled() {
        return;
    }
    let alt = cross_compile::alternate();
    let p = project()
        .file(
            "Cargo.toml",
            &format!(
                r#"
                    [package]
                    name = "foo"
                    version = "0.1.0"

                    [dependencies]
                    bar = {{ path = "bar", artifact = "bin", target = "{alt}" }}
                "#,
            ),
        )
        .file("src/lib.rs", "")
        .file(
            "bar/Cargo.toml",
            &format!(
                r#"
                    [package]
                    name = "bar"
                    version = "0.1.0"

                    [target.{alt}.dependencies]
                    baz = {{ path = "../baz" }}
                "#,
            ),
        )
        .file("bar/src/lib.rs", "")
        .file(
            "baz/Cargo.toml",
            r#"
                [package]
                name = "baz"
                version = "0.1.0"
            "#,
        )
        .file("baz/src/lib.rs", "")
        .build();

    p.cargo("tree -Z bindeps")
        .masquerade_as_nightly_cargo(&["bindeps"])
        .with_stdout("") 
        .run();
    // thread 'main' panicked at 'no entry found for key', src/cargo/ops/tree/graph.rs:387:23
}

Personally, I don't think this should panic, as cargo tree is used only for displaying dependency information. However, I won't fix it and instead pause here until a broader issue gets resolved: How does Cargo display artifact dependencies in cargo tree properly? Some questions might be:

Should we annotate artifact dependency nodes with a special (artifact deps) suffix?
Should we show the information about which target platform an artifact dependency will be built on?
How do we want to display the node of an artifact dependency with { …, lib = true }?

I haven't thought about it much. Will be back when I get more time. If anyone comes up with great ideas to show artifact dependencies in cargo tree, I am all ears!

bstrie · 2022-12-07T16:13:48Z

@weihanglo , my opinions for each question:

Should we annotate artifact dependency nodes with a special (artifact deps) suffix?

There are multiple possible values for the artifact field, so I wouldn't mind having a (bin) or (staticlib) or (cdylib) suffix.

Should we show the information about which target platform an artifact dependency will be built on?

No, I think that's a detail that can be easily answered by checking Cargo.toml. But if you did decide to display this information, I would include it as part of the above annotation, e.g. (wasm32-wasi bin) or (x86_64-unknown-linux-musl staticlib).

How do we want to display the node of an artifact dependency with { …, lib = true }?

I think it should show up multiple times in the tree graph. The lib node should show up as an entirely normal dependency, whereas the artifact node should be annotated as mentioned above.

bstrie · 2022-12-15T16:52:06Z

To elaborate on the above, I think it would be nice if users could think of this:

bar = { version = "*", artifact = "bin", lib = true }

...as though it were this:

bar_1 = { package = "bar", version = "*" }
bar_2 = { package = "bar", version = "*", artifact = "bin" }

...with the only difference that using the lib = "true" form doesn't force you to use renames via package = to avoid a collision. And since it's natural that the second example would have two lines in the cargo tree output, it seems natural that the first example should as well, and the (bin) bit is a convenient disambiguator.

bstrie · 2022-12-15T16:55:45Z

(And to elaborate even further, I think that #10837 suggests that the lib and bin artifacts from the exact same line can actually have different dependencies as a result of per-target feature unification, which is something that we'd want to reflect in cargo tree's output.)

With the default resolver `cargo tree` panics with: ``` thread 'main' panicked at 'activated_features for invalid package: features did not find PackageId { name: "actor-echo-module", version: "0.1.0", source: "/home/rvolosatovs/src/github.com/wasmCloud/wasmCloud/tests/actors/echo-module" } NormalOrDev', src/tools/cargo/src/cargo/core/resolver/features.rs:321:14 ``` Tracking issue: rust-lang/cargo#10593 Signed-off-by: Roman Volosatovs <rvolosatovs@riseup.net>

benwis · 2023-08-07T00:57:24Z

It looks like I am also experiencing this. With resolver=1 it gets past the panic, but tries to build all my optional dependencies. With the default resolver it panics with

❱ cargo -Zbindeps run
thread 'main' panicked at src/cargo/core/resolver/features.rs:322:14:
activated_features for invalid package: features did not find PackageId { name: "libc", version: "0.2.147", source: "registry `crates-io`" } ArtifactDep(CompileTarget { name: "wasm32-unknown-unknown" })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

As others have stated, it seems to happen when specifying a wasm32-unknown-unknown target like this in both a rust script and a regular lib crate

//! cargo
//! [dependencies]
//! benwis_leptos = { artifact = "cdylib", path="../",features=["hydrate"], target = "wasm32-unknown-unknown" }
//!

I'll have to go back to std::process::Command for now.

bstrie · 2023-08-07T16:39:32Z

@benwis If this is the only problem that you're having, then you can also consider using sfackler's standalone cargo-tree tool, as linked above.

benwis · 2023-08-07T19:55:33Z

Unless I'm misunderstanding, this also breaks normal compilation with cargo build, and I'm not sure replacing cargo tree would fix ti

…

On Mon, Aug 7, 2023, at 9:39 AM, bstrie wrote: @benwis <https://github.com/benwis> If this is the only problem that you're having, then you can also consider using sfackler's standalone cargo-tree tool, as linked above. — Reply to this email directly, view it on GitHub <#10593 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVBTCI3YTPQDX4EMC7DKU3XUEK47ANCNFSM5UDISIAQ>. You are receiving this because you were mentioned.Message ID: ***@***.***>

hawkw · 2023-08-08T20:24:36Z

I'm also hitting an issue similar to what @benwis described: the panic occurs any time I attempt to build a crate which has a bindep with a different target. The panic occurs with cargo build, not cargo tree.

Using resolver = "1" isn't an option, as the bindep crate is #![no_std] and has shared dependencies with the crate that depends on the bindep, so resolver = "1"'s feature unification results in those dependencies' std feature flags, breaking my build when building the bindep crate.

I'm happy to provide an additional repro, but I believe @bstrie's reproduction looks like it ought to be sufficient to reproduce the issue with cargo build, as well as with cargo tree?

bstrie · 2023-08-10T19:48:07Z

@benwis When I first filed this issue the bug only exhibited during cargo tree, not cargo build (at least, as far as I remember... I've filed plenty of bindeps issues that related to cargo build, so I feel like I would have mentioned that before mentioning cargo tree). If cargo build causes this same error to happen, that sounds like something must have changed since I filed this issue. @weihanglo has been doing lots of work resolving many of the other bindeps issues that I've filed, perhaps they can chime in with whether or not any recent changes spring to mind?

hawkw · 2023-08-10T19:54:03Z

@bstrie As an update, I've found that the issue I observed with cargo build actually only occurs when the bindep crate is in a separate workspace from the crate that has a bindep on it. When both crates are within the same workspace, cargo build completes without error.

rvolosatovs · 2023-08-10T21:42:52Z

@hawkw just in case it helps, we've moved away from (unstable) bindeps feature and now build our Wasm binaries in a build.rs script largely replicating the same functionality. It only works for crates available locally.

This way we are able to build Wasm binaries as part of the normal cargo build/cargo test, while staying on stable Rust and not breaking things like cargo tree, which, in turn, means also not breaking Dependabot (which is otherwise broken in most cases when using bindeps feature).
Do note that the approach outlined above require the crates-to-be-built to be outside the workspace building it, because otherwise an attempt to build those would cause a deadlock in cargo (due to cargo on-disk per-workspace lock)

weihanglo · 2023-08-10T21:55:32Z

Sorry I haven't got time taking another deep look for -Zbindeps recently. Things continue popping up and focus were shifted.

#12358 is the issue that cargo build panics. It may or may not share the same code path.

bstrie added the C-bug Category: bug label Apr 22, 2022

ehuss added the Z-bindeps Nightly: binary artifact dependencies label Apr 22, 2022

Byron mentioned this issue Apr 22, 2022

Implement 'multidep' (RFC-3176) #10061

Open

18 tasks

weihanglo mentioned this issue Oct 4, 2022

Spurious cargo build error from a combination of binary dependencies, the lib key, the target key, and differing crate features #10837

Closed

weihanglo added the A-features2 Area: issues specifically related to the v2 feature resolver label Oct 4, 2022

weihanglo mentioned this issue Oct 31, 2022

'cargo tree -i <package>' is empty for windows-only dependency #11315

Closed

weihanglo mentioned this issue Nov 21, 2022

WIP: try making cargo tree's Graph from a UnitGraph #11379

Closed

35 tasks

rvolosatovs mentioned this issue Mar 28, 2023

build: set resolver to 1 wasmCloud/wasmCloud#296

Closed

10 tasks

weihanglo mentioned this issue Jul 13, 2023

Cargo panic in ResolvedFeatures::activated_features_int with unstable bindeps feature #12358

Open

weihanglo added the S-needs-mentor Status: Issue or feature is accepted, but needs a team member to commit to helping and reviewing. label Jul 13, 2023

rukai mentioned this issue Dec 26, 2023

Fix panic when running cargo tree on a package with a cross compiled bindep #13207

Open

phil-opp mentioned this issue Mar 4, 2024

Edition 3 post-2 build-dependencies problem phil-opp/blog_os#1303

Closed

OliverKillane mentioned this issue Apr 9, 2024

Panic on use of libraries with proc-macro examples #13726

Closed

JonasKruckenberg mentioned this issue Sep 1, 2024

Panic when combining -Zbuild-std with artifact dependencies #10444

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key #10593

`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key #10593

bstrie commented Apr 22, 2022 •

edited

Loading

bstrie commented Aug 5, 2022

weihanglo commented Nov 16, 2022

bstrie commented Dec 7, 2022 •

edited

Loading

bstrie commented Dec 15, 2022 •

edited

Loading

bstrie commented Dec 15, 2022 •

edited

Loading

benwis commented Aug 7, 2023 •

edited

Loading

bstrie commented Aug 7, 2023

benwis commented Aug 7, 2023 via email

hawkw commented Aug 8, 2023

bstrie commented Aug 10, 2023

hawkw commented Aug 10, 2023

rvolosatovs commented Aug 10, 2023 •

edited

Loading

weihanglo commented Aug 10, 2023

cargo tree panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit target key #10593

cargo tree panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit target key #10593

Comments

bstrie commented Apr 22, 2022 • edited Loading

Problem

Version

bstrie commented Aug 5, 2022

weihanglo commented Nov 16, 2022

bstrie commented Dec 7, 2022 • edited Loading

bstrie commented Dec 15, 2022 • edited Loading

bstrie commented Dec 15, 2022 • edited Loading

benwis commented Aug 7, 2023 • edited Loading

bstrie commented Aug 7, 2023

benwis commented Aug 7, 2023 via email

hawkw commented Aug 8, 2023

bstrie commented Aug 10, 2023

hawkw commented Aug 10, 2023

rvolosatovs commented Aug 10, 2023 • edited Loading

weihanglo commented Aug 10, 2023

`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key #10593

`cargo tree` panics with "activated_features for invalid package: features did not find PackageId" when a binary dependency is specified with an explicit `target` key #10593

bstrie commented Apr 22, 2022 •

edited

Loading

bstrie commented Dec 7, 2022 •

edited

Loading

bstrie commented Dec 15, 2022 •

edited

Loading

bstrie commented Dec 15, 2022 •

edited

Loading

benwis commented Aug 7, 2023 •

edited

Loading

rvolosatovs commented Aug 10, 2023 •

edited

Loading