Workspace dependencies getting rebuilt when child dependency versions differ

[jrobsonchase]

I have a project with a workspace top-level crate that encompasses a number of crates as path dependencies. Without code changes, I can run a build in one of the children, and then in the top crate, and back and forth, etc. and trigger rebuilds every time.

It appears to me that the cause of this is that two of the child crates depend on different versions of the same crate from crates.io, which for some reason is causing it to get rebuilt. My assumption for workspaces was that dependencies should be resolved and cached across all crates, even if there are different versions.

Minimal example project: https://github.com/Pursuit92/cargo_workspace_bug

The real project where I'm seeing the issue has the version discrepancy in the dependency of a dependency: https://github.com/catt-ha/catt-rs

Cargo version: cargo 0.13.0-nightly (19cfb67 2016-09-28)
Rustc version: rustc 1.14.0-nightly (86affcdf6 2016-09-28)

[jrobsonchase]

After a bit more investigating, it appears that the true culprit is a difference in the features in one of the dependencies.

Building child_one:

$ RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info cargo build
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for child_one v0.1.0 (file:///tmp/demo/child_one): new (mio v0.5.1) != old (mio v0.5.1)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for mio v0.5.1: new (miow v0.1.3) != old (miow v0.1.3)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for miow v0.1.3: new (net2 v0.2.26) != old (net2 v0.2.26)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for net2 v0.2.26: features have changed: None != Some(["default", "duration"])
   Compiling net2 v0.2.26
   Compiling miow v0.1.3
   Compiling mio v0.5.1
   Compiling child_one v0.1.0 (file:///tmp/demo/child_one)
    Finished debug [unoptimized + debuginfo] target(s) in 2.37 secs

Building root:

$ RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info cargo build
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for demo v0.1.0 (file:///tmp/demo): new (child_one v0.1.0 (file:///tmp/demo/child_one)) != old (child_one v0.1.0 (file:///tmp/demo/child_one))
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for child_one v0.1.0 (file:///tmp/demo/child_one): new (mio v0.5.1) != old (mio v0.5.1)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for mio v0.5.1: new (miow v0.1.3) != old (miow v0.1.3)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for miow v0.1.3: new (net2 v0.2.26) != old (net2 v0.2.26)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for net2 v0.2.26: features have changed: Some(["default", "duration"]) != None
   Compiling net2 v0.2.26
   Compiling mio v0.6.0
   Compiling miow v0.1.3
   Compiling mio v0.5.1
   Compiling child_one v0.1.0 (file:///tmp/demo/child_one)
   Compiling child_two v0.1.0 (file:///tmp/demo/child_two)
   Compiling demo v0.1.0 (file:///tmp/demo)
    Finished debug [unoptimized + debuginfo] target(s) in 3.15 secs

[alexcrichton]

Ah yeah differences in features would do it. #3102 would help here in that regard, but the other fix would be to ensure that no matter what crate is being compiled the activated feature set is the same.

[jrobsonchase]

Is there any workaround for right now? I've read a little about the [replace] section, but it doesn't look like it's suitable for overriding enabled features.

[alexcrichton]

The "workaround" for now is to just ensure that your top-level targets build the same set of features into each dependency (e.g. in this case activating the default dependency of net2)

[jrobsonchase]

Ah, I see. Didn't realize that I could add net2 as a dependency of my crate, muck with its features, and have that take precedence over the one that mio pulled in. Seems to be working! 👍

[cswindle]

We have also hit this with Iron and Hyper, where Iron defines a different feature set, so whenever hyper is used in a crate without Iron we also need to set the same feature set. This is a bit of a pain to do and would be better if either the union of the features required was produced, or if the library targets included the features used in the name.

[alexcrichton]

Ok, #3102 has landed, so I believe this is fixed. @Pursuit92 can you confirm?

[jrobsonchase]

Looks like it's still a problem - #3102 did indeed fix the recompilation of the transitive net2 dependency, but now the crates that depend on it are still getting rebuilt.

/tmp/cargo_workspace_bug/child_one
josh@tethys child_one $ RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info cargo build
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one): new (mio v0.5.1) != old (mio v0.5.1)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for mio v0.5.1: new (miow v0.1.3) != old (miow v0.1.3)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for miow v0.1.3: new (net2 v0.2.26) != old (net2 v0.2.26)
   Compiling miow v0.1.3
   Compiling mio v0.5.1
   Compiling child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one)
    Finished debug [unoptimized + debuginfo] target(s) in 1.44 secs

josh@tethys cargo_workspace_bug $ RUST_LOG=cargo::ops::cargo_rustc::fingerprint=info cargo build
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for demo v0.1.0 (file:///tmp/cargo_workspace_bug): new (child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one)) != old (child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one))
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one): new (mio v0.5.1) != old (mio v0.5.1)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for mio v0.5.1: new (miow v0.1.3) != old (miow v0.1.3)
INFO:cargo::ops::cargo_rustc::fingerprint: fingerprint error for miow v0.1.3: new (net2 v0.2.26) != old (net2 v0.2.26)
   Compiling miow v0.1.3
   Compiling mio v0.5.1
   Compiling child_one v0.1.0 (file:///tmp/cargo_workspace_bug/child_one)
   Compiling demo v0.1.0 (file:///tmp/cargo_workspace_bug)
    Finished debug [unoptimized + debuginfo] target(s) in 1.48 secs

My guess is that this is because the changes from #3102 need to cascade up the dependency chain. Cargo is only resolving one version of miow which depends on different versions of net2 in different contexts where it should resolve different versions of it for each dependency configuration.

Your other suggested fix (ensuring that the activated features are the same, regardless of where you build from) seems like it might be a cleaner and less build-directory-bloaty option. On the other hand, couldn't the single feature set approach introduce subtle bugs in dependencies if a differing activated feature set than they're expecting causes problems?

[alexcrichton]

Ah yes, right!

Once #3310 lands I'll work towards solving this issue as well.

[fluffysquirrels]

@Pursuit92 said:

Your other suggested fix (ensuring that the activated features are the same, regardless of where you build from) seems like it might be a cleaner and less build-directory-bloaty option. On the other hand, couldn't the single feature set approach introduce subtle bugs in dependencies if a differing activated feature set than they're expecting causes problems?

Unfortunately it seems like conflicting feature sets are going to be a problem either way.

I'd say the situation now is much more subtle than if the activated feature set was consistent across the workspace. Right now in my workspace cargo build --all, cargo build -p project1, cargo build -p project2 rebuild the same low-level dependency with different features and clobber each other. We chose to use workspaces to have consistent dependency versions across all our top-level binaries, and I'd say consistent feature sets is very much aligned with that goal.

+1 for consistent feature sets across the workspace.

[alexcrichton]

I think that #3923 is a duplicate of this issue, but I've written out what's going on there and in addition to #3151 (comment) above I think we've got the bases covered in terms of what's happening.

I'm goign to close this in favor of #3923 just to deduplicate things.