fix(toml): Warn, rather than fail publish, if a target is excluded

[epage]

What does this PR try to resolve?

We have a couple of problems with publishing

• Inconsistent errors: if a target that package doesn't verify is missing path, it will error, while one with path won't, see cargo publish has bad error message when explicit [[bench]] isn't in package.include #13456
• Users may want to exclude targets and their choices are
  • Go ahead and include them. I originally excluded my examples before doc-scraping was a think. The problem was if I had to set required-features, I then could no longer exclude them
  • Muck with Cargo.toml during publish and pass --allow-dirty

This fixes both by auto-stripping targets on publish. We will warn the user that we did so.

This is a mostly-one-way door on behavior because we are turning an error case into a warning.
For the most part, I think this is the right thing to do.
My biggest regret is that the warning is only during package/publish as it will be too late to act on it and people who want to know will want to know when the problem is introduced.
The error is also very late in the process but at least its before a non-reversible action has been taken.
Dry-run and yank help.

Fixes #13456
Fixes #5806

How should we test and review this PR?

Tests are added in the first commit and you can then follow the commits to see how the test output evolved.

The biggest risk factors for this change are

• If the target-stripping logic mis-identifies a path as excluded because of innocuous path differences (e.g. case)
• Setting a minimum MSRV for published packages: auto* were added in 1.27 (Introduce autoXXX keys for target auto-discovery #5335) but were insta-stable. autobins = false did nothing until 1.32 (Make autodiscovery disable inferred targets #6329). I have not checked to see how this behaves pre-1.32 or pre-1.27. Since my memory of that error is vague, I believe it will either do a redundant discovery or it will implicitly skip discovery

Resolved risks

• fix(package): Normalize paths in Cargo.toml #13729 ensured our generated target paths don't have \ in them
• fix(package): Normalize paths in Cargo.toml #13729 ensures the paths are normalize so the list of packaged paths

For case-insensitive filesystems, I added tests to show the original behavior (works locally but will fail when depended on from a case-sensitive filesystem) and tracked how that changed with this PR (on publish warn that those targets are stripped). We could try to normalize the case but it will also follow symlinks and is likely indicative of larger casing problems that the user had. Weighing how broken things are now , it didn't seem changing behavior on this would be too big of a deal.

We should do a Call for Testing when this hits nightly to have people to cargo package and look for targets exclusion warnings that don't make sense.

Additional information

This builds on #13701 and the work before it.

By enumerating all targets in Cargo.toml, it makes it so rust-lang/crates.io#5882 and rust-lang/crates.io#814 can be implemented without any other filesystem interactions.

A follow up PR is need to make much of a difference in performance because we unconditionally walk the file system just in case autodiscover != Some(false) or a target is missing a path.

We cannot turn off auto-discovery of libs, so that will always be done for bin-only packages.

[rustbot]

r? @ehuss

rustbot has assigned @ehuss.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[epage]

Blocked on #13701. Posting now to get Windows testing.

[bors]

⌛ Testing commit 8cc2f39 with merge ba6fb40...

[bors]

☀️ Test successful - checks-actions
Approved by: weihanglo
Pushing ba6fb40 to master...

[glandium]

How is this related to #5806?

[epage]

If I understand correctly cargo vendor will vendor "published" Cargo.toml files. With #5806, you could drop a build.rs and it would be automatically picked up. After this change, it will be ignored and the only way to add a build.rs is to hand-edit the Cargo.toml which will result in a checksum error.

[glandium]

So, if I understand correctly, this only "fixes" #5806 for crates published to crates.io, but e.g. crates vendored from a git repo, will still have the problem.

[epage]

From my understanding cargo vendor effectively runs cargo package on every dependency, regardless of the source. This change affects cargo package.

I tried this out locally by running cargo vendor on https://github.com/crate-ci/cargo-release/ which has git dev-dependencies. The vendored versions of those git dev-dependencies had the standard "this is from cargo package" boilerplate at the top and they had all of the auto* fields set to false from this change.

So I believe this fixes it for git as well. Feel free to test it when this hits nightly and let us know if it works or not!

[glandium]

I tested 1.80.0-nightly (c987ad527 2024-05-01), and vendoring indeed adds the auto* and build fields (and more).

[ia0]

FYI: This PR might have introduced a "warning" regression: #14290.