Custom registries are infectious to downstream dependents

[w4]

Problem

When depending on custom registries over Git, the consumer is currently expected to also have a reference to the same registry in their .cargo/config.toml.

Steps

1. crate-a containing:

   $ cat Cargo.toml
   [package]
   name = "crate-a"
   version = "0.1.0"
   edition = "2021"
   
   [dependencies]
   some-crate = { version = "0.1", registry = "some-registry" }
   
   $ cat .cargo/config.toml
   [registries]
   some-registry = { index = "ssh://registry.x.com/path/to/some-registry" }
   

2. crate-b containing:

   $ cat Cargo.toml
   [package]
   name = "crate-b"
   version = "0.1.0"
   edition = "2021"
   
   [dependencies]
   crate-a = { version = "0.1", git = "https://git.x.com/path/to/crate-a" }
   
   $ cat .cargo/config.toml
   cat: .cargo/config.toml: No such file or directory
   

3. Compiling crate-b gives the cryptic error:

   error: no matching package named `crate-a` found
   location searched: https://git.x.com/path/to/crate-a
   

4. Which can be fixed by adding

   [registries]
   some-registry = { index = "ssh://registry.x.com/path/to/some-registry" }
   

   to crate-b's .cargo/config.toml

Possible Solution(s)

Potentially a flag that can be set in the dependent's .cargo/config.toml to allow git dependencies to use arbitrary registries?

Notes

No response

Version

No response

[Eh2406]

this seems like an arbitrary limitation since the absolute URI to the registry is baked into the published crate's metadata.

I'm not exactly sure I'm following. The absolute URI is baked as part of the publish process. If you pull from a package from registry it will depend on other packages based only on their absolute URI. In your example you have not depended on a package in registry, rather you have have a git dependency on some source code. Cargo checks out the git project you have specified and has a path dependency on the resulting working tree. Nowhere in this process does it bake in any metadata.

[w4]

That's an interesting revelation - thanks, I didn't actually test transient dependencies while going through the registry.

Although, now that I'm testing with that knowledge, that brings up a surprising behaviour that the crate-a's registry = "some-registry" definition on the local dependency leaks into crate-b and requires crate-b to define a registry with the exact name some-registry rather than either:

a. ignoring it because it's going to use the path dependency anyway (see #12272), or
b. using the remote crate's .cargo/config.toml [registries] definition for the build of that crate

[Eh2406]

.cargo/config.toml are never read from dependencies. They are only read from the dir where cargo was invoked (and its parents). They are configuration for the build, not for the package. If you use --manifest-path you can find a different example of this #2930 Ferthermore, there is no guarantee that the maintainers of crate-a even checked in there config.toml. They may have put there [registries] section in ~/.cargo/config.toml for example.

[weihanglo]

Note that currently if you have both path and registry declared in your direct dependency and the registry config is missing, Cargo will error out and stop the build.

There are several scenario of them, and we may only need to change the behavior for git and path dependencies if we really want it.

• ✅ direct dependency -> transitive dependency with both path and registry: shouldn't exist on crates.io. No change needed.
• ✅ direct registry dependency -> transitive dependency with both path and registry: Cargo will stick to registry dependencies transitively. No change needed.
• 🤯 direct git dependency -> transitive dependency with both path and registry: Cargo will use path dependency from the same git repo of the direct git dependency. Currently it errors out as registry cannot be found. This seems to have chance to relax a bit.
• ❓ direct path dependency -> transitive dependency with both path and registry: Cargo will favor path dependencies for transitive path dependencies. It's a bit odd to relax the check for this case, since usually you have the control over your local path dependencies.

[w4]

[moved from #12807] Thanks @weihanglo. I do agree the inconsistency would preferably be avoided. But even now there's an inconsistency between how an unknown registry via direct/transitive dependencies surface - direct dependencies against an unknown registry give a clear error, transitive dependencies silently discard the crate from the workspace - we should maybe fix that if the plan is to keep the consistent functionality)

Maybe hiding this "continue on unknown registry" functionality for the 🤯 case behind a config flag would make it more easily digestable?

[weihanglo]

The Cargo team discussed this in the weekly meeting today. ~/.cargo/config.toml is a environment settings that should only affect user's environment locally. It's a bit risky if a dependency has control over the registry configuration. The infectious nature is an expected behavior to remind people to setup those registries.

Besides the proposed solution, people can leverage config-include share registry configurations, either by git submodules or some network filesystems. The error message is terribly bad, and we have a tracking issue #6211 for it.

Closing as this is an expected behavior and we have #6211 for improving the error message of git dependencies manifest parsing. Thank you.

[w4]

It's a bit risky if a dependency has control over the registry configuration

This is strange to me, dependencies do have control over the registry configuration and can depend on arbitrary registries - if, and only if, they're also depended on via a registry. If you depend on the same crate over Git the experience is completely broken. Can we at least add a environment flag to use sane behaviour (ignore a registry in Cargo.toml if it isn't going to be used anyway...) if we don't want to make it the default?

What will the error message say as a result of #6211? "Please locally define the registry that your dependency is using, but note that it isn't going to be used so this is superfluous, just deal with it"?