Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Providing path to originally selected manifest file #13484

Closed
wmmc88 opened this issue Feb 23, 2024 · 7 comments
Closed

Providing path to originally selected manifest file #13484

wmmc88 opened this issue Feb 23, 2024 · 7 comments
Labels
A-build-scripts Area: build.rs scripts C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` S-propose-close Status: A team member has nominated this for closing, pending further input from the team

Comments

@wmmc88
Copy link

wmmc88 commented Feb 23, 2024

Problem

When running cargo commands like cargo build, packages are automatically selected based off of the cargo manifest selected (either based off of the current working directory of the invocation or the --manifest-path argument). AFAIU, a dependency graph is generated based off of this manifest and then the build is executed. By this logic, before any build-script runs, cargo already knows all of the packages that need to be built and has access to all of its manifest data. I want to access that data in one of the dependency's build.rs.

I have 2 crates- crate A and crate B. crate B has a path dependency on crate A. I would like to be able to read custom metadata in crate B's manifest by using cargo_metadata from within crate A's build.rs. This metadata is something like a globally configured feature proposed in https://internals.rust-lang.org/t/pre-rfc-mutually-excusive-global-features/19618/38. When running cargo metadata cli command from crate B's directory, I can see the metadata in the crate B's manifest, but when getting metadata via crate A's build.rs, I cannot. If both crates existed in the same workspace, it would work, but I intend for crate A to be able to be consumed by others to use and configure via metadata in user's dependent crates.

If the originally selected manifest path was provided to build scripts, I could point to that manifest when using cargo_metadata in crate A's build.rs. As it stands, I need to use OUT_DIR to derive where the selected/root manifest of the build is, but this is fragile and does not work when the default target directory is changed by things like --target-dir(similar issue to #9661 ).

Proposed Solution

A environment variable provided to builds scripts that exposes to originally selected manifest path.

Notes

seems like a similar feature request to #8148

@wmmc88 wmmc88 added C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` S-triage Status: This issue is waiting on initial triage. labels Feb 23, 2024
@weihanglo
Copy link
Member

... cargo already knows all of the packages that need to be built and has access to all of its manifest data. I want to access that data in one of the dependency's build.rs.

This kind of inverse relation that a dependency knows the full picture of a build sounds a bit odd. Could you elaborate more on what you actually want to achieve?

BTW, there is a relevant issue #3946 about finding the root workspace, and the current experiment of that is an unstable env var CARGO_RUSTC_CURRENT_DIR you might be interested in.

@weihanglo weihanglo added S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request. and removed S-triage Status: This issue is waiting on initial triage. labels Feb 23, 2024
@wmmc88
Copy link
Author

wmmc88 commented Feb 23, 2024

This kind of inverse relation that a dependency knows the full picture of a build sounds a bit odd. Could you elaborate more on what you actually want to achieve?

I have a sys crate who's bindings are globally configurable. I want to be able to configure the bindings based off of info in the metadata table in downstream crates. I also considered configuring it via cargo features, but features are meant to be additive and not mutually exclusive, and I also think this metadata approach is a better user experience. More context here: microsoft/windows-drivers-rs#82 (comment)

I need a path to pass to MetadataCommand.manifest_path such that the output matches running cargo metadata from the directory that cargo build was invoked in. My understanding is that providing the path to directory with the Cargo.lock file should achieve what I want.

From the description of CARGO_RUSTC_CURRENT_DIR, it looks like it would also work for my usecase, but unfortunately it looks like it is not exposed to build scripts.

epage added a commit to epage/cargo that referenced this issue Mar 26, 2024
This provides what cargo sets as the `current_dir` for the `rustc`
process.
While `std::file!` is unspecified in what it is relative to,
it is relatively safe, it is generally relative to `rustc`s
`current_dir`.

This can be useful for snapshot testing.
For example, `snapbox` has been using this macro on nightly since
assert-rs/snapbox#247, falling back to finding a parent of
`CARGO_MANIFEST_DIR`, if present.
This has been in use in Cargo since rust-lang#13441.

This was added in rust-lang#12996.
Relevant points discussed in that issue:
- This diverged from the original proposal from the Cargo team of having
  a `CARGO_WORKSPACE_DIR` that is the "workspace" of the package being
  built (ie registry packages would map to `CARGO_MANIFEST_DIR`).
  In looking at the `std::file!` use case, `CARGO_MANIFEST_DIR`, no
  matter how we defined it, would only sort of work because no sane
  definition of that maps to `rustc`'s `current_dir`.a
  This instead focuses on the mechanism currently being used.
- Using "current dir" in the name is meant to be consistent with
  `std::env::current_dir`.
- I can go either way on `CARGO_RUSTC` vs `RUSTC`.  Existing related
  variables:
  - `RUSTC`
  - `RUSTC_WRAPPER`
  - `RUSTC_WORKSPACE_WRAPPER`
  - `RUSTFLAGS` (no `C`)
  - `CARGO_CACHE_RUSTC_INFO`

Note that rust-lang#3946 was overly broad and covered many use cases.
One of those was for packages to look up information on their
dependents.
Issue rust-lang#13484 is being left open to track that.

Fixes rust-lang#3946
epage added a commit to epage/cargo that referenced this issue May 21, 2024
This provides what cargo sets as the `current_dir` for the `rustc`
process.
While `std::file!` is unspecified in what it is relative to,
it is relatively safe, it is generally relative to `rustc`s
`current_dir`.

This can be useful for snapshot testing.
For example, `snapbox` has been using this macro on nightly since
assert-rs/snapbox#247, falling back to finding a parent of
`CARGO_MANIFEST_DIR`, if present.
This has been in use in Cargo since rust-lang#13441.

This was added in rust-lang#12996.
Relevant points discussed in that issue:
- This diverged from the original proposal from the Cargo team of having
  a `CARGO_WORKSPACE_DIR` that is the "workspace" of the package being
  built (ie registry packages would map to `CARGO_MANIFEST_DIR`).
  In looking at the `std::file!` use case, `CARGO_MANIFEST_DIR`, no
  matter how we defined it, would only sort of work because no sane
  definition of that maps to `rustc`'s `current_dir`.a
  This instead focuses on the mechanism currently being used.
- Using "current dir" in the name is meant to be consistent with
  `std::env::current_dir`.
- I can go either way on `CARGO_RUSTC` vs `RUSTC`.  Existing related
  variables:
  - `RUSTC`
  - `RUSTC_WRAPPER`
  - `RUSTC_WORKSPACE_WRAPPER`
  - `RUSTFLAGS` (no `C`)
  - `CARGO_CACHE_RUSTC_INFO`

Note that rust-lang#3946 was overly broad and covered many use cases.
One of those was for packages to look up information on their
dependents.
Issue rust-lang#13484 is being left open to track that.

Fixes rust-lang#3946
@weihanglo weihanglo added A-build-scripts Area: build.rs scripts S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. and removed S-needs-info Status: Needs more info, such as a reproduction or more background for a feature request. labels Jun 9, 2024
@epage epage added S-propose-close Status: A team member has nominated this for closing, pending further input from the team and removed S-needs-design Status: Needs someone to work further on the design for the feature or fix. NOT YET accepted. labels Jun 10, 2024
@epage
Copy link
Contributor

epage commented Jun 10, 2024

I propose we close this feature request.

This creates an inverse relationship in packages. Smoothing out that path I feel like would lean to anti-patterns. Particularly problematic about this is that it assumes a selected manifest is where the config can come from. --manifest-path is where walking starts. --package is what is the "root" and there can be multiple of them. This would likely need a baked in feature like https://internals.rust-lang.org/t/pre-rfc-mutually-excusive-global-features/19618 or #3946 and deal with the config at the workspace level.

@wmmc88
Copy link
Author

wmmc88 commented Jun 11, 2024

This creates an inverse relationship in packages

This would also be true of mutually exclusive global features, right? dependencies can have conditional compilation based on the cfg introduced by globals declared in a dependent's manifest?

Particularly problematic about this is that it assumes a selected manifest is where the config can come from. --manifest-path is where walking starts. --package is what is the "root" and there can be multiple of them.

These are not concerns for my specific use case, since I just use need to know where the walking starts from cargo's perspective. I run cargo metadata with that manifest, and parse and validate the result (i.e. search entire graph for config metadata (including workspace), ensure that there are no conflicting metadata configs, etc.).

#3946 wouldn't work for this use case since the end-user workspace is different from the workspace the dependencies live in. An official feature for "mutually exclusive global features" would replace the need for what I am currently doing with cargo metadata, but until that happens, continuing to use cargo metadata with the fragile OUT_DIR outlined in the OP is probably what I'll need to stick with.

@epage
Copy link
Contributor

epage commented Jun 11, 2024

This would also be true of mutually exclusive global features, right? dependencies can have conditional compilation based on the cfg introduced by globals declared in a dependent's manifest?

Features. mutually exclusive features, etc are a very controlled mechanism for communicating vs giving carte blanche access to dependents.

These are not concerns for my specific use case

Unless you only support workspace config, you would need a way to find which package config is relevant.

We also have to keep in mind the general shape of a feature and not design for one specific user.

@wmmc88
Copy link
Author

wmmc88 commented Jun 11, 2024

These are not concerns for my specific use case

Unless you only support workspace config, you would need a way to find which package config is relevant.

Yeah, in my specific use case, I look at all the package configs and bail if there are conflicting configs

We also have to keep in mind the general shape of a feature and not design for one specific user.

Totally agree with you here. Sounds like closing this in favor of the proposed "mutually exclusive global features" is the conclusion for me here

@weihanglo
Copy link
Member

Totally agree with you here. Sounds like closing this in favor of the proposed "mutually exclusive global features" is the conclusion for me here

Thanks for the conversation from you all. Closing.

@weihanglo weihanglo closed this as not planned Won't fix, can't repro, duplicate, stale Jun 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-build-scripts Area: build.rs scripts C-feature-request Category: proposal for a feature. Before PR, ping rust-lang/cargo if this is not `Feature accepted` S-propose-close Status: A team member has nominated this for closing, pending further input from the team
Projects
None yet
Development

No branches or pull requests

3 participants