Enable customizing the linkage of a platform's C runtime #1721
Conversation
Enable the compiler to select whether a target dynamically or statically links to a platform's standard C runtime through the introduction of three orthogonal and otherwise general purpose features, one of which would likely never become stable.
Revising crt-link
More crt-link edits
|
cc @vadimcn @retep998 |
|
cc @japaric @KennethAdamMiller @zhangyt26 a way forward on musl dynamic linking |
|
Another alternative would be to separate |
|
How would this interact with RFC #1717? #[link(name = "msvcrt", kind="dylib", cfg(not(target_feature = "crt-static")))]
#[link(name = "libcmt", kind="static-nobundle", cfg(target_feature = "crt-static"))]how do we decide whether to emit dllimport's on symbols? Using the default state of that target feature? |
|
@vadimcn ah this was intended to clarify that:
Which is to say that when dllexport/dllimport need to be appplied, the Note, though, that this is not intended to be a general purpose mechanism. It's the unstable feature of this RFC which is never intended to become stable. It does't affect liblibc at all as well because the |
| * Another possibility would be to start storing metadata in the "target name" | ||
| along the lines of `x86_64-pc-windows-msvc+static`. This is a pretty big | ||
| design space, though, which may not play well with Cargo and build scripts, so | ||
| for now it's preferred to avoid this rabbit hole of design if possible. |
|
Well this satisfies my requirements of getting However, it is worth paying attention to how this feature could be extended for other options in the future. For example choosing which winapi family you are targeting, with mutually exclusive options such as Another option might be the minimum version of Windows that needs to be supported, which affects which functions can be directly linked to and adding fallback code paths for certain functionality. Libstd could handle this by always supporting the lowest version possible, but if it could be recompiled it could take advantage of higher minimum version requirements to strip out code branches for older versions. |
| So for a target with the following features enabled | ||
|
|
||
| ``` | ||
| target_feature="sse" |
There was a problem hiding this comment.
Will Cargo set the CARGO_CFG_TARGET_FEATURE variable for CPU features that are implied by the
"llvm-target" field of a target specification? For example:
Will this
cargo build --target x86_64-unknown-linux-gnu
produce this
CARGO_CFG_TARGET_FEATURE="mmx,sse,sse2,."
?
And this
RUSTFLAGS="-mmx,-sse,-sse2" cargo build --target x86_64-unknown-linux-gnu
produce this
CARGO_CFG_TARGET_FEATURE=""
?
Because, otherwise, this scheme would leave the CARGO_CFG_TARGET_FEATURE empty for both cargo build cases and that's not an accurate representation of the target CPU features.
And if Cargo will, indeed, set these implicit CPU features based on the "llvm-target", how will that
be implemented? Will Cargo call ("shell out to") a new rustc --print llvm-target-features command
created for this purpose?
There was a problem hiding this comment.
Will Cargo call ("shell out to") a new rustc --print llvm-target-features command
created for this purpose?
Oh, I just saw the output of rustc --print cfg and seems like it contains the target features I mentioned above, at least sse and sse2 appear for x86_64-unknown-linux-gnu, so I guess this is not really an issue -- Cargo can just use that command.
There was a problem hiding this comment.
Will Cargo set the CARGO_CFG_TARGET_FEATURE variable for CPU features that are implied by the "llvm-target" field of a target specification?
Yes
And if Cargo will, indeed, set these implicit CPU features based on the "llvm-target", how will that be implemented?
This'll be done through --print cfg. Right now the target_feature cfg is unstable so this is only available on nightly, but for this RFC we'd want to stabilize that to have everything move forward at once. If you take a look at rustup run nightly rustc --print cfg, though, you'll see target_feature in there.
Cargo already runs the compiler to learn about various things like filenames, so this'd just be another thing it'd learn about during that process.
|
| script invocations for this target. | ||
|
|
||
| ``` | ||
| export CARGO_CFG_TARGET_OS=linux |
There was a problem hiding this comment.
Will these also be set when one calls e.g. cargo rustc -- -C target-feature=+avx? That would make these variables available for the target crate but not for its dependencies which ... might be surprising. But I guess the same already happens with cargo rustc and cfg(target_feature)?
There was a problem hiding this comment.
Yeah I think cargo rustc is the unsafe escape hatch for those sorts of things. Anything after -- is passed verbatim to rustc right?
There was a problem hiding this comment.
Will these also be set when one calls e.g. cargo rustc -- -C target-feature=+avx?
No, currently only RUSTFLAGS affects how Cargo runs rustc initially.
Okay, so all upstream crates should also be compiled with the same |
|
I think #1711 ought to be a blocker for this. Once we start passing this stuff to build scripts we will have to be much more careful about stability.. |
I believe the idea is that this new link cfg syntax is specifically for lazy cfg evaluation. It will be evaluated at the point of linking, so the upstream crate doesn't need to know what the target-feature flag is when it was compiled, rustc will store both of those attributes with their respective cfgs and evaluate it later when linking. cfg_attr would be evaluated immediately and so would be unusable for libstd as at the time of building libstd it doesn't know how you want the crt to be linked yet.
There's no way dllimport/dllexport can be applied lazily unless you forbid translation of any function that calls extern symbols from such a lazy cfg'd link block until the final compilation unit where linking will occur. Actually, @alexcrichton , how does this interact with dylibs? You need to know which CRT you are linking every time you are linking, and when making a Rust dylib, you are still linking stuff so you need to know which CRT to use. If statically linking the CRT in a given dylib, that means it has its own copy of the CRT which is fundamentally unique from all other dylibs. What happens if code works with a CRT object in a Rust dylib which statically linked the CRT, but that function is then inlined or monomorphized into a different dylib? |
|
CC @eternaleye, you're probably interested. |
Exactly. But I think @alexcrichton meant that for exact dllimport targetting all upstream crates would be compiled with the same target-feature flag.
I believe the CRT symbols will be resolved to the other crate's CRT. Which isn't necessarily a problem if they are all stateless functions. |
Fortunately in the case of libstd it is all stateless functions. However anyone who uses CRT state in their own Rust crates has to be keenly aware that if someone uses the static CRT with their crate, it will completely screw up their code. |
|
|
||
| The default of `crt-static` will be different depending on the target. For | ||
| example `x86_64-unknown-linux-musl` will have it on by default, whereas | ||
| `arm-unknown-linux-musleabi` will have it turned off by default. |
There was a problem hiding this comment.
I there any particular reason why some musl targets will be statically linked by default and some not? If it's just historical then I think we should change it for consistency.
There was a problem hiding this comment.
Yes as we've added them over time they've addressed different use cases. The plan is to not change any of them yet, as that would be a breaking change. Each target (event with the same C library) can choose whether it's static or dynamic by default.
If the crt-static option becomes more ergonomic and ubiquitous we can consider changing defaults in the future, but at this time the fact that it's a breaking change prevents us from doing so.
There was a problem hiding this comment.
I there any particular reason why some musl targets will be statically linked by default and some not?
The mips(el)-musl targets, for instance, can't produce statically linked binaries because static linking depends on libunwind and libunwind doesn't support mips (yet).
There was a problem hiding this comment.
Right, that's why it would make sense to use dynamic linking by default so all musl targets can be consistent.
There was a problem hiding this comment.
To reiterate, we can't do that because it's a breaking change. There are many users relying on the fact that musl is a static target today. We can consider changing this all in the far future, but it is an explicitly stated non-goal of this RFC to attempt to perform any kind of unification of how the CRT is linked on various platforms.
|
|
||
| ```rust | ||
| cfg_if! { | ||
| if #[cfg(target_env = "musl")] { |
There was a problem hiding this comment.
Just to confirm: this shouldn't be limited to musl. glibc (and it's variants) have (some) support for static linking too.
There was a problem hiding this comment.
Yes if support is added to get that working, we can encode that here. The specifics of that, however, are out of scope of this RFC. It's intended though that there's enough plumbing here that one could imagine:
RUSTFLAGS='-C target-feature=+crt-static' cargo build --target x86_64-unknown-linux-gnu
to one day work!
|
I just had someone trying to get some Rust code to link against the debug CRT, so would this be easily extendable to that as well? |
|
Yeah my guess is we could either infer that from |
|
I'd rather not infer it from |
|
@rfcbot reviewed |
1 similar comment
|
@rfcbot reviewed |
|
Can we use something more explicit than |
alexcrichton
added
the
final-comment-period
|
All relevant subteam members have reviewed. No concerns remain. |
|
@michaelwoerister sure yeah, I don't mind basically picking anything for that in the time being. |
|
🔔 This RFC is now entering its week-long final comment period for merging 🔔 |
|
This isn't a concern, but to my knowledge nothing is blocking stdlib deps, and @japaric has done excellent work making std actually buildable by cargo in practice, so perhaps "Lazy link attributes" won't even need to be implemented? I figure its best to keep them in the RFC, and then (r)evaluate as things are implemented. |
|
It has been one week since all blocks to the FCP were resolved. |
|
Looks like no new concerns arose during FCP, so merging! Thanks again for the discussion everyone! |
Centril
added
A-cfg
Enable the compiler to select whether a target dynamically or statically links
to a platform's standard C runtime through the introduction of three orthogonal
and otherwise general purpose features, one of which will likely never become
stable and can be considered an implementation detail of std. These features do
not require the compiler or language to have intrinsic knowledge of the
existence of C runtimes.
The end result is that rustc will be able to reuse its existing standard library binaries for the MSVC and musl targets to build code that links either statically or dynamically to libc.
The design herein additionally paves the way for improved support for dllimport/dllexport, and cpu-specific features, particularly when combined with a std-aware cargo.
Rendered