Experimental MUSL target support for the compiler #24777
Conversation
|
r? @pcwalton (rust_highfive has picked a reviewer for you, use r? to override) |
|
r? @brson |
|
Why 'x86_64-unknown-linux-musl'? It seems reasonable to me, but Google doesn't have a lot of hits for it. |
|
Neat! |
|
@brson I found it followed clang's target triple format, but I personally got the idea from the reddit post. |
|
r=me |
|
@brson I've pushed some more commits clarifying some documentation and also feature gating |
|
⌛ Testing commit 01963a6 with merge 9e28730... |
|
💔 Test failed - auto-win-64-nopt-t |
This adds a new `#[cfg]` matcher against the `target_env` property of the destination target triple. For example all windows triples today end with `-gnu` but we will also hopefully support non-`gnu` targets for Windows, at which point we'll need to differentiate between the two. This new `target_env` matches is provided and filled in with the target's environment name. Currently the only non-empty value of this name is `gnu`, but `musl` will be shortly added for the linux triples.
This commit adds support to the makefiles, configuration script, and build system to understand MUSL. This is broken up into a few parts: * Any target of the form `*-musl` requires the `--musl-root` option to `./configure` which will indicate the root of the MUSL installation. It is also expected that there is a libunwind build inside of that installation built against that MUSL. * Objects from MUSL are copied into the build tree for Rust to be statically linked into the appropriate Rust library. * Objects for binary startup and shutdown are included in each Rust installation by default for MUSL. This requires MUSL to only be installed on the machine compiling rust. Only a linker will be necessary for compiling against MUSL on a target machine. Eventually a MUSL and/or libunwind build may be integrated by default into the build but for now they are just always assumed to exist externally.
This commit adds support for x86_64-unknown-linux-musl as a target of the compiler. There's some comments in the commit about some of the more flavorful flags passed to the linker as it's not quite as trivial as the normal specs.
MUSL for example provides its own start/end objects in place of the standard ones shipped by gcc.
This commit modifies the standard library and its dependencies to link correctly when built against MUSL. This primarily ensures that the right libraries are linked against and when they're linked against they're linked against statically.
MUSL support is currently only with static builds, so building a dylib will always fail.
Statically linked executables do not succeed (aka MUSL-based executables).
Add the ability to ignore a test based on the environment of the triple being used.
|
⌛ Testing commit faa1a81 with merge a574d45... |
|
💔 Test failed - auto-win-32-nopt-t |
There were a few test cases to fix: * Dynamic libraries are not supported with MUSL right now, so all of those related test which force or require dylibs are ignored. * Looks like the default stack for MUSL is smaller than glibc, so a few stack allocations in benchmarks were boxed up (shouldn't have a perf impact). * Some small linkage tweaks here and there * Out-of-stack detection does not currently work with MUSL
These commits build on [some great work on reddit](http://www.reddit.com/r/rust/comments/33boew/weekend_experiment_link_rust_programs_against/) for adding MUSL support to the compiler. This goal of this PR is to enable a `--target x86_64-unknown-linux-musl` argument to the compiler to work A-OK. The outcome here is that there are 0 compile-time dependencies for a MUSL-targeting build *except for a linker*. Currently this also assumes that MUSL is being used for statically linked binaries so there is no support for dynamically linked binaries with MUSL. MUSL support largely just entailed munging around with the linker and where libs are located, and the major highlights are: * The entirety of `libc.a` is included in `liblibc.rlib` (statically included as an archive). * The entirety of `libunwind.a` is included in `libstd.rlib` (like with liblibc). * The target specification for MUSL passes a number of ... flavorful options! Each option is documented in the relevant commit. * The entire test suite currently passes with MUSL as a target, except for: * Dynamic linking tests are all ignored as it's not supported with MUSL * Stack overflow detection is not working MUSL yet (I'm not sure why) * There is a language change included in this PR to add a `target_env` `#[cfg]` directive. This is used to conditionally build code for only MUSL (or for linux distros not MUSL). I highly suspect that this will also be used by Windows to target MSVC instead of a MinGW-based toolchain. To build a compiler targeting MUSL you need to follow these steps: 1. Clone the current MUSL repo from `git://git.musl-libc.org/musl`. Build this as usual and install it. 2. Clone and build LLVM's [libcxxabi](http://libcxxabi.llvm.org/) library. Only the `libunwind.a` artifact is needed. I have tried using upstream libunwind's source repo but I have not gotten unwinding to work with it unfortunately. Move `libunwind.a` adjacent to MUSL's `libc.a` 3. Configure a Rust checkout with `--target=x86_64-unknown-linux-musl --musl-root=$MUSL_ROOT` where `MUSL_ROOT` is where you installed MUSL in step 1. I hope to improve building a copy of libunwind as it's still a little sketchy and difficult to do today, but other than that everything should "just work"! This PR is not intended to include 100% comprehensive support for MUSL, as future modifications will probably be necessary.
|
@alexcrichton got any details of the issues you had with libunwind upstream? I only ask because a toy program I tried seemed to work ok, so I guess it's related to the additional complexity in the compiler? |
|
@aidanhs unfortunately I could never get any more detail other than "unwinding didn't work". I was able to build libunwind upstream with In your test program, did you trigger an unwind, or was it just running as usual? Also, did you configure libunwind with any special flags or anything? |
| let lib = match DynamicLibrary::open(None) { | ||
| Ok(l) => l, | ||
| Err(..) => return, | ||
| }; |
There was a problem hiding this comment.
This might be why stack overflow detection is not working.
There was a problem hiding this comment.
Hm, could you elaborate a little more? This should in theory just affect the return value of this function, which I'm not sure how connected it is to stack sizes.
There was a problem hiding this comment.
Before it failed loud and clear when this didn’t work, but now min_stack_size returns PTHREAD_STACK_MIN instead which might or might not be a correct value to use in the stack size calculations.
/me shrugs.
|
@alexcrichton oops, I didn't get unwinding working (though normal execution worked). I did get a different error to you though: Sorry for the noise! |
Ah that's actually the same error that I was seeing, oh well! |
…dor, r=joshtriplett,Centril Stabilize cfg_target_vendor This stabilizes the use of `cfg(target_vendor = "...")` and removes the corresponding `cfg_target_vendor` feature. Other unstable cfgs remain behind their existing feature gates. This functionality was added back in 2015 in rust-lang#28612 to complete the coverage of target tuples (`<arch><sub>-<vendor>-<os>-<env>`). [RFC 131](https://github.com/rust-lang/rfcs/blob/master/text/0131-target-specification.md) governs the target specification, not including `target_vendor` seems to have just been an oversight. `target_os`, `target_family`, and `target_arch` are stable as of 1.0.0. `target_env` was also not mentioned in RFC 131, was added in rust-lang#24777, never behind a feature_gate, and insta-stable at 1.1.0. The functionality is tested in [test/run-pass/cfg/cfg-target-vendor.rs](https://github.com/rust-lang/rust/blob/master/src/test/run-pass/cfg/cfg-target-vendor.rs). Closes rust-lang#29718
…dor, r=joshtriplett,Centril Stabilize cfg_target_vendor This stabilizes the use of `cfg(target_vendor = "...")` and removes the corresponding `cfg_target_vendor` feature. Other unstable cfgs remain behind their existing feature gates. This functionality was added back in 2015 in rust-lang#28612 to complete the coverage of target tuples (`<arch><sub>-<vendor>-<os>-<env>`). [RFC 131](https://github.com/rust-lang/rfcs/blob/master/text/0131-target-specification.md) governs the target specification, not including `target_vendor` seems to have just been an oversight. `target_os`, `target_family`, and `target_arch` are stable as of 1.0.0. `target_env` was also not mentioned in RFC 131, was added in rust-lang#24777, never behind a feature_gate, and insta-stable at 1.1.0. The functionality is tested in [test/run-pass/cfg/cfg-target-vendor.rs](https://github.com/rust-lang/rust/blob/master/src/test/run-pass/cfg/cfg-target-vendor.rs). Closes rust-lang#29718
|
Is there any update with this, please? I am getting The freshly and successfully compiled binary is of |
These commits build on some great work on reddit for adding MUSL support to the compiler. This goal of this PR is to enable a
--target x86_64-unknown-linux-muslargument to the compiler to work A-OK. The outcome here is that there are 0 compile-time dependencies for a MUSL-targeting build except for a linker. Currently this also assumes that MUSL is being used for statically linked binaries so there is no support for dynamically linked binaries with MUSL.MUSL support largely just entailed munging around with the linker and where libs are located, and the major highlights are:
libc.ais included inliblibc.rlib(statically included as an archive).libunwind.ais included inlibstd.rlib(like with liblibc).target_env#[cfg]directive. This is used to conditionally build code for only MUSL (or for linux distros not MUSL). I highly suspect that this will also be used by Windows to target MSVC instead of a MinGW-based toolchain.To build a compiler targeting MUSL you need to follow these steps:
git://git.musl-libc.org/musl. Build this as usual and install it.libunwind.aartifact is needed. I have tried using upstream libunwind's source repo but I have not gotten unwinding to work with it unfortunately. Movelibunwind.aadjacent to MUSL'slibc.a--target=x86_64-unknown-linux-musl --musl-root=$MUSL_ROOTwhereMUSL_ROOTis where you installed MUSL in step 1.I hope to improve building a copy of libunwind as it's still a little sketchy and difficult to do today, but other than that everything should "just work"! This PR is not intended to include 100% comprehensive support for MUSL, as future modifications will probably be necessary.