rustbuild: Fix cross compiling to FreeBSD #32239
Conversation
|
r? @brson (rust_highfive has picked a reviewer for you, use r? to override) |
This commit fixes our support for cross compiling a compiler to run on FreeBSD. Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1] which I hope to soon use to start producing FreeBSD nightly compilers. With the `make dist` support added in rust-lang#32237 we should be able to produce standard rustc/rust-std packages for FreeBSD through a new slave with this cross compiler. Currently, however, we don't "Just Work" when cross compiling FreeBSD and a number of changes were required (part of this PR). They include: * A few build fixes were needed in LLVM. Our own branch has been rebased on the actual 3.8 release and I applied one extra commit [2] which contains two fixes: 1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately this doesn't take into account when we're cross compiling, and as predicted the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which is what we're compiling for which fixes the problem. 2. The `PATH_MAX` constant is apparently defined in a different location than many other Unix systems, so a file which required this just needed some help to keep compiling. * Support for compiling compiler-rt with CMake has been added to rustbuild. It looks like it just emulates Linux in what it compiles as it didn't seem to naturally produce anything else... At least the architecture is right, so seems good for now at least! [1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile [2]: rust-lang/llvm@be89e4b5 [3]: https://bugs.webkit.org/show_bug.cgi?id=138420
alexcrichton
force-pushed
the
fix-cross-to-freebsd
branch
from
1bb076e to
155735a
Compare
|
@bors r+ |
|
📌 Commit 155735a has been approved by |
rustbuild: Fix cross compiling to FreeBSD This commit fixes our support for cross compiling a compiler to run on FreeBSD. Over the weekend I managed to get a cross compiler from Linux to FreeBSD [1] which I hope to soon use to start producing FreeBSD nightly compilers. With the `make dist` support added in #32237 we should be able to produce standard rustc/rust-std packages for FreeBSD through a new slave with this cross compiler. Currently, however, we don't "Just Work" when cross compiling FreeBSD and a number of changes were required (part of this PR). They include: * A few build fixes were needed in LLVM. Our own branch has been rebased on the actual 3.8 release and I applied one extra commit [2] which contains two fixes: 1. The LLVM CMake build system passes the `-Wl,-z,defs` flag on many platforms, but *not* when `CMAKE_SYSTEM_NAME` is "FreeBSD". Unfortunately this doesn't take into account when we're cross compiling, and as predicted the build will fail if `-Wl,-z,defs` is passed (see [3] for more info). To fix this we test `TARGET_TRIPLE` instead of the `CMAKE_SYSTEM_NAME` which is what we're compiling for which fixes the problem. 2. The `PATH_MAX` constant is apparently defined in a different location than many other Unix systems, so a file which required this just needed some help to keep compiling. * Support for compiling compiler-rt with CMake has been added to rustbuild. It looks like it just emulates Linux in what it compiles as it didn't seem to naturally produce anything else... At least the architecture is right, so seems good for now at least! [1]: https://github.com/alexcrichton/port-of-rust/blob/master/prebuilt/freebsd/Dockerfile [2]: rust-lang/llvm@be89e4b5 [3]: https://bugs.webkit.org/show_bug.cgi?id=138420
The LLVM change [1] in rust-lang#32239 unfortunately broke the LLVM build on MinGW, so this LLVM submodule update brings in one more fix [2] which should hopefully remedy that. Once this lands we should be able to immediately start gating on this to prevent it from happening again. [1]: rust-lang/llvm@be89e4b5 [2]: rust-lang/llvm@3dcd2c84
rustbuild: Fix LLVM compile on MinGW The LLVM change [1] in #32239 unfortunately broke the LLVM build on MinGW, so this LLVM submodule update brings in one more fix [2] which should hopefully remedy that. Once this lands we should be able to immediately start gating on this to prevent it from happening again. [1]: rust-lang/llvm@be89e4b5 [2]: rust-lang/llvm@3dcd2c84
|
@alexcrichton this is great! will this allow me to take the FreeBSD buildbot down once it is in place and working? |
|
@dhuseby it's up to you I think. We're not producing FreeBSD nightlies and standard libraries (as of yesterday), but we're unfortunately not running any tests just yet. If you want to keep the bots online to run tests we can do that, but for nightlies we'll at least have that taken care of |
|
@alexcrichton so this is building nightlies and snapshots? what about cargo nightlies and snapshots? |
|
Yeah we've got FreeBSD nightlies (not snapshots) for both rustc and Cargo now |
|
@alexcrichton so do these show up in the buildbot builders lists? if we have these for linux/windows/mac, why do we still have buildbots for those platforms? |
|
Yeah the auto-linux-64-x-freebsd and auto-linux-64-x-netbsd builders are the auto builders, and the nightly-dist-rustc-cross-linux is just one massive builder creating a mess of compilers. I left the other FreeBSD bots for now because the cross-compiled ones aren't running tests (whereas the current ones are), but if you'd like I can turn those off. |
|
@alexcrichton so what is it going to take to get them to start running tests? |
|
@alexcrichton I suppose that's not possible unless we have a machine running freebsd right? |
|
Yeah we don't have any time frame for getting a FreeBSD bot, but we could in theory also run them under qemu somehow (or some subset). Either way that's pretty far out and not currently planned. |
|
@alexcrichton i've been interested in figuring out how to make gitian work with the bsd OS's just like it does with Linux now. Currently, it relies on qemu to boot a linux image, install dependencies, clone the code, run a build, and extract the build outputs. If we could make it also drive qemu to boot a bsd image, install dependencies, clone the code, run a build, and extract the build outputs, we could get away from cross-compiling from Linux. What do you think? |
|
Hey so long as it works and id easy to manage I'm game :). I've had bad luck with performance in the past running code in qemu (took forever to finish), but maybe I was passing the wrong options or there's some optimizations along the way? |
|
@alexcrichton All of my buildbot slaves are run as qemu vm's (with kvm acceleration) on my server. It seems to run fine. |
|
Interesting! Seems like we should definitely at least try to get these running! |
This commit fixes our support for cross compiling a compiler to run on FreeBSD.
Over the weekend I managed to get a cross compiler from Linux to FreeBSD 1
which I hope to soon use to start producing FreeBSD nightly compilers. With the
make distsupport added in #32237 we should be able to produce standardrustc/rust-std packages for FreeBSD through a new slave with this cross compiler.
Currently, however, we don't "Just Work" when cross compiling FreeBSD and a
number of changes were required (part of this PR). They include:
actual 3.8 release and I applied one extra commit 2 which contains two fixes:
-Wl,-z,defsflag on manyplatforms, but not when
CMAKE_SYSTEM_NAMEis "FreeBSD". Unfortunatelythis doesn't take into account when we're cross compiling, and as predicted
the build will fail if
-Wl,-z,defsis passed (see 3 for more info). Tofix this we test
TARGET_TRIPLEinstead of theCMAKE_SYSTEM_NAMEwhichis what we're compiling for which fixes the problem.
PATH_MAXconstant is apparently defined in a different location thanmany other Unix systems, so a file which required this just needed some
help to keep compiling.
looks like it just emulates Linux in what it compiles as it didn't seem to
naturally produce anything else... At least the architecture is right, so
seems good for now at least!