From eafb8fbd3215f0fe6b1d4a0360a9c4f761677179 Mon Sep 17 00:00:00 2001 From: Rahul Butani Date: Fri, 27 Jan 2023 11:26:14 -0800 Subject: [PATCH] llvmPackages_15.llvm: fix the tests on `x86_64-darwin` Details within but ultimately there isn't a satisfying resolution for any of the three test failures we were seeing and all three deserve further exploration. For the `sw_vers` macOS version issue in particular, it's possible to observe the nixpkgs provided `CoreFoundation` vs system `CoreFoundation` for `x86_64` and `aarch64` like so (on a host running macOS `13.0.1`): ```console $ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command "sw_vers" ProductName: macOS ProductVersion: 13.0.1 BuildVersion: 22A400 $ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command "sw_vers" ProductName: Mac OS X ProductVersion: 10.16 BuildVersion: 22A400 ``` Where `/System/Library/CoreServices/SystemVersion.plist` has: ```console $ cat /System/Library/CoreServices/SystemVersion.plist | grep ProductVersion -A 1 ProductVersion 13.0.1 ``` Further: ```console $ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command 'otool -L $(which sw_vers)' /nix/store/nb2q33ak2zif49ndcpc6m823z0vhmy8y-DarwinTools-1/bin/sw_vers: /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1) $ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -L $(which sw_vers)' /nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers: @rpath/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1454.90.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2) ``` For the `x86_64` `sw_vers` binary we can see rpath: ```console $ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -l $(which sw_vers)' | grep LC_RPATH -A 2 -B 1 Load command 13 cmd LC_RPATH cmdsize 120 path /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks (offset 12) ``` And we can confirm that the nixpkgs provided `CoreFoundation` is what ultimately gets loaded: ```console $ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'DYLD_PRINT_LIBRARIES=1 sw_vers' dyld[16215]: /nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers dyld[16215]: /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation dyld[16215]: /nix/store/xd2a4xh8kdwq0j67hzgw720npdw5hzkk-ICU-66108/lib/libicucore.A.dylib ``` ```bash nix-diff \ $(nix path-info nixpkgs#legacyPackages.aarch64-darwin.darwin.DarwinTools --derivation) \ $(nix path-info nixpkgs#legacyPackages.x86_64-darwin.darwin.DarwinTools --derivation) ``` doesn't show any _obvious_ discrepancies --- .../compilers/llvm/15/llvm/default.nix | 51 ++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/pkgs/development/compilers/llvm/15/llvm/default.nix b/pkgs/development/compilers/llvm/15/llvm/default.nix index 775270b29b63c..3a307cc22ac82 100644 --- a/pkgs/development/compilers/llvm/15/llvm/default.nix +++ b/pkgs/development/compilers/llvm/15/llvm/default.nix @@ -150,9 +150,58 @@ in stdenv.mkDerivation (rec { rm test/MC/ELF/cfi-version.ll # This test tries to call `sw_vers` by absolute path (`/usr/bin/sw_vers`) - # thus fails under the sandbox: + # and thus fails under the sandbox: substituteInPlace unittests/Support/Host.cpp \ --replace '/usr/bin/sw_vers' "${(builtins.toString darwin.DarwinTools) + "/bin/sw_vers" }" + '' + optionalString (stdenv.isDarwin && stdenv.hostPlatform.isx86) '' + # This test tries to call the intrinsics `@llvm.roundeven.f32` and + # `@llvm.roundeven.f64` which seem to (incorrectly?) lower to `roundevenf` + # and `roundeven` on x86_64 macOS. + # + # However these functions are glibc specific so the test fails: + # - https://www.gnu.org/software/gnulib/manual/html_node/roundevenf.html + # - https://www.gnu.org/software/gnulib/manual/html_node/roundeven.html + # + # TODO(@rrbutani): this seems to run fine on `aarch64-darwin`, why does it + # pass there? + substituteInPlace test/ExecutionEngine/Interpreter/intrinsics.ll \ + --replace "%roundeven32 = call float @llvm.roundeven.f32(float 0.000000e+00)" "" \ + --replace "%roundeven64 = call double @llvm.roundeven.f64(double 0.000000e+00)" "" + + # This test fails on darwin x86_64 because `sw_vers` reports a different + # macOS version than what LLVM finds by reading + # `/System/Library/CoreServices/SystemVersion.plist` (which is passed into + # the sandbox on macOS). + # + # The `sw_vers` provided by nixpkgs reports the macOS version associated + # with the `CoreFoundation` framework with which it was built. Because + # nixpkgs pins the SDK for `aarch64-darwin` and `x86_64-darwin` what + # `sw_vers` reports is not guaranteed to match the macOS version of the host + # that's building this derivation. + # + # Astute readers will note that we only _patch_ this test on aarch64-darwin + # (to use the nixpkgs provided `sw_vers`) instead of disabling it outright. + # So why does this test pass on aarch64? + # + # Well, it seems that `sw_vers` on aarch64 actually links against the _host_ + # CoreFoundation framework instead of the nixpkgs provided one. + # + # Not entirely sure what the right fix is here. I'm assuming aarch64 + # `sw_vers` doesn't intentionally link against the host `CoreFoundation` + # (still digging into how this ends up happening, will follow up) but that + # aside I think the more pertinent question is: should we be patching LLVM's + # macOS version detection logic to use `sw_vers` instead of reading host + # paths? This *is* a way in which details about builder machines can creep + # into the artifacts that are produced, affecting reproducibility, but it's + # not clear to me when/where/for what this even gets used in LLVM. + # + # TODO(@rrbutani): fix/follow-up + substituteInPlace unittests/Support/Host.cpp \ + --replace "getMacOSHostVersion" "DISABLED_getMacOSHostVersion" + + # This test fails with a `dysmutil` crash; have not yet dug into what's + # going on here (TODO(@rrbutani)). + rm test/tools/dsymutil/ARM/obfuscated.test '' + '' # FileSystem permissions tests fail with various special bits substituteInPlace unittests/Support/CMakeLists.txt \