Skip to content

Commit

Permalink
llvmPackages_15.llvm: fix the tests on x86_64-darwin
Browse files Browse the repository at this point in the history
Details within but ultimately there isn't a satisfying resolution for
any of the three test failures we were seeing and all three deserve
further exploration.

For the `sw_vers` macOS version issue in particular, it's possible to
observe the nixpkgs provided `CoreFoundation` vs system `CoreFoundation`
for `x86_64` and `aarch64` like so (on a host running macOS `13.0.1`):

```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command "sw_vers"
ProductName:    macOS
ProductVersion: 13.0.1
BuildVersion:   22A400

$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command "sw_vers"
ProductName:    Mac OS X
ProductVersion: 10.16
BuildVersion:   22A400
```

Where `/System/Library/CoreServices/SystemVersion.plist` has:
```console
$ cat /System/Library/CoreServices/SystemVersion.plist | grep ProductVersion
-A 1
	<key>ProductVersion</key>
	<string>13.0.1</string>
```

Further:
```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/nb2q33ak2zif49ndcpc6m823z0vhmy8y-DarwinTools-1/bin/sw_vers:
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)

$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers:
	@rpath/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1454.90.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
```

For the `x86_64` `sw_vers` binary we can see rpath:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -l $(which sw_vers)' | grep LC_RPATH -A 2 -B 1
Load command 13
          cmd LC_RPATH
      cmdsize 120
         path /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks (offset 12)
```

And we can confirm that the nixpkgs provided `CoreFoundation` is what
ultimately gets loaded:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'DYLD_PRINT_LIBRARIES=1 sw_vers'
dyld[16215]: <no uuid> /nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers
dyld[16215]: <no uuid> /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
dyld[16215]: <no uuid> /nix/store/xd2a4xh8kdwq0j67hzgw720npdw5hzkk-ICU-66108/lib/libicucore.A.dylib
<snipped>
```

```bash
nix-diff \
    $(nix path-info nixpkgs#legacyPackages.aarch64-darwin.darwin.DarwinTools --derivation) \
    $(nix path-info nixpkgs#legacyPackages.x86_64-darwin.darwin.DarwinTools --derivation)
```
doesn't show any _obvious_ discrepancies
  • Loading branch information
rrbutani committed Jan 27, 2023
1 parent 0ee5251 commit eafb8fb
Showing 1 changed file with 50 additions and 1 deletion.
51 changes: 50 additions & 1 deletion pkgs/development/compilers/llvm/15/llvm/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -150,9 +150,58 @@ in stdenv.mkDerivation (rec {
rm test/MC/ELF/cfi-version.ll
# This test tries to call `sw_vers` by absolute path (`/usr/bin/sw_vers`)
# thus fails under the sandbox:
# and thus fails under the sandbox:
substituteInPlace unittests/Support/Host.cpp \
--replace '/usr/bin/sw_vers' "${(builtins.toString darwin.DarwinTools) + "/bin/sw_vers" }"
'' + optionalString (stdenv.isDarwin && stdenv.hostPlatform.isx86) ''
# This test tries to call the intrinsics `@llvm.roundeven.f32` and
# `@llvm.roundeven.f64` which seem to (incorrectly?) lower to `roundevenf`
# and `roundeven` on x86_64 macOS.
#
# However these functions are glibc specific so the test fails:
# - https://www.gnu.org/software/gnulib/manual/html_node/roundevenf.html
# - https://www.gnu.org/software/gnulib/manual/html_node/roundeven.html
#
# TODO(@rrbutani): this seems to run fine on `aarch64-darwin`, why does it
# pass there?
substituteInPlace test/ExecutionEngine/Interpreter/intrinsics.ll \
--replace "%roundeven32 = call float @llvm.roundeven.f32(float 0.000000e+00)" "" \
--replace "%roundeven64 = call double @llvm.roundeven.f64(double 0.000000e+00)" ""
# This test fails on darwin x86_64 because `sw_vers` reports a different
# macOS version than what LLVM finds by reading
# `/System/Library/CoreServices/SystemVersion.plist` (which is passed into
# the sandbox on macOS).
#
# The `sw_vers` provided by nixpkgs reports the macOS version associated
# with the `CoreFoundation` framework with which it was built. Because
# nixpkgs pins the SDK for `aarch64-darwin` and `x86_64-darwin` what
# `sw_vers` reports is not guaranteed to match the macOS version of the host
# that's building this derivation.
#
# Astute readers will note that we only _patch_ this test on aarch64-darwin
# (to use the nixpkgs provided `sw_vers`) instead of disabling it outright.
# So why does this test pass on aarch64?
#
# Well, it seems that `sw_vers` on aarch64 actually links against the _host_
# CoreFoundation framework instead of the nixpkgs provided one.
#
# Not entirely sure what the right fix is here. I'm assuming aarch64
# `sw_vers` doesn't intentionally link against the host `CoreFoundation`
# (still digging into how this ends up happening, will follow up) but that
# aside I think the more pertinent question is: should we be patching LLVM's
# macOS version detection logic to use `sw_vers` instead of reading host
# paths? This *is* a way in which details about builder machines can creep
# into the artifacts that are produced, affecting reproducibility, but it's
# not clear to me when/where/for what this even gets used in LLVM.
#
# TODO(@rrbutani): fix/follow-up
substituteInPlace unittests/Support/Host.cpp \
--replace "getMacOSHostVersion" "DISABLED_getMacOSHostVersion"
# This test fails with a `dysmutil` crash; have not yet dug into what's
# going on here (TODO(@rrbutani)).
rm test/tools/dsymutil/ARM/obfuscated.test
'' + ''
# FileSystem permissions tests fail with various special bits
substituteInPlace unittests/Support/CMakeLists.txt \
Expand Down

0 comments on commit eafb8fb

Please sign in to comment.