CMake build failure on macOS: __libcheck.c:4:10: fatal error: 'err.h' file not found

[karkhaz]

[Edit] Current status: the workaround to this bug is to set the absolute path to the compiler when configuring CBMC, like this:

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

---

original bug report follows

---

CBMC fails to build with CMake on macOS. For some reason, building using the Makefile in the src directory works.

I've been unable to find a commit that broke this, and I also recently updated my system (both macOS itself and also the command-line tools). So I'm wondering if that was to blame. I haven't been able to find any information online about moved headers, though. err.h still exists:

> find /Library/Developer -name err.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/err.h

but I don't have an older system to compare against.

Command:

> rm -rf build/*
> cmake -H. -Bbuild -GNinja -DWITH_JBMC=0 -DCMAKE_CXX_FLAGS="-fcolor-diagnostics"
> cmake --build build
# ...
[18/803] Generating library-check.stamp
FAILED: src/ansi-c/library-check.stamp
cd /Users/karkhaz/cbmc/src/ansi-c && /Users/karkhaz/cbmc/src/ansi-c/library_check.sh /Library/Developer/CommandLineTools/usr/bin/clang /Users/karkhaz/cbmc/src/ansi-c/library/cegis.c /Users/karkhaz/cbmc/src/ansi-c/library/ctype.c /Users/karkhaz/cbmc/src/ansi-c/library/err.c /Users/karkhaz/cbmc/src/ansi-c/library/errno.c /Users/karkhaz/cbmc/src/ansi-c/library/fcntl.c /Users/karkhaz/cbmc/src/ansi-c/library/fenv.c /Users/karkhaz/cbmc/src/ansi-c/library/float.c /Users/karkhaz/cbmc/src/ansi-c/library/gcc.c /Users/karkhaz/cbmc/src/ansi-c/library/getopt.c /Users/karkhaz/cbmc/src/ansi-c/library/inet.c /Users/karkhaz/cbmc/src/ansi-c/library/intrin.c /Users/karkhaz/cbmc/src/ansi-c/library/locale.c /Users/karkhaz/cbmc/src/ansi-c/library/math.c /Users/karkhaz/cbmc/src/ansi-c/library/netdb.c /Users/karkhaz/cbmc/src/ansi-c/library/noop.c /Users/karkhaz/cbmc/src/ansi-c/library/process.c /Users/karkhaz/cbmc/src/ansi-c/library/pthread_lib.c /Users/karkhaz/cbmc/src/ansi-c/library/semaphore.c /Users/karkhaz/cbmc/src/ansi-c/library/setjmp.c /Users/karkhaz/cbmc/src/ansi-c/library/signal.c /Users/karkhaz/cbmc/src/ansi-c/library/stdio.c /Users/karkhaz/cbmc/src/ansi-c/library/stdlib.c /Users/karkhaz/cbmc/src/ansi-c/library/string.c /Users/karkhaz/cbmc/src/ansi-c/library/strings.c /Users/karkhaz/cbmc/src/ansi-c/library/syslog.c /Users/karkhaz/cbmc/src/ansi-c/library/time.c /Users/karkhaz/cbmc/src/ansi-c/library/unistd.c /Users/karkhaz/cbmc/src/ansi-c/library/windows.c /Users/karkhaz/cbmc/src/ansi-c/library/x86_assembler.c && /usr/local/Cellar/cmake/3.15.0/bin/cmake -E touch /Users/karkhaz/cbmc/build/src/ansi-c/library-check.stamp
Checking /Users/karkhaz/cbmc/src/ansi-c/library/cegis.c
Checking /Users/karkhaz/cbmc/src/ansi-c/library/ctype.c
Checking /Users/karkhaz/cbmc/src/ansi-c/library/err.c
__libcheck.c:4:10: fatal error: 'err.h' file not found
#include <err.h>
         ^~~~~~~
1 error generated.
clang: error: no such file or directory: '__libcheck.i'
clang: error: no input files
rm: __libcheck.s: No such file or directory
rm: __libcheck.i: No such file or directory
[21/803] Building CXX object src/util/CMakeFiles/util.dir/nondet.cpp.o
ninja: build stopped: subcommand failed.

My command-line tools:

> pkgutil --pkg-info=com.apple.pkg.CLTools_Executables
package-id: com.apple.pkg.CLTools_Executables
version: 10.3.0.0.1.1562985497
volume: /
location: /
install-time: 1564051272
groups: com.apple.FindSystemFiles.pkg-group

Operating system is macOS 10.14.6. Commit is 306697d6, but I've tried starting a bisect waaaaay in the past (d201bad22) and this still shows up even then, so I'm guessing it's an OS issue.

[tautschnig]

It seems that /Library/Developer/CommandLineTools/usr/bin/clang has different behaviour than using /usr/bin/clang. The latter processes a file containing just #include <err.h> just fine, the former fails in the way that you describe.

[tautschnig]

Running both commands with -v is actually pretty revealing. It seems /Library/Developer/CommandLineTools/usr/bin/clang has basically nothing in its search path. I'm not sure this is actually meant to be invoked directly?

[karkhaz]

Thank you! Confirming that this works:

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

You're right about the search path. I don't know why the /Library clang is being invoked, it's not on my $PATH, and which seems to find the right one...

> which clang
/usr/bin/clang
> which clang++
/usr/bin/clang++

Is the fix here to hardcode the /usr/bin path in CMakeLists.txt if we detect that we're running on macOS? That seems ugly, I can't think of anything else, and I also can't discount that this is a unique problem on my laptop. Perhaps I could close this and re-open if anybody else has a similar problem?

[tautschnig]

Maybe from your CMake logs you can figure out how it came to pick that path?

[karkhaz]

The logs aren't that helpful, they mention what compiler was used but don't really give a clue to why. I'm now wondering if CMake was upgraded together with my command-line tools? My CMakeVersion is 3.15.0, perhaps a new version of CMake introduced a bug where it looks in the wrong place?

The system is: Darwin - 18.7.0 - x86_64
Compiling the C compiler identification source file "CMakeCCompilerId.c" succeeded.
Compiler: /Library/Developer/CommandLineTools/usr/bin/clang
Build flags:
Id flags:

The output was:
0


Compilation of the C compiler identification source "CMakeCCompilerId.c" produced "a.out"

The C compiler identification is AppleClang, found in "/Users/karkhaz/cbmc/build/CMakeFiles/3.15.0/CompilerIdC/a.out"

[karkhaz]

Closing this, as I haven't made progress and it may well be a unique problem with my machine.

[dlohar]

I have the same issue on MacOS 10.14.5 with CMake 3.15.1. I have the "err.h" file in the exact same location as mentioned here.

[karkhaz]

Confirming that this error exhibits on a brand-new machine running macOS 10.14.6. I still don't understand why.

The workaround is still to build using

> cmake -DCMAKE_C_COMPILER=/usr/bin/clang ...

at some point, I'll try to change the CMake files to hardcode this path to clang if the OS version is >= 10.14.5. Unless somebody figures out what the actual problem is, but I'm pretty flummoxed. I'm also not ruling out that it's a recent CMake bug rather than a macOS Clang bug; I haven't yet bisected cmake or clang, so I'll get round to that at some stage.

(if it's a CMake or upstream Clang problem, then I could at least theoretically fix it. Whereas if it's a problem with how Apple are building the Clang that they ship, then it's hopeless anyway. Whether I get time to fix this is a different matter...)

[karkhaz]

I haven't had time to look into this, but I'll point out that I've been ramping up new CBMC users over the past few months, and they've all experienced this issue

[karkhaz]

@chrisr-diffblue this is the macOS build issue that I mentioned.

[chrisr-diffblue]

Thanks @karkhaz - yes, this is the same behaviour I've found. I've not entirely figured out what happens here, but I believe it is an issue with Apple's developer tools, possibly when you have a combination of CommandLineTools and XCode installed, and how Apple handle their system frameworks. I use a similar workaround (-DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang where /usr/bin is on the PATH first). I'll see if I can figure out a proper fix though.

[chrisr-diffblue]

I'm told that on macOS 10.14 the following command will resolve the issue:
sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /
I'm currently on macOS 10.15 (so I can't test the above - but @peterschrammel has used that command), and there doesn't appear to be a separate headers pkg like that for 10.15. So for 10.15 the solution still seems to be making sure /usr/bin/clang++ and /usr/bin/clang are both specified.

[andyp1per]

I am told that something like sudo ln -s `xcrun --show-sdk-path`/usr/include /usr/local/include might fix. But has not worked for me so far!

[chrisr-diffblue]

Yuck :-) I really don't think adding symlinks should be the way to go :-)

[andyp1per]

Or maybe
export CPATH=`xcrun --show-sdk-path`/usr/include

[hannes-steffenhagen-diffblue]

Thanks for adding the information to the documentation. I don’t believe there is much we can or should be doing this outside of documenting this behaviour though, as this is an issue with how CMake detects compilers and frameworks and as far as I can tell isn’t directly related to cbmc.

@karkhaz would you be OK with closing this on those grounds?

[hannes-steffenhagen-diffblue]

Tracking this internally as ADA-565 – currently not planning to do anything about it as it appears to be an issue with CMake itself, not with anything in CBMC.

[hannes-steffenhagen-diffblue]

Closing this for now – this is an environmental issue that we can’t really do anything about on our end, and it’s been documented.