zig cc: unsupported darwin linker args

[opti-mix]

Thanks a lot for adding zig cc and zig c++ functionality. It is really useful.

I tried to use zig c++ instead of the standard LLVM toolchain for one of my projects. I build it on MacOS. In general, it mostly works, but I've discovered the following issues:

1. There is a number of flags passed to the linker using -Wl,my_linker_option. They are important for proper linking of the project. But some of them are not supported yet.

warning: unsupported linker arg: -search_paths_first
warning: unsupported linker arg: -headerpad_max_install_names
warning: unsupported linker arg: -force_load
warning: unsupported linker arg: -force_load

2. If I configure my CMake-based project to use make and then run make, it seems to work fine as long as make is using just a single thread. But if I use e.g. make -j 2 I start seeing some issues:

zig -c test.cpp -std=c++14 -ffast-math -fno-finite-math-only -g -I/path/to/include -emit-llvm -O0 -o test.o
Invalid argument: -c

It seems as if there is a race condition somewhere. As you can seen in zig's command line it misses c++ after zig somehow.

3. If I configure my CMake-based project to use ninja and then run ninja, I see issues with dependency tracking supported by clang. Specifically, there seems to be an issue with handling of -MF flags (see below). Same project can be build just fine if I use the vanilla LLVM toolchain.

zig: error: no such file or directory: 'test.cpp.o.d'

The following command failed when running ninja:
zig clang -c -MD -MV -MF zig-cache/tmp/eamNMvLXEZYC-test.o.d  -MD -MT -MF test.cpp.o.d -o zig-cache/tmp/eamNMvLXEZYC-test.o test.cpp

It would be nice if these issues could be resolved.

Please let me know if I can provide any additional information that would allow to fix these issues.

[opti-mix]

Oh, one more thing I forgot to mention:

4. If I use CXX="zig c++" cmake it runs into some problems for some targets that play with the compiler name. It seems like cmake does not correctly handle the space between zig and c++ in some cases. So, I created a small shell-wrapper called zigc++ which looks like this:

#!/bin/bash
zig c++ $@

If I then use it like CXX="zigc++" cmake, then the issue disappears. So, maybe it is a good idea to provide such wrappers for zig cc and zig c++ as part of Zig pre-built binaries?

[andrewrk]

zig -c test.cpp -std=c++14 -ffast-math -fno-finite-math-only -g -I/path/to/include -emit-llvm -O0 -o test.o
Invalid argument: -c

you're missing c++ here. zig c++ -c ...

edit: oops, I see that you noticed this. can you confirm that it's a problem on zig's end though?

[opti-mix]

edit: oops, I see that you noticed this. can you confirm that it's a problem on zig's end though?

The thing is: if I use a single-threaded make, it does not happen. It seems to happen only with make -j N, where N > 1. Which indicates that command-line options are passed correctly to zig in a single-threaded make.

[andrewrk]

You could verify this with strace -f -e trace=execve make ... and then see what argv is being passed to zig

[opti-mix]

You could verify this with strace -f -e trace=execve make ... and then see what argv is being passed to zig

Seems like it is a make issue, because make dumps a wrong command line in verbose mode. I tried to use the zigc++ wrapper instead of zig c++ and the issue disappears. Seems like the whitespace between zig and c++ is an issue for CMake or make or both of them.

[andrewrk]

(2) is a make issue as noted above
(3) is solved in e4edc6d
(4) there is no plan to do this; I consider that to be out of scope of the project

As for (1) I am repurposing this issue to address it. If you run into any more independent problems, let's handle those in new issues.

[andrewrk]

Alright so the remaining issue has to do with the fact that Zig uses LLD as the linker, and until very recently the mach-o LLD code was unmaintained. It does not support these flags.

However now I expect this new team to quickly bring LLD up to speed for mach-o, and so I expect this issue to be resolved with the release of LLVM 11 (in 6 months).

[opti-mix]

@andrewrk BTW, I was about to download the latest builds from https://ziglang.org/download/#release-master, but it seems to be not updated since yesterday for some reason. Don't know if it is expected or not.

[opti-mix]

Alright so the remaining issue has to do with the fact that Zig uses LLD as the linker, and until very recently the mach-o LLD code was unmaintained. It does not support these flags.

I don't quite understand this comment. Couldn't Zig just pass the provided -Wl,my_linker_option to a linker "as is", without doing anything about these flags?

Or do you say that the error messages I see are from the LLD linker and not from Zig?

[opti-mix]

(2) is a make issue as noted above
(4) there is no plan to do this; I consider that to be out of scope of the project

Regarding (2) and (4), my impression it is essentially the same problem, which is a space between zig and cc or c++. I don't have a strong opinion here, but IMHO if this is something that makes use of zig with make/cmake as a C/C++ compiler problematic, it would be nice to have a standard solution for that (e.g. a simple wrapper), especially if it is so trivial. After all, most people use some build systems to build their projects and ideally they shouldn't need any custom workarounds to use Zig. But the decision about this topic is of course yours.

(3) is solved in e4edc6d

I'll test and report back ASAP.

[andrewrk]

@andrewrk BTW, I was about to download the latest builds from https://ziglang.org/download/#release-master, but it seems to be not updated since yesterday for some reason. Don't know if it is expected or not.

Thanks for letting me know. Fixed in ziglang/www.ziglang.org@88683aa (had to update the update-download-page.zig script to the latest zig std lib after making breaking changes).

[andrewrk]

I don't quite understand this comment. Couldn't Zig just pass the provided -Wl,my_linker_option to a linker "as is", without doing anything about these flags?

a linker

Which one?

[opti-mix]

Which one?

E.g. -Wl,-search_paths_first, -Wl,-force-load, -Wl,-headerpad_max_install_names

[andrewrk]

sorry, I meant, which linker?

[opti-mix]

sorry, I meant, which linker?

Clang on MacOS uses /usr/bin/ld if it is asked to link an executable. So, I assumed Zig would do the same.

[andrewrk]

The macOS linker /usr/bin/ld is not available on non-macOS systems, so relying on it would prevent cross-compiling for macOS. Also /usr/bin/ld is not always available on macOS systems, for example if xcode command line tools have not been installed. This is why Zig uses LLD as the linker, rather than depending on a system linker, and it means that zig cc is an alternative to installing the xcode command line tools. All of that relies on LLD working well on macOS however, which has not yet been the case. But finally, LLD's macos support is seeing some love, and I expect this situation to change during this release cycle of LLVM.

[opti-mix]

Ah, OK. Thanks for the clarifications regarding those linker issues! I cross my fingers that LLD's macos support will be improved soon :-)

[opti-mix]

Quick question: Have you considered adding something like -fuse-ld=/path/to/your/linker, which is supported by clang. This could be useful for using non-standard linkers in case it is needed.

[opti-mix]

(3) is solved in e4edc6d

@andrewrk I tried with the latest build, to check if -MF, -MT and -MD flags are now processed properly.

I reproduced with a very simple simple.cpp file, which just prints "Hello World!".

zig c++ -MD -MT simple.cpp.o -MF simple.cpp.o.d -o simple.o -c simple.cpp

And I get
zig: error: no such file or directory: 'simple.cpp.o.d'

The following command failed:
~/src/zig-macos-x86_64-0.5.0+b5526d0b3/zig clang -c -nostdinc -fno-spell-checking -isystem ~/src/zig-macos-x86_64-0.5.0+b5526d0b3/lib/zig/libcxx/include -D_LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS -D_LIBCXXABI_DISABLE_VISIBILITY_ANNOTATIONS -target x86_64-unknown-macosx-gnu -isystem ~/src/zig-macos-x86_64-0.5.0+b5526d0b3/lib/zig/include -isystem /usr/include -Xclang -target-cpu -Xclang skylake -Xclang -target-feature -Xclang -3dnow,-3dnowa,+64bit,+adx,+aes,+avx,+avx2,-avx512bf16,-avx512bitalg,-avx512bw,-avx512cd,-avx512dq,-avx512er,-avx512f,-avx512ifma,-avx512pf,-avx512vbmi,-avx512vbmi2,-avx512vl,-avx512vnni,-avx512vp2intersect,-avx512vpopcntdq,+bmi,+bmi2,-branchfusion,-cldemote,+clflushopt,-clwb,-clzero,+cmov,+cx16,+cx8,-enqcmd,+ermsb,+f16c,-false-deps-lzcnt-tzcnt,+false-deps-popcnt,-fast-11bytenop,-fast-15bytenop,-fast-bextr,+fast-gather,-fast-hops,-fast-lzcnt,+fast-scalar-fsqrt,-fast-scalar-shift-masks,+fast-shld-rotate,+fast-variable-shuffle,+fast-vector-fsqrt,-fast-vector-shift-masks,+fma,-fma4,+fsgsbase,+fxsr,-gfni,-idivl-to-divb,+idivq-to-divl,+invpcid,-lea-sp,-lea-uses-ag,-lwp,+lzcnt,+macrofusion,+merge-to-threeway-branch,+mmx,+movbe,-movdir64b,-movdiri,-mpx,-mwaitx,+nopl,-pad-short-functions,+pclmul,-pconfig,-pku,+popcnt,-prefer-128-bit,-prefer-256-bit,-prefer-mask-registers,-prefetchwt1,+prfchw,-ptwrite,-rdpid,+rdrnd,+rdseed,-retpoline,-retpoline-external-thunk,-retpoline-indirect-branches,-retpoline-indirect-calls,+rtm,+sahf,+sgx,-sha,-shstk,+slow-3ops-lea,-slow-incdec,-slow-lea,-slow-pmaddwd,-slow-pmulld,-slow-shld,-slow-two-mem-ops,-slow-unaligned-mem-16,-slow-unaligned-mem-32,-soft-float,+sse,-sse-unaligned-mem,+sse2,+sse3,+sse4.1,+sse4.2,-sse4a,+ssse3,-tbm,-use-aa,-use-glm-div-sqrt-costs,-vaes,-vpclmulqdq,+vzeroupper,-waitpkg,-wbnoinvd,+x87,-xop,+xsave,+xsavec,+xsaveopt,+xsaves -fno-omit-frame-pointer -fsanitize=undefined -fsanitize-trap=undefined -D_DEBUG -Og -fstack-protector-strong --param ssp-buffer-size=4 -fPIC -MT -MF simple.cpp.o.d -o zig-cache/tmp/k0VTvsQBoTOp-simple.o simple.cpp

If I try a slightly different command-line without -MT simple.cpp.o, then there is no failure:
zig c++ -MD -MF simple.cpp.o.d -o simple.o -c simple.cpp

[andrewrk]

Thanks for the report. Solved in f8cc6a1.

[andrewrk]

Quick question: Have you considered adding something like -fuse-ld=/path/to/your/linker, which is supported by clang. This could be useful for using non-standard linkers in case it is needed.

This is a big topic, too big to answer as a quick question

[opti-mix]

Thanks for the report. Solved in f8cc6a1.

Great! I can confirm that -MT issues are solved now. My project can be build almost correctly, except for the -Wl,-force-load issue mentioned before, which results in the wrongly linked executables missing some parts.

[andrewrk]

Good to know. Let's revisit the linker issues once LLVM 11 is a bit further along in the release cycle.

[Mouvedia]

LLVM 11 seems to have be have been released in October. Is this fixed?

[kubkon]

I can shed some more light on (1):

There is a number of flags passed to the linker using -Wl,my_linker_option. They are important for proper linking of the project. But some of them are not supported yet.
warning: unsupported linker arg: -search_paths_first
warning: unsupported linker arg: -headerpad_max_install_names
warning: unsupported linker arg: -force_load
warning: unsupported linker arg: -force_load

All of the flags with the exception of force_load are not currently supported by LLD, however, there is an ongoing effort in LLVM to replace now dated Mach-O backend with a new one that supports these options. Having said that, LLVM11 still uses the old backend by default, and while it is technically possible to switch to the new one already, it currently fails to even build Zig on x86_64 (aarch64 support is not even started). FYI currently supported flags by the (old) backend are listed here.

While not a permanent fix, it is now possible to fall back to the system linker on macOS by setting export ZIG_SYSTEM_LINKER_HACK=1 env var before linking.

[mikdusan]

warning: unsupported linker arg: -search_paths_first
warning: unsupported linker arg: -headerpad_max_install_names
warning: unsupported linker arg: -force_load

These linker args are now supported by zld.

$ sw_vers
ProductName:		macOS
ProductVersion:		13.1
BuildVersion:		22C65
$ zig version
0.11.0-dev.1023+1c711b0a6
$ zig c++ -o main main.cpp -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-force_load -Wl,/opt/local-2022.10/lib/libz.a -v
MachO Flush... zig ld -dynamic -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX13.1.sdk -search_paths_first -headerpad_max_install_names /opt/local-2022.10/lib/libz.a /Users/mike/project/zig/work/main/zig-cache/o/a8ee99607d7d9ab78e53561974b39bdd/main.o /Users/mike/project/zig/work/main/zig-cache/o/d56d9e695295e64a617c9618d425bcc5/libcompiler_rt.a /Users/mike/project/zig/work/main/zig-cache/o/1c116bd4f79c02580e72b1128a68a868/libc++abi.a /Users/mike/project/zig/work/main/zig-cache/o/5b02b02b8b0a4913d1f1edf075b37d0c/libc++.a -o main -lSystem -lc -L/usr/lib -F/System/Library/Frameworks -force_load /opt/local-2022.10/lib/libz.a