Linker error for atomic functions on aarch64-linux #201254

avdv · 2022-11-14T22:18:37Z

Describe the bug

When using some atomic functions on aarch64 linux fails during the linking phase when using the -z defs linker option (as observed while trying to compile bobcat on aarch64-linux):

/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: /build/test.o: in function `foo(int)':
zzfr0vqndwngbh5b2skgdgl90aqqrpwl-test.cpp:(.text+0x2c): undefined reference to `__aarch64_ldadd4_acq_rel'
collect2: error: ld returned 1 exit status

Steps To Reproduce

Steps to reproduce the behavior:

save the following nix code into fail.nix
on an aarch64 Linux system, run nix-build -I nixpkgs=PATH_TO_NIXPKGS fail.nix

{ pkgs ? import <nixpkgs> {} }:

pkgs.gcc10Stdenv.mkDerivation {
  name = "link failure on aarch64";
  version = "0.1";

  dontUnpack = true;

  src = pkgs.writeText "test.cpp" ''
      #include <atomic>

      int foo(int i) {
         std::atomic<int> f(i);

         return f.fetch_add(3);
      }
  '';

  buildPhase = ''
    g++ --std=c++20 -c $src -o $TMP/test.o
    g++ --std=c++20 -shared -Wl,-z,defs $TMP/test.o -o libtest.so 
  '';
}

Expected behavior

Linking should succeed.

Additional context

According to https://bugzilla.redhat.com/show_bug.cgi?id=1830472

On aarch64 -moutline-atomics has been turned on by default, and those symbols are solely in libgcc.a, not in libgcc_s.so.*.

Indeed libgcc_s.so from gcc-lib is a linker script:

$ cat /nix/store/q2nal9l25ff4zvd99l2ifjnpaw1ij21n-gcc-10.4.0-lib/lib/libgcc_s.so
/* GNU ld script
   Use the shared library, but some functions are only in
   the static library.  */
GROUP ( libgcc_s.so.1 -lgcc )

Adding the -v flag to the invocation of g++ one can observe this call to collect2:

/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/libexec/gcc/aarch64-unknown-linux-gnu/10.4.0/collect2 \
-plugin \ 
/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/libexec/gcc/aarch64-unknown-linux-gnu/10.4.0/liblto_plugin.so \
-plugin-opt=/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/libexec/gcc/aarch64-unknown-linux-gnu/10.4.0/lto-wrapper \
-plugin-opt=-fresolution=/run/user/1000/ccYTLoAU.res \
-plugin-opt=-pass-through=-lgcc_s \
-plugin-opt=-pass-through=-lc \                                                                                                                                                  
-plugin-opt=-pass-through=-lgcc_s \                                                     
--eh-frame-hdr \
-shared \
-X \
-EL \
-maarch64linux \
-o \
libtest.so \
/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib/crti.o \
/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/lib/gcc/aarch64-unknown-linux-gnu/10.4.0/crtbeginS.o \
-L/nix/store/dwk83gbvz3dfzm7kk18lrphsbjn8qxq0-libmilter-8.15.2/lib \
-L/nix/store/bl0wv9qaj5ws3d1j9hl983qlpghx334f-libxcb-1.14/lib \
-L/nix/store/0gx2fkm4ndif74l9vaw2gi8ffm8vqp84-libX11-1.8.1/lib \
-L/nix/store/l95ixfjbvi6wj1il5a062b8sprcc1m3a-openssl-3.0.7/lib \
-L/nix/store/c51zf8aa3zna0qzjbyl2ac3j2q9nns27-ncurses-6.3-p20220507/lib \
-L/nix/store/cbdlrdy51iiszglwyq96q8dk3yg725hs-readline-6.3p08/lib \
-L/nix/store/94qlip4nc7c8kwvrrwav3sc008maxwhk-util-linux-2.38.1-lib/lib \
-L/nix/store/dwk83gbvz3dfzm7kk18lrphsbjn8qxq0-libmilter-8.15.2/lib \
-L/nix/store/bl0wv9qaj5ws3d1j9hl983qlpghx334f-libxcb-1.14/lib \
-L/nix/store/0gx2fkm4ndif74l9vaw2gi8ffm8vqp84-libX11-1.8.1/lib \
-L/nix/store/l95ixfjbvi6wj1il5a062b8sprcc1m3a-openssl-3.0.7/lib \
-L/nix/store/c51zf8aa3zna0qzjbyl2ac3j2q9nns27-ncurses-6.3-p20220507/lib \
-L/nix/store/cbdlrdy51iiszglwyq96q8dk3yg725hs-readline-6.3p08/lib \
-L/nix/store/94qlip4nc7c8kwvrrwav3sc008maxwhk-util-linux-2.38.1-lib/lib \
-L/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib \
-L/nix/store/vqlcm9k500r8wsnam86rlan95yxnalxa-gcc-10.4.0-lib/lib \
-L/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib \
-L/nix/store/vqlcm9k500r8wsnam86rlan95yxnalxa-gcc-10.4.0-lib/lib \
-L/nix/store/zk5rgbk1nrcy6r6fafamfqnnsc59z9n8-gcc-wrapper-10.4.0/bin \
-L/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/lib/gcc/aarch64-unknown-linux-gnu/10.4.0 \
-L/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/lib/gcc/aarch64-unknown-linux-gnu/10.4.0/../../../../lib64 \
-L/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/lib/gcc/aarch64-unknown-linux-gnu/10.4.0/../../.. \
-dynamic-linker=/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib/ld-linux-aarch64.so.1 \
-z \
defs \
test.o \
-rpath \
/nix/store/bspxp2j6ggc7v5an4s3qky6k6h6fw5gj-bobcat-5.10.01/lib \
-lstdc++ \
-lm \
-lgcc_s \
-lc \
-lgcc_s \
/nix/store/rvjj7j3hl1zm3xgy9vh8wck5r7cqp0py-gcc-10.4.0/lib/gcc/aarch64-unknown-linux-gnu/10.4.0/crtendS.o \
/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib/crtn.o \

Removing the glibc lib directory from the library path (ie. removing -L/nix/store/g14swv4f0rg83naqj87g1mjmyvrmrxmf-glibc-2.35-163/lib), makes this call successful.

The root cause is that glibc contains a libgcc_so.1 library in its lib folder:

$ file /nix/store/xl52bf8jch51zzh84rszxn0hpbxadll4-glibc-2.35-163/lib/libgcc_s.so
/nix/store/xl52bf8jch51zzh84rszxn0hpbxadll4-glibc-2.35-163/lib/libgcc_s.so: symbolic link to libgcc_s.so.1

This is found before the right one from gcc, and this breaks linking when it would just work if libgcc_so.1 would be the proper linker script.

Why does glibc contain the libgcc_s.so library?

Notify maintainers

@edolstra @Ma27

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"aarch64-linux"`
 - host os: `Linux 5.15.74-v8+, Debian GNU/Linux, 11 (bullseye), nobuild`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.11.1`
 - channels(claudio): `"nixpkgs"`
 - nixpkgs: `/home/claudio/.nix-defexpr/channels/nixpkgs`

The text was updated successfully, but these errors were encountered:

avdv · 2022-11-15T18:14:29Z

BTW, this also makes building llvm fail on aarch64-linux:

...
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x2e4): undefined reference to `__aarch64_ldadd8_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x390): undefined reference to `__aarch64_ldadd4_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x3a0): undefined reference to `__aarch64_ldadd4_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x3b0): undefined reference to `__aarch64_ldadd4_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x3d8): undefined reference to `__aarch64_ldadd4_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt10unique_ptrINS0_29MaterializationResponsibil
ityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x3e8): undefined reference to `__aarch64_ldadd4_acq_rel'                                                                 
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: ../../lib/libLLVMOrcJIT.a(Speculation.cpp.o):Speculation.cpp:(.text._ZN4llvm3orc18IRSpeculationLayer4emitESt
10unique_ptrINS0_29MaterializationResponsibilityESt14default_deleteIS3_EENS0_16ThreadSafeModuleE+0x3f8): more undefined references to `__aarch64_ldadd4_acq_rel' follow       
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: ../../lib/libLLVMOrcJIT.a(Speculation.cpp.o): in function `llvm::orc::Speculator::launchCompile(unsigned lon
g)':                                                                                                                                                                          
Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileEm[_ZN4llvm3orc10Speculator13launchCompileEm]+0x164): undefined reference to `__aarch64_ldadd8_acq_rel'         
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileEm[_ZN4llvm3orc10Speculator13launchCompileEm]+
0x200): undefined reference to `__aarch64_ldadd8_acq_rel'                                                                                                                     
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileEm[_ZN4llvm3orc10Speculator13launchCompileEm]+
0x2d8): undefined reference to `__aarch64_ldadd8_acq_rel'                                                                                                                     
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileEm[_ZN4llvm3orc10Speculator13launchCompileEm]+
0x37c): undefined reference to `__aarch64_ldadd8_acq_rel'                                                                                                                     
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileEm[_ZN4llvm3orc10Speculator13launchCompileEm]+
0x5b8): undefined reference to `__aarch64_ldadd8_acq_rel'                                                                                                                     
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: ../../lib/libLLVMOrcJIT.a(Speculation.cpp.o):Speculation.cpp:(.text._ZN4llvm3orc10Speculator13launchCompileE
m[_ZN4llvm3orc10Speculator13launchCompileEm]+0x680): more undefined references to `__aarch64_ldadd8_acq_rel' follow                                                           
/nix/store/1g5g7vzlcprmg1d6864fs5r4nf1h5b5j-binutils-2.39/bin/ld: ../../lib/libLLVMOrcJIT.a(ExecutorProcessControl.cpp.o): in function `void llvm::detail::UniqueFunctionBase<
void, llvm::orc::shared::WrapperFunctionResult>::CallImpl<llvm::orc::SelfExecutorProcessControl::jitDispatchViaWrapperFunctionManager(void*, void const*, char const*, unsigne
d long)::{lambda(llvm::orc::shared::WrapperFunctionResult)#1}>(void*, llvm::orc::shared::WrapperFunctionResult&)':                                                            
ExecutorProcessControl.cpp:(.text._ZN4llvm6detail18UniqueFunctionBaseIvJNS_3orc6shared21WrapperFunctionResultEEE8CallImplIZNS2_26SelfExecutorProcessControl36jitDispatchViaWra
pperFunctionManagerEPvPKvPKcmEUlS4_E_EEvS8_RS4_+0x110): undefined reference to `__aarch64_swp4_rel'

…r-rt and clang This is required to workaround #201254.

trofi · 2023-01-03T17:40:04Z

It's a long-standing build problem in nixpkgs bootstrap process. We'll try to address it in #208412.

This change switches to using GCC 11 by default on aarch64-linux, as well as passing `-lgcc` to the linker, per NixOS#201485. See NixOS#201254 and NixOS#208412 for wider context on the issue.

This is required to workaround #201254

treewide: mark #201254

This change switches to using GCC 11 by default on aarch64-linux, as well as passing `-lgcc` to the linker, per NixOS#201485. See NixOS#201254 and NixOS#208412 for wider context on the issue. (cherry picked from commit 8442601)

This is required to workaround NixOS#201254 (cherry picked from commit ba3db3e)

bew · 2023-04-07T14:58:40Z

PR #209870 was marked to close this issue. Since that PR is now merged, this can be closed?

lovesegfault · 2023-04-07T15:13:51Z

We should wait until it hits master, it's still in staging.

bew · 2023-04-07T15:19:42Z

Oh you're right
On the PR, all other related issues/PR are already closed, so I thought if was just a mistake that this one isn't.

avdv added the 0.kind: bug Something is broken label Nov 14, 2022

avdv mentioned this issue Nov 16, 2022

llvmPackages_14: Fix build on aarch64-linux #201485

Merged

13 tasks

This was referenced Nov 27, 2022

root: fix build on aarch64-linux #203267

Merged

ladybird: fix build #203532

Merged

ehmry pushed a commit that referenced this issue Nov 30, 2022

llvmPackages_14: Add -lgcc to NIX_LDFLAGS to lld, lldb, llvm, compile…

38c793c

…r-rt and clang This is required to workaround #201254.

wegank mentioned this issue Jan 4, 2023

rust: bring back 1.65 for aarch64-linux #209078

Closed

13 tasks

winterqt mentioned this issue Jan 4, 2023

rust: fix on aarch64-linux by using GCC 11 and passing -lgcc #209113

Merged

13 tasks

This was referenced Jan 7, 2023

aarch64: enable rebootstrap, upgrade to gcc11 #209462

Closed

stdenv: Nix-driven bootstrap of gcc #209870

Merged

glibc: add enableCopyLibGccHack #210112

Closed

wegank mentioned this issue Jan 15, 2023

vte: migrate to gcc12Stdenv #210979

Closed

13 tasks

misuzu mentioned this issue Jan 17, 2023

zerotierone: add -lgcc to NIX_LDFLAGS #211275

Merged

13 tasks

flokli pushed a commit that referenced this issue Jan 18, 2023

zerotierone: add -lgcc to NIX_LDFLAGS

ba3db3e

This is required to workaround #201254

NickCao mentioned this issue Jan 22, 2023

mtxclient, nheko: fix build on aarch64-linux #212051

Merged

13 tasks

wegank mentioned this issue Jan 22, 2023

rpm-ostree: 2022.13 -> 2023.1 #196164

Closed

wegank added a commit to wegank/nixpkgs that referenced this issue Jan 26, 2023

treewide: mark NixOS#201254

52a2870

wegank added a commit that referenced this issue Jan 26, 2023

Merge pull request #212714 from wegank/aarch64-gcc-comments

dcd69bc

treewide: mark #201254

wegank mentioned this issue Jan 30, 2023

python310Packages.identify: 2.5.16 -> 2.5.17 #213634

Merged

13 tasks

This was referenced Feb 6, 2023

amarok: remove unused arguments #214953

Merged

suricata: 6.0.8 -> 6.0.10 #215766

Merged

xanderio pushed a commit to xanderio/nixpkgs that referenced this issue Feb 13, 2023

treewide: mark NixOS#201254

96b6c8a

winterqt pushed a commit to winterqt/nixpkgs that referenced this issue Feb 15, 2023

zerotierone: add -lgcc to NIX_LDFLAGS

2097723

This is required to workaround NixOS#201254 (cherry picked from commit ba3db3e)

This was referenced Mar 1, 2023

rust, firefox, spidermonkey: cleanup #219011

Merged

treewide: remove -lgcc from NIX_LDFLAGS #219425

Merged

Artturin added the 2.status: fix-in-staging This issue has been fixed in staging but is not yet in the channels or master label Apr 7, 2023

K900 closed this as completed Apr 11, 2023

K900 reopened this Apr 11, 2023

vcunat closed this as completed in 7553d0f Apr 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Linker error for atomic functions on aarch64-linux #201254

Linker error for atomic functions on aarch64-linux #201254

avdv commented Nov 14, 2022

avdv commented Nov 15, 2022

trofi commented Jan 3, 2023

bew commented Apr 7, 2023

lovesegfault commented Apr 7, 2023

bew commented Apr 7, 2023

Linker error for atomic functions on aarch64-linux #201254

Linker error for atomic functions on aarch64-linux #201254

Comments

avdv commented Nov 14, 2022

Describe the bug

Steps To Reproduce

Expected behavior

Additional context

Notify maintainers

Metadata

avdv commented Nov 15, 2022

trofi commented Jan 3, 2023

bew commented Apr 7, 2023

lovesegfault commented Apr 7, 2023

bew commented Apr 7, 2023