Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AddressSanitizer:DEADLYSIGNAL in getaddrinfo() when compiled with clang-15 with "-nodefaultlibs" without "-lresolv" #59007

Closed
mmatrosov opened this issue Nov 15, 2022 · 8 comments · Fixed by #119071
Labels
compiler-rt:msan Memory sanitizer

Comments

@mmatrosov
Copy link

mmatrosov commented Nov 15, 2022

The following code

#include <arpa/inet.h>
#include <netdb.h>
#include <stdio.h>

int main() {
  addrinfo hints = {};
  addrinfo* res;
  getaddrinfo("api.binance.com", nullptr, &hints, &res);
  printf("%s\n", inet_ntoa(reinterpret_cast<sockaddr_in*>(res->ai_addr)->sin_addr));
  freeaddrinfo(res);
  return 0;
}

compiled with

clang-15 -fsanitize=address test.cpp -nodefaultlibs -lgcc_eh -lc -lm -lpthread -ldl

produces the following error:

$ ./a.out
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1010333==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x7ffd91eac050 sp 0x7ffd91eab808 T0)
==1010333==Hint: pc points to the zero page.
==1010333==The signal is caused by a READ memory access.
==1010333==Hint: address points to the zero page.
    #0 0x0  (<unknown module>)
    #1 0x7f5bab6b6805 in gaih_getanswer_slice /build/glibc-SzIz7B/glibc-2.31/resolv/nss_dns/dns-host.c:1147:8

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (<unknown module>)
==1010333==ABORTING

If -lresolv option is added, it works as expected:

$ clang-15 -fsanitize=address test.cpp -nodefaultlibs -lgcc_eh -lc -lm -lpthread -ldl -lresolv -o b.out && ./b.out
13.32.142.177

It also works as expected if I use clang-14 instead of clang-15.

This is a very practical issue, because it is reproduced when using python's requests module with dynamically loaded sanitizer runtime (necessary when using sanitized python extensions):

$ LD_PRELOAD=$(clang++-15 -print-file-name=libclang_rt.asan-x86_64.so) python -c 'import requests
print(requests.get("https://api.binance.com/"))'
AddressSanitizer:DEADLYSIGNAL
=================================================================
==919440==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000000000000 bp 0x7ffe180677c0 sp 0x7ffe18066f78 T0)
==919440==Hint: pc points to the zero page.
==919440==The signal is caused by a READ memory access.
==919440==Hint: address points to the zero page.
    #0 0x0  (<unknown module>)
    #1 0x7f3048604805 in gaih_getanswer_slice /build/glibc-SzIz7B/glibc-2.31/resolv/nss_dns/dns-host.c:1147:8

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV (<unknown module>)
==919440==ABORTING

More info:

$ ldd ./a.out # this one crashes
	linux-vdso.so.1 (0x00007ffc6537a000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f77efcbf000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f77efb70000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f77efb4d000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f77efb47000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f77f0930000)

$ ldd ./b.out # this one does not crash
	linux-vdso.so.1 (0x00007ffde1385000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f15e3c18000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f15e3ac9000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f15e3aa6000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f15e3aa0000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f15e3a84000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f15e488c000)

$ clang-15 --version
Ubuntu clang version 15.0.4-++20221031075612+08bd84e8a635-1~exp1~20221031075700.87
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

$ python --version
Python 3.8.10

$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 20.04.5 LTS
Release:	20.04
Codename:	focal

I posted the same bug at google/sanitizers#1592, because I am not sure if it is related to sanitizers or clang-15.

@BrandonSchaefer
Copy link

I can confirm this as well. Built LLVM 15.0.1 against centos7, and compiling with a shared ASan from that hitting this on centos7, Ubuntu 18.04, and Ubuntu 20.04. Ubuntu 22.04 actually does not here for me but -lresolv does fixes the issue for me on these other OS versions.

@domidimi
Copy link

domidimi commented Dec 15, 2023

The lresolv dependency seems first added with [1]
[2] Fixes some cases by adding -lresolv conditionally
With conditionally I mean only for the static lib. For the dynamic library this would probably require changing
compiler-rt/lib/asan/CMakeLists.txt and adding the new dependency to ASAN_DYNAMIC_LIBS

1- https://reviews.llvm.org/D126851
2- https://reviews.llvm.org/D127145

@domidimi
Copy link

E.g. following patch works for me.

diff --git a/compiler-rt/cmake/config-ix.cmake b/compiler-rt/cmake/config-ix.cmake
index 2dccd4954b25..b4c93624bedd 100644
--- a/compiler-rt/cmake/config-ix.cmake
+++ b/compiler-rt/cmake/config-ix.cmake
@@ -177,6 +177,7 @@ check_include_files("sys/auxv.h"    COMPILER_RT_HAS_AUXV)
 # Libraries.
 check_library_exists(dl dlopen "" COMPILER_RT_HAS_LIBDL)
 check_library_exists(rt shm_open "" COMPILER_RT_HAS_LIBRT)
+check_library_exists(resolv __dn_expand "" COMPILER_RT_HAS_LIBRESOLV)
 check_library_exists(m pow "" COMPILER_RT_HAS_LIBM)
 check_library_exists(pthread pthread_create "" COMPILER_RT_HAS_LIBPTHREAD)
 check_library_exists(execinfo backtrace "" COMPILER_RT_HAS_LIBEXECINFO)
diff --git a/compiler-rt/lib/asan/CMakeLists.txt b/compiler-rt/lib/asan/CMakeLists.txt
index f993521d3ca8..b882d44d057b 100644
--- a/compiler-rt/lib/asan/CMakeLists.txt
+++ b/compiler-rt/lib/asan/CMakeLists.txt
@@ -128,6 +128,7 @@ append_list_if(COMPILER_RT_HAS_LIBM m ASAN_DYNAMIC_LIBS)
 append_list_if(COMPILER_RT_HAS_LIBPTHREAD pthread ASAN_DYNAMIC_LIBS)
 append_list_if(COMPILER_RT_HAS_LIBLOG log ASAN_DYNAMIC_LIBS)
 append_list_if(MINGW "${MINGW_LIBRARIES}" ASAN_DYNAMIC_LIBS)
+append_list_if(COMPILER_RT_HAS_LIBRESOLV resolv ASAN_DYNAMIC_LIBS)

 # Compile ASan sources into an object library.

@domidimi
Copy link

CC @kda

@aaronpuchert
Copy link
Member

aaronpuchert commented Nov 13, 2024

Let's first try to understand what happened. Here is a stack trace with some source information (with Clang 16):

#1  0x00007ffff757130d in __interceptor___dn_expand () at ../compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2546
#2 in getanswer_r.constprop () from /lib64/libnss_dns.so.2
#3 in gethostbyname3_context () from /lib64/libnss_dns.so.2
#4 in _nss_dns_gethostbyname3_r () from /lib64/libnss_dns.so.2
#5 in _nss_dns_gethostbyname2_r () from /lib64/libnss_dns.so.2
#6 in gethostbyname2_r@@GLIBC_2.2.5 () from /lib64/libc.so.6
#7 in gaih_inet.constprop () from /lib64/libc.so.6
#8 in getaddrinfo () from /lib64/libc.so.6
#9 in __interceptor_getaddrinfo () at ../compiler-rt/lib/asan/../sanitizer_common/sanitizer_common_interceptors.inc:2800

Let's have a look at the top frame: we call into REAL(DN_EXPAND_INTERCEPTOR_NAME), which expands to __interception::real_dn_expand or __interception::real___dn_expand depending on the glibc version, which is a function pointer. The function pointer is written in InitializeCommonInterceptors by INIT_DN_COMP_EXPAND, which expands to a call to __interception::InterceptFunction plus error handling. And that boils down to dlsym(RTLD_NEXT, "..."). When that doesn't find anything (for example because the library isn't loaded), it writes a null pointer. This is what happens here, because libc.so loads libnss_dns.so on demand, so it isn't loaded when the process starts and the "real" function pointers for interception are set up. So the library needs to be loaded on startup.

The issue with -nodefaultlibs is not a bug in my opinion. The compiler driver correctly adds -lresolv to the linker command line when building with sanitizers. However, if you build with -nodefaultlibs, you get none of the default libraries and have to add the required libraries yourself. That seems to work as it should.

The issue with the shared runtime seems valid however. The difference between the static and shared runtimes is that the static runtime comes with dependencies on the linker command line, whereas the shared runtime gets the dependencies from NEEDED entries in .dynamic of the shared runtime library. So I think the proposed fix for this is correct.

@aaronpuchert
Copy link
Member

The specific issue should only occur with older glibc versions: the call to dn_expand was removed from getanswer_r in glibc 2.26. With version 2.34, dn_expand has been moved from libresolv.so to libc.so. However, I think we should still add a link to -lresolv, if only because that's where the functions are documented to sit. And my understanding is that it's a no-op on newer systems anyway.

@vient
Copy link
Member

vient commented Nov 15, 2024

In original issue dn_expand call happens from another function, gaih_getanswer_slice with glibc 2.31. This means that versions up to 2.33 are affected?

@aaronpuchert
Copy link
Member

You're right, I didn't see that the original crash stack had a different caller. I also didn't see that call to dn_expand because it was also removed later in glibc 2.37. So yes, then versions up to 2.33 should be affected, at least with regards to dn_expand. (Not sure what other functions in libresolv are intercepted that might not have been moved for 2.34. A full list can be found in the release annoucement.)

aaronpuchert added a commit to aaronpuchert/llvm-project that referenced this issue Nov 15, 2024
Since the interceptor was added in 1a729bc, a process using
sanitizers might need libresolv for the real version of __dn_expand.
The linker flag for the static runtime was added in 6dce56b.
But the dynamic runtime didn't get the dependency. This can cause
crashes even when libresolv is not used directly, because it is also
used by DNS lookups via e.g. getaddrinfo when the DNS reply uses
compression.

We observed the issue with the address sanitizer, but a quick look into
the shared runtime libraries revealed that the memprof and tsan runtimes
also export dn_expand and should thus be prone to the same issue.

It should be mentioned that glibc plans to move libresolv into libc,
which was partially completed in version 2.34 with the move of dn_comp
and dn_expand. However, that is not an issue here: once the move is
complete, there will only be a static libresolv.a stub left and so
building against a version after the move will no longer produce a
runtime dependency to libresolv.so.

Fixes llvm#59007.
aaronpuchert added a commit to aaronpuchert/llvm-project that referenced this issue Dec 7, 2024
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes llvm#59007: while `-lresolv`
had been added for the static runtime in 6dce56b, it wasn't added
to the shared runtimes.

Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
  directly call ENSURE_MSAN_INITED() like most other interceptors. It
  seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
  to use it.
aaronpuchert added a commit to aaronpuchert/llvm-project that referenced this issue Dec 7, 2024
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes llvm#59007: while `-lresolv`
had been added for the static runtime in 6dce56b, it wasn't added
to the shared runtimes.

Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
  directly call ENSURE_MSAN_INITED() like most other interceptors. It
  seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
  to use it.
aaronpuchert added a commit to aaronpuchert/llvm-project that referenced this issue Dec 7, 2024
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes llvm#59007: while `-lresolv`
had been added for the static runtime in 6dce56b, it wasn't added
to the shared runtimes.

Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
  directly call ENSURE_MSAN_INITED() like most other interceptors. It
  seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
  to use it.
@EugeneZelenko EugeneZelenko added compiler-rt:msan Memory sanitizer and removed compiler-rt:asan Address sanitizer labels Dec 9, 2024
broxigarchen pushed a commit to broxigarchen/llvm-project that referenced this issue Dec 10, 2024
The functions are not relevant for most sanitizers and only required for
MSan to see which regions have been written to. This eliminates a link
dependency for all other sanitizers and fixes llvm#59007: while `-lresolv`
had been added for the static runtime in 6dce56b, it wasn't added
to the shared runtimes.

Instead of just moving the interceptors, we adapt them to MSan
conventions:
* We don't skip intercepting when `msan_init_is_running` is true, but
directly call ENSURE_MSAN_INITED() like most other interceptors. It
seems unlikely that these functions are called during initialization.
* We don't unpoison `errno`, because none of the functions is specified
to use it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler-rt:msan Memory sanitizer
Projects
None yet
6 participants