[compiler-rt] r336067 - [asan] Fix deadlock issue on FreeBSD, caused by use of .preinit_array in rL325240

[Fangrui Song via llvm-commits]

Author: maskray
Date: Sun Jul  1 10:52:41 2018
New Revision: 336067

URL: http://llvm.org/viewvc/llvm-project?rev=336067&view=rev
Log:
[asan] Fix deadlock issue on FreeBSD, caused by use of .preinit_array in rL325240

Summary:
Without this patch,
clang -fsanitize=address -xc =(printf 'int main(){}') -o a; ./a => deadlock in __asan_init>AsanInitInternal>AsanTSDInit>...>__getcontextx_size>_rtld_bind>rlock_acquire(rtld_bind_lock, &lockstate)

libexec/rtld-elf/rtld.c
  wlock_acquire(rtld_bind_lock, &lockstate);
  if (obj_main->crt_no_init)
    preinit_main(); // unresolved PLT functions cannot be called here

lib/libthr/thread/thr_rtld.c
  uc_len = __getcontextx_size(); // unresolved PLT function in libthr.so.3

check-xray tests currently rely on .preinit_array so we special case in
xray_init.cc

Subscribers: srhines, kubamracek, krytarowski, delcypher, llvm-commits, #sanitizers

Differential Revision: https://reviews.llvm.org/D48806

Modified:
    compiler-rt/trunk/lib/sanitizer_common/sanitizer_internal_defs.h
    compiler-rt/trunk/lib/xray/xray_init.cc

Modified: compiler-rt/trunk/lib/sanitizer_common/sanitizer_internal_defs.h
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/sanitizer_common/sanitizer_internal_defs.h?rev=336067&r1=336066&r2=336067&view=diff
==============================================================================
--- compiler-rt/trunk/lib/sanitizer_common/sanitizer_internal_defs.h (original)
+++ compiler-rt/trunk/lib/sanitizer_common/sanitizer_internal_defs.h Sun Jul  1 10:52:41 2018
@@ -98,11 +98,16 @@
 
 // We can use .preinit_array section on Linux to call sanitizer initialization
 // functions very early in the process startup (unless PIC macro is defined).
+//
+// On FreeBSD, .preinit_array functions are called with rtld_bind_lock writer
+// lock held. It will lead to dead lock if unresolved PLT functions (which helds
+// rtld_bind_lock reader lock) are called inside .preinit_array functions.
+//
 // FIXME: do we have anything like this on Mac?
 #ifndef SANITIZER_CAN_USE_PREINIT_ARRAY
-#if ((SANITIZER_LINUX && !SANITIZER_ANDROID) || \
-  SANITIZER_FREEBSD || SANITIZER_OPENBSD) && !defined(PIC)
-# define SANITIZER_CAN_USE_PREINIT_ARRAY 1
+#if ((SANITIZER_LINUX && !SANITIZER_ANDROID) || SANITIZER_OPENBSD) && \
+    !defined(PIC)
+#define SANITIZER_CAN_USE_PREINIT_ARRAY 1
 // Before Solaris 11.4, .preinit_array is fully supported only with GNU ld.
 // FIXME: Check for those conditions.
 #elif SANITIZER_SOLARIS && !defined(PIC)

Modified: compiler-rt/trunk/lib/xray/xray_init.cc
URL: http://llvm.org/viewvc/llvm-project/compiler-rt/trunk/lib/xray/xray_init.cc?rev=336067&r1=336066&r2=336067&view=diff
==============================================================================
--- compiler-rt/trunk/lib/xray/xray_init.cc (original)
+++ compiler-rt/trunk/lib/xray/xray_init.cc Sun Jul  1 10:52:41 2018
@@ -84,7 +84,13 @@ void __xray_init() XRAY_NEVER_INSTRUMENT
 #endif
 }
 
-#if !defined(XRAY_NO_PREINIT) && SANITIZER_CAN_USE_PREINIT_ARRAY
+// FIXME: Make check-xray tests work on FreeBSD without
+// SANITIZER_CAN_USE_PREINIT_ARRAY.
+// See sanitizer_internal_defs.h where the macro is defined.
+// Calling unresolved PLT functions in .preinit_array can lead to deadlock on
+// FreeBSD but here it seems benign.
+#if !defined(XRAY_NO_PREINIT) &&                                               \
+    (SANITIZER_CAN_USE_PREINIT_ARRAY || SANITIZER_FREEBSD)
 // Only add the preinit array initialization if the sanitizers can.
 __attribute__((section(".preinit_array"),
                used)) void (*__local_xray_preinit)(void) = __xray_init;