[compiler-rt] [sanitizer_common] Add experimental flag to tweak dlopen(<main program>) (PR #71715)

[Thurston Dang via llvm-commits]

https://github.com/thurstond created https://github.com/llvm/llvm-project/pull/71715

This introduces an experimental flag 'test_only_replace_dlopen_main_program'.
When enabled, this will replace dlopen(<main program,...> with dlopen(NULL,...),
which is the correct way to get a handle to the main program.

This can be useful when ASan is statically linked, since dladdr((void*)pthread_join)
or similar will return the path to the main program.

Note that dlopen(<main program>,...) never ends well:
- PIE in recent glibc versions (glibc bugzilla 24323), or non-PIE: return an error
- PIE in current GRTE and older glibc: attempt to load the main program
  again, leading to reinitializing ASan and failing to remap the shadow
  memory.


>From 2a73a5fba63e5dc769fb10166315f3c912bcf9fa Mon Sep 17 00:00:00 2001
From: Thurston Dang <thurston at google.com>
Date: Wed, 8 Nov 2023 17:50:29 +0000
Subject: [PATCH] [sanitizer_common] Add experimental flag to tweak
 dlopen(<main program>)

This introduces an experimental flag 'test_only_replace_dlopen_main_program'.
When enabled, this will replace dlopen(<main program,...> with dlopen(NULL,...),
which is the correct way to get a handle to the main program.

This can be useful when ASan is statically linked, since dladdr((void*)pthread_join)
or similar will return the path to the main program.

Note that dlopen(<main program>,...) never ends well:
- PIE in recent glibc versions (glibc bugzilla 24323), or non-PIE: return an error
- PIE in current GRTE and older glibc: attempt to load the main program
  again, leading to reinitializing ASan and failing to remap the shadow
  memory.
---
 .../sanitizer_common_interceptors.inc         | 30 ++++++++++++++++++-
 .../lib/sanitizer_common/sanitizer_flags.inc  |  6 ++++
 2 files changed, 35 insertions(+), 1 deletion(-)

diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc b/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc
index 80efaf54a0607f6..0f7d9da9d148443 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_common_interceptors.inc
@@ -6304,10 +6304,38 @@ INTERCEPTOR(int, fclose, __sanitizer_FILE *fp) {
 #endif
 
 #if SANITIZER_INTERCEPT_DLOPEN_DLCLOSE
+// Returns 1 if key is a suffix of str, 0 otherwise
+static int internal_strcmp_suffix(const char *key, const char *str) {
+  if (!key || !str)
+    return 0;
+
+  if (internal_strlen(key) > internal_strlen(str))
+    return 0;
+
+  return !internal_strcmp(str + internal_strlen(str) - internal_strlen(key),
+                          key);
+}
+
+#  if SANITIZER_GLIBC
+extern char *__progname;
+#  endif
+
 INTERCEPTOR(void*, dlopen, const char *filename, int flag) {
   void *ctx;
   COMMON_INTERCEPTOR_ENTER_NOIGNORE(ctx, dlopen, filename, flag);
-  if (filename) COMMON_INTERCEPTOR_READ_STRING(ctx, filename, 0);
+
+  if (filename) {
+#  if SANITIZER_GLIBC
+    if (common_flags()->test_only_replace_dlopen_main_program &&
+        internal_strcmp_suffix(__progname, filename)) {
+      VPrintf(1, "dlopen interceptor: replacing %s because it matches %s\n",
+              filename, __progname);
+      filename = (char *)0;  // RTLD_DEFAULT
+    }
+#  endif
+    COMMON_INTERCEPTOR_READ_STRING(ctx, filename, 0);
+  }
+
   void *res = COMMON_INTERCEPTOR_DLOPEN(filename, flag);
   Symbolizer::GetOrInit()->InvalidateModuleList();
   COMMON_INTERCEPTOR_LIBRARY_LOADED(filename, res);
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc b/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
index 6148ae56067cae0..949bdbd148b6b89 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
@@ -269,3 +269,9 @@ COMMON_FLAG(bool, detect_write_exec, false,
 COMMON_FLAG(bool, test_only_emulate_no_memorymap, false,
             "TEST ONLY fail to read memory mappings to emulate sanitized "
             "\"init\"")
+// With static linking, dladdr((void*)pthread_join) or similar will return the
+// path to the main program. This flag will replace dlopen(<main program,...>
+// with dlopen(NULL,...), which is the correct way to get a handle to the main
+// program.
+COMMON_FLAG(bool, test_only_replace_dlopen_main_program, false,
+            "TEST ONLY replace dlopen(<main program>,...) with dlopen(NULL)")