[compiler-rt] [win/asan] Don't intercept memset etc. in ntdll (PR #120397)

[Hans Wennborg via llvm-commits]

https://github.com/zmodem created https://github.com/llvm/llvm-project/pull/120397

When ntdll was added to the list of of "interesting DLLs" list (in
d58230b9dcb3b312a2da8f874daa0cc8dc27da9b), the intention was not to
intercept the "mini CRT" functions it exports. OverrideFunction would
only intercept the *first* function it found when searching the list of
DLLs, and ntdll was put last in that list.

However, after 42cdfbcf3e92466754c175cb0e1e237e9f66749e,
OverrideFunction intercepts *all* matching functions in those DLLs. As
a side-effect, the runtime would now intercept functions like memset
etc. also in ntdll.

This causes a problem when ntdll-internal functions like
RtlDispatchException call the intercepted memset, which tries to
inspect uncommitted shadow memory, raising an exception, and getting
stuck in that loop until the stack overflows.

Since we never intended to intercept ntdll's memset etc., the simplest
fix seems to be to actively ignore ntdll when intercepting those
functions.

Fixes #114793

>From 91b49f148e96297c1205405093dbb4afadb51d34 Mon Sep 17 00:00:00 2001
From: Hans Wennborg <hans at chromium.org>
Date: Tue, 17 Dec 2024 16:18:26 +0100
Subject: [PATCH] [win/asan] Don't intercept memset etc. in ntdll

---
 .../lib/interception/interception_win.cpp        | 16 ++++++++++++++--
 1 file changed, 14 insertions(+), 2 deletions(-)

diff --git a/compiler-rt/lib/interception/interception_win.cpp b/compiler-rt/lib/interception/interception_win.cpp
index a5897274521e92..094953d3544fba 100644
--- a/compiler-rt/lib/interception/interception_win.cpp
+++ b/compiler-rt/lib/interception/interception_win.cpp
@@ -1177,8 +1177,7 @@ static void **InterestingDLLsAvailable() {
     "libc++.dll",     // libc++
     "libunwind.dll",  // libunwind
 #  endif
-    // NTDLL should go last as it exports some functions that we should
-    // override in the CRT [presumably only used internally].
+    // NTDLL must go last as it gets special treatment in OverrideFunction.
     "ntdll.dll",
     NULL
   };
@@ -1281,9 +1280,22 @@ uptr InternalGetProcAddress(void *module, const char *func_name) {
 
 bool OverrideFunction(
     const char *func_name, uptr new_func, uptr *orig_old_func) {
+  static const char *kNtDllIgnore[] = {
+    "memcmp", "memcpy", "memmove", "memset"
+  };
+
   bool hooked = false;
   void **DLLs = InterestingDLLsAvailable();
   for (size_t i = 0; DLLs[i]; ++i) {
+    if (DLLs[i + 1] == nullptr) {
+      // This is the last DLL, i.e. NTDLL. It exports some functions that
+      // we only want to override in the CRT.
+      for (const char *ignored : kNtDllIgnore) {
+        if (strcmp(func_name, ignored) == 0)
+          return hooked;
+      }
+    }
+
     uptr func_addr = InternalGetProcAddress(DLLs[i], func_name);
     if (func_addr &&
         OverrideFunction(func_addr, new_func, orig_old_func)) {