[compiler-rt] [ASan] Prevent ASan/LSan deadlock by preloading modules before error reporting (PR #131756)
via llvm-commits
llvm-commits at lists.llvm.org
Fri Mar 21 05:14:52 PDT 2025
https://github.com/Camsyn updated https://github.com/llvm/llvm-project/pull/131756
>From 943948f05369c333066b7f04356df82d516dc1ea Mon Sep 17 00:00:00 2001
From: Camsyn <65994555+Camsyn at users.noreply.github.com>
Date: Tue, 18 Mar 2025 16:02:24 +0800
Subject: [PATCH 1/2] [ASan] Prevent ASan/LSan deadlock by preloading modules
before error reporting
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
This fixes a circular lock dependency between ASan's thread registry
lock (B) and libdl's dl_iterate_phdr lock (A) that occurs when:
1. ASan error reporting (holding B) calls dl_iterate_phdr (needs A)
2. LSan leak check (holding A) acquires ASan thread registry lock (B)
The fix proactively initializes module information before acquiring
ASan's thread registry lock during error reporting. This eliminates
the need to call `dl_iterate_phdr` while already holding lock B,
breaking the dangerous A→B/B→A cross-thread locking pattern.
Add test case (compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp)
that reproduces the deadlock without this patch.
---
compiler-rt/lib/asan/asan_report.cpp | 21 ++++++
.../asan/TestCases/asan_lsan_deadlock.cpp | 73 +++++++++++++++++++
2 files changed, 94 insertions(+)
create mode 100644 compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
diff --git a/compiler-rt/lib/asan/asan_report.cpp b/compiler-rt/lib/asan/asan_report.cpp
index 45aa607dcda07..8aeb938b2ee3d 100644
--- a/compiler-rt/lib/asan/asan_report.cpp
+++ b/compiler-rt/lib/asan/asan_report.cpp
@@ -126,6 +126,27 @@ class ScopedInErrorReport {
public:
explicit ScopedInErrorReport(bool fatal = false)
: halt_on_error_(fatal || flags()->halt_on_error) {
+ /*
+ * Deadlock Prevention Between ASan and LSan
+ *
+ * Background:
+ * - The `dl_iterate_phdr` function requires holding libdl's internal lock (Lock A).
+ * - LSan acquires the ASan thread registry lock (Lock B) *after* calling `dl_iterate_phdr`.
+ *
+ * Problem Scenario:
+ * When ASan attempts to call `dl_iterate_phdr` while holding Lock B (e.g., during
+ * error reporting via `ErrorDescription::Print`), a circular lock dependency may occur:
+ * 1. Thread 1: Holds Lock B → Requests Lock A (via dl_iterate_phdr)
+ * 2. Thread 2: Holds Lock A → Requests Lock B (via LSan operations)
+ *
+ * Solution:
+ * Proactively load all required modules before acquiring Lock B. This ensures:
+ * 1. Any `dl_iterate_phdr` calls during module loading complete before locking
+ * 2. Subsequent error reporting avoids nested lock acquisition patterns
+ * 3. Eliminates the lock order inversion risk between libdl and ASan's thread registry
+ */
+ Symbolizer::GetOrInit()->GetRefreshedListOfModules();
+
// Make sure the registry and sanitizer report mutexes are locked while
// we're printing an error report.
// We can lock them only here to avoid self-deadlock in case of
diff --git a/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp b/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
new file mode 100644
index 0000000000000..6d4c91eb49241
--- /dev/null
+++ b/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
@@ -0,0 +1,73 @@
+// Test for potential deadlock in LeakSanitizer+AddressSanitizer.
+// REQUIRES: leak-detection
+//
+// RUN: %clangxx_asan -O0 %s -o %t
+// RUN: %env_asan_opts=detect_leaks=1 not %run %t 2>&1 | FileCheck %s
+
+/*
+ * Purpose: Verify deadlock prevention between ASan error reporting and LSan leak checking.
+ *
+ * Test Design:
+ * 1. Creates contention scenario between:
+ * - ASan's error reporting (requires lock B -> lock A ordering)
+ * - LSan's leak check (requires lock A -> lock B ordering)
+ * 2. Thread timing:
+ * - Main thread: Holds 'in' mutex -> Triggers LSan check (lock A then B)
+ * - Worker thread: Triggers ASan OOB error (lock B then A via symbolization)
+ *
+ * Deadlock Condition (if unfixed):
+ * Circular lock dependency forms when:
+ * [Main Thread] LSan: lock A -> requests lock B
+ * [Worker Thread] ASan: lock B -> requests lock A
+ *
+ * Success Criteria:
+ * With proper lock ordering enforcement, watchdog should NOT trigger - test exits normally.
+ * If deadlock occurs, watchdog terminates via _exit(1) after 10s timeout.
+ */
+
+#include <mutex>
+#include <sanitizer/lsan_interface.h>
+#include <stdio.h>
+#include <thread>
+#include <unistd.h>
+
+std::mutex in;
+
+void Watchdog() {
+ // Safety mechanism: Turn infinite deadlock into finite test failure
+ usleep(10000000);
+ // CHECK-NOT: Timeout! Deadlock detected.
+ puts("Timeout! Deadlock detected.");
+ fflush(stdout);
+ _exit(1);
+}
+
+int main(int argc, char **argv) {
+ int arr[1] = {0};
+ in.lock();
+
+ std::thread w(Watchdog);
+ w.detach();
+
+ std::thread t([&]() {
+ in.unlock();
+ /*
+ * Provoke ASan error: ASan's error reporting acquires:
+ * 1. ASan's thread registry lock (B) during the reporting
+ * 2. dl_iterate_phdr lock (A) during symbolization
+ */
+ // CHECK: SUMMARY: AddressSanitizer: stack-buffer-overflow
+ arr[argc] = 1; // Deliberate OOB access
+ });
+
+ in.lock();
+ /*
+ * Critical section: LSan's check acquires:
+ * 1. dl_iterate_phdr lock (A)
+ * 2. ASan's thread registry lock (B)
+ * before Stop The World.
+ */
+ __lsan_do_leak_check();
+ t.join();
+ return 0;
+}
\ No newline at end of file
>From fb7a15508a5418b89ec6ed56d6e593858e80eaf2 Mon Sep 17 00:00:00 2001
From: Camsyn <camsyn at foxmail.com>
Date: Fri, 21 Mar 2025 20:13:42 +0800
Subject: [PATCH 2/2] Fix some format issues adhering the PR suggestion
---
compiler-rt/lib/asan/asan_report.cpp | 42 ++++++++++---------
.../asan/TestCases/asan_lsan_deadlock.cpp | 5 +--
2 files changed, 25 insertions(+), 22 deletions(-)
diff --git a/compiler-rt/lib/asan/asan_report.cpp b/compiler-rt/lib/asan/asan_report.cpp
index 8aeb938b2ee3d..a64db485520ee 100644
--- a/compiler-rt/lib/asan/asan_report.cpp
+++ b/compiler-rt/lib/asan/asan_report.cpp
@@ -126,25 +126,29 @@ class ScopedInErrorReport {
public:
explicit ScopedInErrorReport(bool fatal = false)
: halt_on_error_(fatal || flags()->halt_on_error) {
- /*
- * Deadlock Prevention Between ASan and LSan
- *
- * Background:
- * - The `dl_iterate_phdr` function requires holding libdl's internal lock (Lock A).
- * - LSan acquires the ASan thread registry lock (Lock B) *after* calling `dl_iterate_phdr`.
- *
- * Problem Scenario:
- * When ASan attempts to call `dl_iterate_phdr` while holding Lock B (e.g., during
- * error reporting via `ErrorDescription::Print`), a circular lock dependency may occur:
- * 1. Thread 1: Holds Lock B → Requests Lock A (via dl_iterate_phdr)
- * 2. Thread 2: Holds Lock A → Requests Lock B (via LSan operations)
- *
- * Solution:
- * Proactively load all required modules before acquiring Lock B. This ensures:
- * 1. Any `dl_iterate_phdr` calls during module loading complete before locking
- * 2. Subsequent error reporting avoids nested lock acquisition patterns
- * 3. Eliminates the lock order inversion risk between libdl and ASan's thread registry
- */
+ // Deadlock Prevention Between ASan and LSan
+ //
+ // Background:
+ // - The `dl_iterate_phdr` function requires holding libdl's internal lock
+ // (Lock A).
+ // - LSan acquires the ASan thread registry lock (Lock B) *after* calling
+ // `dl_iterate_phdr`.
+ //
+ // Problem Scenario:
+ // When ASan attempts to call `dl_iterate_phdr` while holding Lock B (e.g.,
+ // during error reporting via `ErrorDescription::Print`), a circular lock
+ // dependency may occur:
+ // 1. Thread 1: Holds Lock B → Requests Lock A (via dl_iterate_phdr)
+ // 2. Thread 2: Holds Lock A → Requests Lock B (via LSan operations)
+ //
+ // Solution:
+ // Proactively load all required modules before acquiring Lock B.
+ // This ensures:
+ // 1. Any `dl_iterate_phdr` calls during module loading complete before
+ // locking.
+ // 2. Subsequent error reporting avoids nested lock acquisition patterns.
+ // 3. Eliminates the lock order inversion risk between libdl and ASan's
+ // thread registry.
Symbolizer::GetOrInit()->GetRefreshedListOfModules();
// Make sure the registry and sanitizer report mutexes are locked while
diff --git a/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp b/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
index 6d4c91eb49241..4e1a2415ad013 100644
--- a/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
+++ b/compiler-rt/test/asan/TestCases/asan_lsan_deadlock.cpp
@@ -31,8 +31,6 @@
#include <thread>
#include <unistd.h>
-std::mutex in;
-
void Watchdog() {
// Safety mechanism: Turn infinite deadlock into finite test failure
usleep(10000000);
@@ -44,6 +42,7 @@ void Watchdog() {
int main(int argc, char **argv) {
int arr[1] = {0};
+ std::mutex in;
in.lock();
std::thread w(Watchdog);
@@ -70,4 +69,4 @@ int main(int argc, char **argv) {
__lsan_do_leak_check();
t.join();
return 0;
-}
\ No newline at end of file
+}
More information about the llvm-commits
mailing list