[compiler-rt] [tsan] Lazily call 'personality' to minimize sandbox violations (PR #79334)

[Thurston Dang via llvm-commits]

https://github.com/thurstond created https://github.com/llvm/llvm-project/pull/79334

My previous patch, "Re-exec TSan with no ASLR if memory layout is incompatible on Linux (#78351)" (0784b1eefa36d4acbb0dacd2d18796e26313b6c5) hoisted the 'personality' call, to
share the code between Android and non-Android Linux. Unfortunately, this eager call to
'personality' may trigger sandbox violations on non-Android Linux.

This patch fixes the issue by only calling 'personality' on non-Android Linux if the
memory mapping is incompatible. This may still cause a sandbox violation, but only if it
was going to abort anyway due to an incompatible memory mapping.

(The behavior on Android Linux is unchanged by this patch or the previous patch.)


>From a323748346a3e25918761eaaa785399a008be3ea Mon Sep 17 00:00:00 2001
From: Thurston Dang <thurston at google.com>
Date: Wed, 24 Jan 2024 17:27:20 +0000
Subject: [PATCH] [tsan] Lazily call 'personality' to minimize sandbox
 violations

My previous patch, "Re-exec TSan with no ASLR if memory layout is incompatible on Linux (#78351)" (0784b1eefa36d4acbb0dacd2d18796e26313b6c5) hoisted the 'personality' call, to
share the code between Android and non-Android Linux. Unfortunately, this eager call to
'personality' may trigger sandbox violations on non-Android Linux.

This patch fixes the issue by only calling 'personality' on non-Android Linux if the
memory mapping is incompatible. This may still cause a sandbox violation, but only if it
was going to abort anyway due to an incompatible memory mapping.

(The behavior on Android Linux is unchanged by this patch or the previous patch.)
---
 compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp b/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp
index 0d0b1aba1f852a5..c723dba556ed2f7 100644
--- a/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp
+++ b/compiler-rt/lib/tsan/rtl/tsan_platform_linux.cpp
@@ -244,12 +244,12 @@ static void ReExecIfNeeded() {
   }
 
 #    if SANITIZER_LINUX
+#      if SANITIZER_ANDROID && (defined(__aarch64__) || defined(__x86_64__))
   // ASLR personality check.
   int old_personality = personality(0xffffffff);
   bool aslr_on =
       (old_personality != -1) && ((old_personality & ADDR_NO_RANDOMIZE) == 0);
 
-#      if SANITIZER_ANDROID && (defined(__aarch64__) || defined(__x86_64__))
   // After patch "arm64: mm: support ARCH_MMAP_RND_BITS." is introduced in
   // linux kernel, the random gap between stack and mapped area is increased
   // from 128M to 36G on 39-bit aarch64. As it is almost impossible to cover
@@ -267,6 +267,14 @@ static void ReExecIfNeeded() {
   if (reexec) {
     // Don't check the address space since we're going to re-exec anyway.
   } else if (!CheckAndProtect(false, false, false)) {
+    // ASLR personality check.
+    // N.B. 'personality' is sometimes forbidden by sandboxes, so we only call
+    // this as a last resort (when the memory mapping is incompatible and TSan
+    // would fail anyway).
+    int old_personality = personality(0xffffffff);
+    bool aslr_on =
+        (old_personality != -1) && ((old_personality & ADDR_NO_RANDOMIZE) == 0);
+
     if (aslr_on) {
       // Disable ASLR if the memory layout was incompatible.
       // Alternatively, we could just keep re-execing until we get lucky