[compiler-rt] [asan] Print diagnostic if unlimited stack size detected (PR #133170)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-compiler-rt-sanitizer

Author: Thurston Dang (thurstond)

<details>
<summary>Changes</summary>

This adds a diagnostic message if the stack size is unlimited. This would have simplified the diagnosis of
https://github.com/google/sanitizers/issues/856#issuecomment-2747076811; we anticipate this may help diagnose future issues too.

---
Full diff: https://github.com/llvm/llvm-project/pull/133170.diff


1 Files Affected:

- (modified) compiler-rt/lib/asan/asan_rtl.cpp (+12) 


``````````diff
diff --git a/compiler-rt/lib/asan/asan_rtl.cpp b/compiler-rt/lib/asan/asan_rtl.cpp
index 19c6c210b564c..d98b350828448 100644
--- a/compiler-rt/lib/asan/asan_rtl.cpp
+++ b/compiler-rt/lib/asan/asan_rtl.cpp
@@ -451,6 +451,18 @@ static bool AsanInitInternal() {
 
   DisableCoreDumperIfNecessary();
 
+#if SANITIZER_POSIX
+  if (StackSizeIsUnlimited()) {
+    VPrintf(1,
+            "WARNING: Unlimited stack size detected. This may affect "
+            "compatibility with the shadow mappings.\n");
+    // MSan and TSan re-exec with a fixed size stack. We don't do that because
+    // it may break the program. InitializeShadowMemory() will, if needed,
+    // re-exec without ASLR, which solves most shadow mapping compatibility
+    // issues.
+  }
+#endif
+
   InitializeShadowMemory();
 
   AsanTSDInit(PlatformTSDDtor);

``````````

</details>


https://github.com/llvm/llvm-project/pull/133170