[compiler-rt] [llvm] [asan] Adopt sanitizer_common operator-new framework (PR #196388)

[Justin T. Gibbs via llvm-commits]

================
@@ -51,51 +60,131 @@ using namespace __asan;
 // This code has issues on OSX.
 // See https://github.com/google/sanitizers/issues/131.
 
-// Fake std::nothrow_t and std::align_val_t to avoid including <new>.
+#if SANITIZER_WINDOWS
+// Forward-declare just enough of std for the operator overrides and for
+// RunNewHandlerChain below. std::get_new_handler is supplied by the C++
+// runtime the asan DLL already links against (vcruntime / msvcprt / mingw
+// libstdc++) — it doesn't need to be in <new>.
 namespace std {
 struct nothrow_t {};
-enum class align_val_t: size_t {};
+enum class align_val_t : size_t {};
+using new_handler = void (*)();
+new_handler get_new_handler() noexcept;
 }  // namespace std
+#endif  // SANITIZER_WINDOWS
+
+// All eight operator new variants route through three templates so the
+// per-platform difference (Linux/Apple throw, Windows abort) is centralized
+// here rather than duplicated in eight OPERATOR_NEW_BODY* macros.
+//
+// Contract: the Alloc callable must always return nullptr on failure, never
+// abort. asan_new / asan_new_array / asan_new_aligned / asan_new_array_aligned
+// in asan_allocator.cpp force may_return_null=true to honor this.
+
+// Runs std::get_new_handler() per [new.delete.single]/3+/4 until the
+// allocation succeeds or the handler is null. A handler that throws
+// propagates out of this function — callers wrap in try/catch as needed.
+template <typename Alloc>
+static void* RunNewHandlerChain(Alloc alloc) {
+  for (;;) {
+    void* res = alloc();
+    if (LIKELY(res != nullptr))
+      return res;
+    std::new_handler handler = std::get_new_handler();
+    if (!handler)
+      return nullptr;
+    handler();
+  }
+}
+
+// Chain-exhausted decision for the nothrow form: nullptr if
+// allocator_may_return_null is set, else ReportOutOfMemory + Die(). The
+// throwing form intentionally does NOT route through here — it must never
+// return nullptr to its caller (per [basic.stc.dynamic.allocation]/3) so on
+// Windows, where it can't throw bad_alloc, it must abort regardless of the
+// flag.
+static void* NewImplNothrowExhausted(uptr size, BufferedStackTrace* stack) {
+  if (AllocatorMayReturnNull())
+    return nullptr;
+  ReportOutOfMemory(size, stack);
+  __builtin_unreachable();
+}
 
-// TODO(alekseyshl): throw std::bad_alloc instead of dying on OOM.
-// For local pool allocation, align to SHADOW_GRANULARITY to match asan
-// allocator behavior.
-#define OPERATOR_NEW_BODY             \
-  GET_STACK_TRACE_MALLOC;             \
-  void *res = asan_new(size, &stack); \
-  if (UNLIKELY(!res))                 \
-    ReportOutOfMemory(size, &stack);  \
-  return res
+// Throwing operator new: chain, then on exhaustion throw std::bad_alloc
+// (non-Windows + allocator_may_return_null=1) or abort via
+// ReportOutOfMemory + Die() (default flag, or Windows for any flag value).
+template <typename Alloc>
+static void* NewImplThrowing(uptr size, BufferedStackTrace* stack,
+                             Alloc alloc) {
+  void* res = RunNewHandlerChain(alloc);
+  if (LIKELY(res != nullptr))
+    return res;
+#if !SANITIZER_WINDOWS
+  if (AllocatorMayReturnNull())
+    throw std::bad_alloc();
+#endif
+  ReportOutOfMemory(size, stack);
+  __builtin_unreachable();
+}
+
+// Nothrow operator new: per [new.delete.single]/4 behaves as-if the throwing
+// form is called within a try/catch. On Windows there's no try/catch; a
+// user handler that throws there yields UB (asan is built without
+// exceptions on Windows).
+template <typename Alloc>
----------------
scsiguy wrote:

 This comment is gone from ASan. The equivalent logic now lives in `sanitizer_new_handler.h`
 (landed in #201151), which assumes Windows runtimes are built without exceptions and so
 never throws `std::bad_alloc` there. @zacklj89, if MSVC ASan builds its runtime with
 exceptions, let me know and we can relax that guard in a follow-up.

https://github.com/llvm/llvm-project/pull/196388