[compiler-rt] [sanitizer_common] Add operator new chain-handling framework (PR #201151)
Justin T. Gibbs via llvm-commits
llvm-commits at lists.llvm.org
Mon Jun 8 13:48:03 PDT 2026
https://github.com/scsiguy updated https://github.com/llvm/llvm-project/pull/201151
>From e6b0b1833f13d1241242071427548a095963deda Mon Sep 17 00:00:00 2001
From: "Justin T. Gibbs" <gibbs at scsiguy.com>
Date: Sat, 30 May 2026 17:31:24 -0700
Subject: [PATCH] [sanitizer_common] Add operator new chain-handling framework
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Implement the operator new wrapper machinery required by
[new.delete.single]/3+/4 in shared sanitizer_common files so every
sanitizer can reuse it (avoids ~130 lines of duplication).
sanitizer_new_handler.h provides three main templates in namespace
__sanitizer (plus a NORETURN InvokeOnExhausted wrapper used internally):
* RunNewHandlerChain<Alloc>(alloc)
Runs std::get_new_handler() in a loop until either the
allocation succeeds or the chain is exhausted (returns nullptr).
* NewImplThrowing<Alloc, OnExhausted>(alloc, on_exhausted)
Throwing operator new: runs the chain; on exhaustion either
throws std::bad_alloc (AllocatorMayReturnNull()=true opts into
standards-conformant throw) or invokes on_exhausted (default;
historical abort-on-OOM).
* NewImplNothrow<Alloc, OnExhausted>(alloc, on_exhausted) noexcept
Nothrow operator new: per [new.delete.single]/4 behaves as-if
the throwing form is called within a try/catch converting any
exceptions to a nullptr return.
NOTE: Windows builds do not support exceptions, so the throwing
std::bad_alloc behavior described above is converted to an
invocation of on_exhausted(); a user new_handler that throws
on Windows trips the noexcept on NewImplNothrow and aborts via
std::terminate.
NOTE: TUs using this framework must be built with -fexceptions on
non-Windows.
NOTE: Instantiating the throwing-form template introduces a
dependency on `std::bad_alloc` in the consuming TU.
Adopters must link a C++ ABI library (libstdc++ / libc++abi)
into the resulting runtime.
sanitizer_new_operators.inc builds the eight standard
OPERATOR_NEW_BODY* macros on top of these templates. A consuming
sanitizer defines six ingredient macros (a stack-trace setup,
a report-OOM invocation, and four alloc helpers) and then includes the
file. The resulting OPERATOR_NEW_BODY / OPERATOR_NEW_BODY_NOTHROW /
OPERATOR_NEW_BODY_ARRAY / ... / OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW
macros can then be used directly as the bodies of the eight operator new
overrides.
NFC. No consumer yet — this is a prerequisite for a follow-up that
restructures compiler-rt/lib/asan/asan_new_delete.cpp to use the
shared framework.
Assisted by: Claude Opus 4.7
---
.../lib/sanitizer_common/CMakeLists.txt | 2 +
.../sanitizer_common/sanitizer_new_handler.h | 138 ++++++++++++++++++
.../sanitizer_new_operators.inc | 118 +++++++++++++++
3 files changed, 258 insertions(+)
create mode 100644 compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
create mode 100644 compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
diff --git a/compiler-rt/lib/sanitizer_common/CMakeLists.txt b/compiler-rt/lib/sanitizer_common/CMakeLists.txt
index 96c23c6d8ab82..07e9cad572b3d 100644
--- a/compiler-rt/lib/sanitizer_common/CMakeLists.txt
+++ b/compiler-rt/lib/sanitizer_common/CMakeLists.txt
@@ -170,6 +170,8 @@ set(SANITIZER_IMPL_HEADERS
sanitizer_mac.h
sanitizer_malloc_mac.inc
sanitizer_mutex.h
+ sanitizer_new_handler.h
+ sanitizer_new_operators.inc
sanitizer_placement_new.h
sanitizer_platform.h
sanitizer_platform_interceptors.h
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
new file mode 100644
index 0000000000000..91845843dff05
--- /dev/null
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
@@ -0,0 +1,138 @@
+//===-- sanitizer_new_handler.h ---------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is shared between run-time libraries of sanitizers.
+//
+// It provides the operator new chain-handling framework required by
+// [new.delete.single]/3+/4: a std::get_new_handler() loop plus the
+// throwing / nothrow exhaustion policies that compose with it. Each
+// sanitizer's operator new wrapper supplies two small lambdas:
+//
+// * Alloc — invokes the sanitizer's internal allocator, returning
+// nullptr on OOM (never aborting on OOM). Non-OOM
+// failure modes (e.g. invalid alignment) are the
+// caller's responsibility to detect and report before
+// invoking the framework — those conditions are UB/API
+// misuse, not allocation failures, and should die with
+// the caller's specific diagnostic rather than be
+// routed through the std::get_new_handler() chain.
+//
+// * OnExhausted — invokes the sanitizer's "abort with diagnostic"
+// handler (e.g. asan's ReportOutOfMemory + Die()).
+// Required to never return (enforced by UNREACHABLE()
+// annotations in this common infrastructure).
+//===----------------------------------------------------------------------===//
+#ifndef SANITIZER_NEW_HANDLER_H
+#define SANITIZER_NEW_HANDLER_H
+
+#include "sanitizer_allocator.h"
+#include "sanitizer_internal_defs.h"
+#include "sanitizer_platform.h"
+
+#if !SANITIZER_WINDOWS
+# include <new>
+#else
+// On Windows the sanitizer runtimes are built without exceptions and can't
+// include <new>. Forward-declare just enough of std for the loop; the real
+// std::get_new_handler is supplied by the C++ runtime the sanitizer DLL
+// already links against (vcruntime / msvcprt / mingw libstdc++) — it
+// doesn't need to come from <new>.
+namespace std {
+struct nothrow_t {};
+enum class align_val_t : __sanitizer::usize {};
+using new_handler = void (*)();
+new_handler get_new_handler() noexcept;
+} // namespace std
+#endif // !SANITIZER_WINDOWS
+
+namespace __sanitizer {
+
+// Runs std::get_new_handler() per [new.delete.single]/3+/4 until the
+// allocation succeeds or the chain is exhausted. Returns the allocated
+// pointer on success, nullptr if the handler chain is exhausted.
+//
+// NOTE: A handler that throws, propagates out of this function.
+// Callers that need to convert exceptions to a nullptr return
+// (e.g. NewImplNothrow below) must wrap the call in try/catch
+// themselves.
+template <typename Alloc>
+void* RunNewHandlerChain(Alloc alloc) {
+ for (;;) {
+ void* res = alloc();
+ if (LIKELY(res != nullptr))
+ return res;
+ std::new_handler handler = std::get_new_handler();
+ if (!handler)
+ return nullptr;
+ handler();
+ }
+}
+
+// NORETURN wrapper that promotes the OnExhausted callable's
+// morally-noreturn contract into a real noreturn function signature
+// the compiler can see at every call site.
+//
+// NOTE: in C++23 (CWG2511 / P2173R1), [[noreturn]] may be attached directly
+// to a lambda's call operator, e.g.
+//
+// [&]() [[noreturn]] { sanitizer_report(); }
+//
+// which removes the need for this wrapper entirely. Once compiler-rt
+// requires C++23 (and the supported clang/MSVC versions ship the fix),
+// this helper can be deleted in favor of marking the OnExhausted
+// lambdas in sanitizer_new_operators.inc as [[noreturn]] directly.
+template <typename OnExhausted>
+static NORETURN void InvokeOnExhausted(OnExhausted on_exhausted) {
+ on_exhausted();
+ UNREACHABLE("operator new OnExhausted callable returned");
+}
+
+// Throwing operator new: chain, then on exhaustion throw std::bad_alloc
+// (non-Windows + AllocatorMayReturnNull()=true) or invoke on_exhausted
+// (default flag, or Windows for any flag value).
+template <typename Alloc, typename OnExhausted>
+void* NewImplThrowing(Alloc alloc, OnExhausted on_exhausted) {
+ void* res = RunNewHandlerChain(alloc);
+ if (LIKELY(res != nullptr))
+ return res;
+ // Windows builds can't throw exceptions, so always fallthrough
+ // to the abort path.
+#if !SANITIZER_WINDOWS
+ if (AllocatorMayReturnNull())
+ throw std::bad_alloc();
+#endif
+ InvokeOnExhausted(on_exhausted);
+}
+
+// Nothrow operator new: per [new.delete.single]/4 behaves as-if the
+// throwing form is called within a try/catch. The lack of exception
+// support in Windows builds prevents us from doing this literally —
+// we have to avoid throwing std::bad_alloc at all. So the main body here
+// is identical to NewImplThrowing with "throw std::bad_alloc()" replaced
+// by "return nullptr".
+template <typename Alloc, typename OnExhausted>
+void* NewImplNothrow(Alloc alloc, OnExhausted on_exhausted) noexcept {
+#if !SANITIZER_WINDOWS
+ try {
+#endif
+ void* res = RunNewHandlerChain(alloc);
+ if (LIKELY(res != nullptr))
+ return res;
+ if (AllocatorMayReturnNull())
+ return nullptr;
+ InvokeOnExhausted(on_exhausted);
+#if !SANITIZER_WINDOWS
+ } catch (...) {
+ return nullptr;
+ }
+#endif
+}
+
+} // namespace __sanitizer
+
+#endif // SANITIZER_NEW_HANDLER_H
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
new file mode 100644
index 0000000000000..50ecdac45be5d
--- /dev/null
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
@@ -0,0 +1,118 @@
+//===-- sanitizer_new_operators.inc -----------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Shared OPERATOR_NEW_BODY* macros for sanitizer operator new overrides.
+//
+// All eight variants compose the same per-call setup (a sanitizer-specific
+// stack-trace acquisition) with one of the two chain-handling templates from
+// sanitizer_new_handler.h and two sanitizer-supplied lambdas: an Alloc that
+// invokes the sanitizer's internal allocator (returning nullptr on failure)
+// and an OnExhausted that calls the sanitizer's "abort with diagnostic"
+// handler.
+//
+// This file is included after the consuming TU has defined the ingredient
+// macros listed below. Names reference symbols (e.g. `stack`, `size`) that
+// exist only in the macro's expansion context, so this is an .inc, not a
+// freestanding header.
+//
+// The consuming TU must define before including:
+//
+// SANITIZER_NEW_STACK_TRACE
+// A statement that, when expanded inside an operator new body, puts a
+// BufferedStackTrace named `stack` in scope. e.g. for asan:
+// #define SANITIZER_NEW_STACK_TRACE GET_STACK_TRACE_MALLOC
+//
+// SANITIZER_NEW_REPORT_OOM(size)
+// The sanitizer's noreturn diagnostic path for chain-exhausted OOM.
+// Must call a function that does not return (the framework will run
+// UNREACHABLE() and abort if the macro's expansion ever returns).
+// e.g. for asan:
+// #define SANITIZER_NEW_REPORT_OOM(size) ReportOutOfMemory(size,
+// &stack)
+//
+// SANITIZER_NEW(size)
+// SANITIZER_NEW_ARRAY(size)
+// SANITIZER_NEW_ALIGNED(size, align)
+// SANITIZER_NEW_ARRAY_ALIGNED(size, align)
+// The sanitizer's four internal alloc helpers. Each must return nullptr
+// on OOM (so the chain-handling templates can drive the
+// std::get_new_handler() loop). Non-OOM failure modes (e.g. invalid
+// alignment) are UB/API misuse and should die with the helper's own
+// diagnostic rather than be routed through the framework.
+//
+// After the include, the eight OPERATOR_NEW_BODY* macros below are defined
+// and may be used as the bodies of the eight operator new overrides.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SANITIZER_NEW_STACK_TRACE
+# error "SANITIZER_NEW_STACK_TRACE must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_REPORT_OOM
+# error "SANITIZER_NEW_REPORT_OOM must be defined before including"
+#endif
+#ifndef SANITIZER_NEW
+# error "SANITIZER_NEW must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ARRAY
+# error "SANITIZER_NEW_ARRAY must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ALIGNED
+# error "SANITIZER_NEW_ALIGNED must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ARRAY_ALIGNED
+# error "SANITIZER_NEW_ARRAY_ALIGNED must be defined before including"
+#endif
+
+#define OPERATOR_NEW_BODY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ARRAY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ARRAY(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ARRAY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ARRAY(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_ARRAY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ARRAY_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ARRAY_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
More information about the llvm-commits
mailing list