[compiler-rt] [sanitizer_common] Add operator new chain-handling framework (PR #201151)
Justin T. Gibbs via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 25 15:52:40 PDT 2026
https://github.com/scsiguy updated https://github.com/llvm/llvm-project/pull/201151
>From 2785baa5a01a41323fe7f3a5d1ae5b35aefcb844 Mon Sep 17 00:00:00 2001
From: "Justin T. Gibbs" <gibbs at scsiguy.com>
Date: Sat, 30 May 2026 17:31:24 -0700
Subject: [PATCH] [sanitizer_common] Add operator new chain-handling framework
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Implement the operator new wrapper machinery required by
[new.delete.single]/3+/4 in shared sanitizer_common files so every
sanitizer can reuse it (avoids ~130 lines of duplication).
sanitizer_new_handler.h provides three main templates in namespace
__sanitizer (plus a NORETURN InvokeOnExhausted wrapper used internally):
* RunNewHandlerChain<Alloc>(alloc)
Runs std::get_new_handler() in a loop until either the
allocation succeeds or the chain is exhausted (returns nullptr).
* NewImplThrowing<Alloc, OnExhausted>(alloc, on_exhausted)
Throwing operator new: runs the chain; on exhaustion either
throws std::bad_alloc (AllocatorMayReturnNull()=true opts into
standards-conformant throw) or invokes on_exhausted (default;
historical abort-on-OOM).
* NewImplNothrow<Alloc, OnExhausted>(alloc, on_exhausted) noexcept
Nothrow operator new: per [new.delete.single]/4 behaves as-if
the throwing form is called within a try/catch converting any
exceptions to a nullptr return.
NOTE: Windows builds do not support exceptions, so the throwing
std::bad_alloc behavior described above is converted to an
invocation of on_exhausted(); a user new_handler that throws
on Windows trips the noexcept on NewImplNothrow and aborts via
std::terminate.
NOTE: The framework auto-detects exception support via the standard
__cpp_exceptions feature-test macro: when a consuming TU is
compiled with -fno-exceptions, the throw/try-catch logic is
compiled out and the framework collapses to the abort path
(same shape as Windows). This means TUs do not have to be
built with -fexceptions, and a -fno-exceptions adopter incurs
no std::bad_alloc symbol dependency. When -fexceptions IS used,
instantiating the throwing-form template introduces a runtime
dependency on std::bad_alloc — adopters must link a C++ ABI
library (libstdc++ / libc++abi) into the resulting runtime.
sanitizer_new_operators.inc builds the eight standard
OPERATOR_NEW_BODY* macros on top of these templates. A consuming
sanitizer defines six ingredient macros (a stack-trace setup,
a report-OOM invocation, and four alloc helpers) and then includes the
file. The resulting OPERATOR_NEW_BODY / OPERATOR_NEW_BODY_NOTHROW /
OPERATOR_NEW_BODY_ARRAY / ... / OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW
macros can then be used directly as the bodies of the eight operator new
overrides.
NFC. No consumer yet — this is a prerequisite for a follow-up that
restructures compiler-rt/lib/asan/asan_new_delete.cpp to use the
shared framework.
Assisted by: Claude Opus 4.7
---
.../lib/sanitizer_common/CMakeLists.txt | 2 +
.../sanitizer_common/sanitizer_new_handler.h | 122 ++++++++++++++++++
.../sanitizer_new_operators.inc | 114 ++++++++++++++++
3 files changed, 238 insertions(+)
create mode 100644 compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
create mode 100644 compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
diff --git a/compiler-rt/lib/sanitizer_common/CMakeLists.txt b/compiler-rt/lib/sanitizer_common/CMakeLists.txt
index 96c23c6d8ab82..07e9cad572b3d 100644
--- a/compiler-rt/lib/sanitizer_common/CMakeLists.txt
+++ b/compiler-rt/lib/sanitizer_common/CMakeLists.txt
@@ -170,6 +170,8 @@ set(SANITIZER_IMPL_HEADERS
sanitizer_mac.h
sanitizer_malloc_mac.inc
sanitizer_mutex.h
+ sanitizer_new_handler.h
+ sanitizer_new_operators.inc
sanitizer_placement_new.h
sanitizer_platform.h
sanitizer_platform_interceptors.h
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
new file mode 100644
index 0000000000000..6a5a361ec6324
--- /dev/null
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
@@ -0,0 +1,122 @@
+//===-- sanitizer_new_handler.h ---------------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file is shared between run-time libraries of sanitizers.
+//
+// It provides the operator new chain-handling framework required by
+// [new.delete.single]/3+/4: a std::get_new_handler() loop plus the
+// throwing / nothrow exhaustion policies that compose with it. Each
+// sanitizer's operator new wrapper supplies two small lambdas:
+//
+// * Alloc — invokes the sanitizer's internal allocator, returning
+// nullptr on OOM (never aborting on OOM). Other detected
+// failure modes (e.g. invalid alignment) should abort with
+// a diagnostic.
+//
+// * OnExhausted — invokes the sanitizer's "abort with diagnostic"
+// handler (e.g. asan's ReportOutOfMemory + Die()).
+// Required to never return (enforced by UNREACHABLE()).
+//===----------------------------------------------------------------------===//
+#ifndef SANITIZER_NEW_HANDLER_H
+#define SANITIZER_NEW_HANDLER_H
+
+#include "sanitizer_allocator.h"
+#include "sanitizer_internal_defs.h"
+#include "sanitizer_platform.h"
+
+#if !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+# include <new>
+#else
+// Builds without exception support can't include <new>: on Windows the
+// sanitizer runtimes are built without exceptions; elsewhere this TU may
+// have been compiled with -fno-exceptions (e.g. with
+// -DCOMPILER_RT_ASAN_ENABLE_EXCEPTIONS=OFF). Forward-declare just enough
+// of std for the loop; the real std::get_new_handler is supplied by the
+// C++ runtime the sanitizer links against (vcruntime / msvcprt /
+// MinGW libstdc++ / libstdc++ / libc++) — it doesn't need to come from
+// <new>.
+namespace std {
+struct nothrow_t {};
+enum class align_val_t : __sanitizer::usize {};
+using new_handler = void (*)();
+new_handler get_new_handler() noexcept;
+} // namespace std
+#endif // !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+
+namespace __sanitizer {
+
+// Runs std::get_new_handler() per [new.delete.single]/3+/4 until the
+// allocation succeeds or the chain is exhausted. Returns the allocated
+// pointer on success, nullptr if the handler chain is exhausted.
+//
+// NOTE: Exceptions thrown by Alloc or std::new_handler callbacks escape this
+// function. Callers that need to convert exceptions to a nullptr return
+// (e.g. NewImplNothrow below) must wrap the call in try/catch themselves.
+template <typename Alloc>
+void* RunNewHandlerChain(Alloc alloc) {
+ for (;;) {
+ void* res = alloc();
+ if (LIKELY(res != nullptr))
+ return res;
+ std::new_handler handler = std::get_new_handler();
+ if (!handler)
+ return nullptr;
+ handler();
+ }
+}
+
+// NORETURN wrapper providing pre-C++23 compatibility.
+template <typename OnExhausted>
+NORETURN void InvokeOnExhausted(OnExhausted on_exhausted) {
+ on_exhausted();
+ UNREACHABLE("operator new OnExhausted callable returned");
+}
+
+// Throwing operator new: chain, then on exhaustion throw std::bad_alloc
+// when this TU was compiled with exception support and AllocatorMayReturnNull()
+// is true. Otherwise (Windows runtimes, -fno-exceptions builds, or the
+// default flag value) fall through to the abort path.
+template <typename Alloc, typename OnExhausted>
+void* NewImplThrowing(Alloc alloc, OnExhausted on_exhausted) {
+ void* res = RunNewHandlerChain(alloc);
+ if (LIKELY(res != nullptr))
+ return res;
+#if !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+ if (AllocatorMayReturnNull())
+ throw std::bad_alloc();
+#endif
+ InvokeOnExhausted(on_exhausted);
+}
+
+// Nothrow operator new: per [new.delete.single]/4 behaves as-if the
+// throwing form is called within a try/catch. When exception support is
+// absent (Windows runtimes or -fno-exceptions TUs) we can't write the
+// try/catch literally — instead we avoid the throw path entirely, so the
+// main body matches NewImplThrowing with "throw std::bad_alloc()" replaced
+// by "return nullptr".
+template <typename Alloc, typename OnExhausted>
+void* NewImplNothrow(Alloc alloc, OnExhausted on_exhausted) noexcept {
+#if !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+ try {
+#endif
+ void* res = RunNewHandlerChain(alloc);
+ if (LIKELY(res != nullptr))
+ return res;
+ if (AllocatorMayReturnNull())
+ return nullptr;
+ InvokeOnExhausted(on_exhausted);
+#if !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+ } catch (...) {
+ return nullptr;
+ }
+#endif
+}
+
+} // namespace __sanitizer
+
+#endif // SANITIZER_NEW_HANDLER_H
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
new file mode 100644
index 0000000000000..66dd83f2fddc5
--- /dev/null
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
@@ -0,0 +1,114 @@
+//===-- sanitizer_new_operators.inc -----------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Shared OPERATOR_NEW_BODY* macros for sanitizer operator new overrides.
+//
+// All eight variants compose the same per-call setup (a sanitizer-specific
+// stack-trace acquisition) with one of the two chain-handling templates from
+// sanitizer_new_handler.h and two sanitizer-supplied lambdas: an Alloc that
+// invokes the sanitizer's internal allocator (returning nullptr on failure)
+// and an OnExhausted that calls the sanitizer's "abort with diagnostic"
+// handler.
+//
+// This file is included after the consuming TU has defined the prerequisite
+// macros listed below. Names reference symbols (e.g. `stack`, `size`) that
+// exist only in the macro's expansion context, so this is an .inc, not a
+// freestanding header.
+//
+// The consuming TU must define before including:
+//
+// SANITIZER_NEW_STACK_TRACE
+// A statement that, when expanded inside an operator new body, puts a
+// BufferedStackTrace named `stack` in scope. e.g. for asan:
+// #define SANITIZER_NEW_STACK_TRACE GET_STACK_TRACE_MALLOC
+//
+// SANITIZER_NEW_REPORT_OOM(size)
+// The sanitizer's noreturn diagnostic path for chain-exhausted OOM.
+// Must call a function that does not return (the framework will run
+// UNREACHABLE() and abort if the macro's expansion ever returns).
+// e.g. for asan:
+// #define SANITIZER_NEW_REPORT_OOM(size) \
+// ReportOutOfMemory(size, &stack)
+//
+// SANITIZER_NEW(size)
+// SANITIZER_NEW_ARRAY(size)
+// SANITIZER_NEW_ALIGNED(size, align)
+// SANITIZER_NEW_ARRAY_ALIGNED(size, align)
+// The sanitizer's four internal alloc helpers. Each must return nullptr
+// on OOM (so the chain-handling templates can drive the
+// std::get_new_handler() loop). Other failure modes (e.g. invalid
+// alignment) should cause the helper to abort with a diagnostic.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef SANITIZER_NEW_STACK_TRACE
+# error "SANITIZER_NEW_STACK_TRACE must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_REPORT_OOM
+# error "SANITIZER_NEW_REPORT_OOM must be defined before including"
+#endif
+#ifndef SANITIZER_NEW
+# error "SANITIZER_NEW must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ARRAY
+# error "SANITIZER_NEW_ARRAY must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ALIGNED
+# error "SANITIZER_NEW_ALIGNED must be defined before including"
+#endif
+#ifndef SANITIZER_NEW_ARRAY_ALIGNED
+# error "SANITIZER_NEW_ARRAY_ALIGNED must be defined before including"
+#endif
+
+#define OPERATOR_NEW_BODY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ARRAY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ARRAY(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ARRAY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ARRAY(size); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_ARRAY \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplThrowing( \
+ [&]() { return SANITIZER_NEW_ARRAY_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
+
+#define OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW \
+ SANITIZER_NEW_STACK_TRACE; \
+ return __sanitizer::NewImplNothrow( \
+ [&]() { return SANITIZER_NEW_ARRAY_ALIGNED(size, align); }, \
+ [&]() { SANITIZER_NEW_REPORT_OOM(size); })
More information about the llvm-commits
mailing list