[compiler-rt] [llvm] [asan] Adopt sanitizer_common operator-new framework (PR #196388)
Justin T. Gibbs via llvm-commits
llvm-commits at lists.llvm.org
Sun Sep 27 13:24:44 PDT 2026
https://github.com/scsiguy updated https://github.com/llvm/llvm-project/pull/196388
>From 24ddd5aa5630df892cebe03034956d0efe2583cd Mon Sep 17 00:00:00 2001
From: "Justin T. Gibbs" <gibbs at scsiguy.com>
Date: Tue, 22 Sep 2026 09:44:19 -0700
Subject: [PATCH 1/3] [sanitizer] Always report invalid aligned operator new
requests
Aligned `operator new` in every sanitizer now reports an invalid
`std::align_val_t` (not a positive power of two) with
`ReportInvalidAllocationAlignment()`, regardless of
`allocator_may_return_null`. This ensures the caller's UB API usage is
clearly reported instead of being treated as an out-of-storage failure.
Previously, all sanitizers but ASan also silently accepted an alignment
of zero.
The operator new framework comments now state the resulting contract:
an allocation callback returns nullptr only when storage cannot be
obtained.
Test Plan:
- New sanitizer_common test covers all four aligned overloads.
- NSan, MemProf and DFSan (unsupported by sanitizer_common tests) have
their own tests of the single-object overloads, which share one
implementation with the array overloads.
History:
Every sanitizer implemented aligned `operator new` on top of its C
`memalign`, inheriting that API's flag-gated return of
`nullptr+errno=EINVAL` for invalid alignment. ASan later moved aligned
new to a dedicated helper that copied the policy (see table); the other
runtimes still call `memalign` directly. C++ does not require a program
to continue after this UB. Removing the flag-controlled path makes a
nullptr return *always* mean an out-of-storage failure, one the
`std::get_new_handler()` loop can retry.
ASan's history, as an example:
| Commit | Year | Change |
| -------------- | ---- | -------------------------------------------- |
| `c7cc93ad0723` | 2016 | Aligned `operator new` added as a call to |
| | | `asan_memalign()` (D24771). |
| `31e8173c9425` | 2017 | `asan_memalign()` gains the flag-gated |
| | | `EINVAL` path for C `memalign` (D35440). |
| `10f50a44c1fa` | 2018 | That path becomes an explicit |
| | | `AllocatorMayReturnNull()` check plus |
| | | `ReportInvalidAllocationAlignment()` |
| | | (D44404). |
| `681c2ee4dfbf` | 2025 | The dedicated `asan_new_aligned()` helper |
| | | copies the branch (#145087). |
Assisted-by: Claude Opus 5.5
---
compiler-rt/lib/asan/asan_allocator.cpp | 6 +-
compiler-rt/lib/dfsan/dfsan_new_delete.cpp | 17 ++++--
compiler-rt/lib/hwasan/hwasan_new_delete.cpp | 20 +++++--
compiler-rt/lib/lsan/lsan_interceptors.cpp | 14 +++--
.../lib/memprof/memprof_new_delete.cpp | 3 +
compiler-rt/lib/msan/msan_new_delete.cpp | 21 ++++---
compiler-rt/lib/nsan/nsan_new_delete.cpp | 6 ++
.../sanitizer_allocator_checks.h | 6 ++
.../sanitizer_common/sanitizer_new_handler.h | 14 +++--
.../sanitizer_new_operators.inc | 13 +++--
compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp | 31 ++++++-----
.../test/dfsan/invalid_aligned_new.cpp | 43 +++++++++++++++
.../memprof/TestCases/invalid_aligned_new.cpp | 34 ++++++++++++
compiler-rt/test/nsan/invalid_aligned_new.cpp | 34 ++++++++++++
.../TestCases/invalid_aligned_new.cpp | 55 +++++++++++++++++++
15 files changed, 263 insertions(+), 54 deletions(-)
create mode 100644 compiler-rt/test/dfsan/invalid_aligned_new.cpp
create mode 100644 compiler-rt/test/memprof/TestCases/invalid_aligned_new.cpp
create mode 100644 compiler-rt/test/nsan/invalid_aligned_new.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/invalid_aligned_new.cpp
diff --git a/compiler-rt/lib/asan/asan_allocator.cpp b/compiler-rt/lib/asan/asan_allocator.cpp
index 6af197ab4cb1c..b75a8ea09e34a 100644
--- a/compiler-rt/lib/asan/asan_allocator.cpp
+++ b/compiler-rt/lib/asan/asan_allocator.cpp
@@ -1214,12 +1214,8 @@ void* asan_new(uptr size, BufferedStackTrace* stack, bool array) {
void* asan_new_aligned(uptr size, uptr alignment, BufferedStackTrace* stack,
bool array) {
- if (UNLIKELY(alignment == 0 || !IsPowerOfTwo(alignment))) {
- errno = errno_EINVAL;
- if (AllocatorMayReturnNull())
- return nullptr;
+ if (UNLIKELY(!CheckAlignedNewAlignment(alignment)))
ReportInvalidAllocationAlignment(alignment, stack);
- }
return SetErrnoOnNull(instance.Allocate(size, alignment, stack,
array ? FROM_NEW_BR : FROM_NEW,
/*can_fill=*/true));
diff --git a/compiler-rt/lib/dfsan/dfsan_new_delete.cpp b/compiler-rt/lib/dfsan/dfsan_new_delete.cpp
index 4482e22951040..0031ae9737b0f 100644
--- a/compiler-rt/lib/dfsan/dfsan_new_delete.cpp
+++ b/compiler-rt/lib/dfsan/dfsan_new_delete.cpp
@@ -16,6 +16,7 @@
#include "dfsan.h"
#include "interception/interception.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
using namespace __dfsan;
@@ -34,12 +35,16 @@ enum class align_val_t : size_t {};
ReportOutOfMemory(size, &stack); \
} \
return res
-#define OPERATOR_NEW_BODY_ALIGN(nothrow) \
- void *res = dfsan_memalign((uptr)align, size); \
- if (!nothrow && UNLIKELY(!res)) { \
- UNINITIALIZED BufferedStackTrace stack; \
- ReportOutOfMemory(size, &stack); \
- } \
+#define OPERATOR_NEW_BODY_ALIGN(nothrow) \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) { \
+ UNINITIALIZED BufferedStackTrace stack; \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
+ } \
+ void* res = dfsan_memalign((uptr)align, size); \
+ if (!nothrow && UNLIKELY(!res)) { \
+ UNINITIALIZED BufferedStackTrace stack; \
+ ReportOutOfMemory(size, &stack); \
+ } \
return res;
INTERCEPTOR_ATTRIBUTE
diff --git a/compiler-rt/lib/hwasan/hwasan_new_delete.cpp b/compiler-rt/lib/hwasan/hwasan_new_delete.cpp
index 232eb0eb6da67..f4a32476e3775 100644
--- a/compiler-rt/lib/hwasan/hwasan_new_delete.cpp
+++ b/compiler-rt/lib/hwasan/hwasan_new_delete.cpp
@@ -11,14 +11,15 @@
// Interceptors for operators new and delete.
//===----------------------------------------------------------------------===//
+#include <stddef.h>
+#include <stdlib.h>
+
#include "hwasan.h"
#include "interception/interception.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
-#include <stddef.h>
-#include <stdlib.h>
-
#if HWASAN_REPLACE_OPERATORS_NEW_AND_DELETE
// TODO(alekseys): throw std::bad_alloc instead of dying on OOM.
@@ -40,23 +41,32 @@
# define OPERATOR_NEW_BODY_ARRAY_NOTHROW \
GET_MALLOC_STACK_TRACE; \
return hwasan_malloc(size, &stack)
+# define CHECK_ALIGNED_NEW_ALIGNMENT \
+ do { \
+ if (UNLIKELY(!CheckAlignedNewAlignment(static_cast<uptr>(align)))) \
+ ReportInvalidAllocationAlignment(static_cast<uptr>(align), &stack); \
+ } while (0)
# define OPERATOR_NEW_BODY_ALIGN \
GET_MALLOC_STACK_TRACE; \
- void *res = hwasan_memalign(static_cast<uptr>(align), size, &stack); \
+ CHECK_ALIGNED_NEW_ALIGNMENT; \
+ void* res = hwasan_memalign(static_cast<uptr>(align), size, &stack); \
if (UNLIKELY(!res)) \
ReportOutOfMemory(size, &stack); \
return res
# define OPERATOR_NEW_BODY_ALIGN_NOTHROW \
GET_MALLOC_STACK_TRACE; \
+ CHECK_ALIGNED_NEW_ALIGNMENT; \
return hwasan_memalign(static_cast<uptr>(align), size, &stack)
# define OPERATOR_NEW_BODY_ALIGN_ARRAY \
GET_MALLOC_STACK_TRACE; \
- void *res = hwasan_memalign(static_cast<uptr>(align), size, &stack); \
+ CHECK_ALIGNED_NEW_ALIGNMENT; \
+ void* res = hwasan_memalign(static_cast<uptr>(align), size, &stack); \
if (UNLIKELY(!res)) \
ReportOutOfMemory(size, &stack); \
return res
# define OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW \
GET_MALLOC_STACK_TRACE; \
+ CHECK_ALIGNED_NEW_ALIGNMENT; \
return hwasan_memalign(static_cast<uptr>(align), size, &stack)
# define OPERATOR_DELETE_BODY \
diff --git a/compiler-rt/lib/lsan/lsan_interceptors.cpp b/compiler-rt/lib/lsan/lsan_interceptors.cpp
index 5340c6ffba607..13bc6f22b2060 100644
--- a/compiler-rt/lib/lsan/lsan_interceptors.cpp
+++ b/compiler-rt/lib/lsan/lsan_interceptors.cpp
@@ -13,6 +13,7 @@
#include "interception/interception.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_dlsym.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
#include "sanitizer_common/sanitizer_atomic.h"
@@ -256,11 +257,14 @@ INTERCEPTOR(int, mprobe, void *ptr) {
void *res = lsan_malloc(size, stack);\
if (!nothrow && UNLIKELY(!res)) ReportOutOfMemory(size, &stack);\
return res;
-#define OPERATOR_NEW_BODY_ALIGN(nothrow)\
- ENSURE_LSAN_INITED;\
- GET_STACK_TRACE_MALLOC;\
- void *res = lsan_memalign((uptr)align, size, stack);\
- if (!nothrow && UNLIKELY(!res)) ReportOutOfMemory(size, &stack);\
+#define OPERATOR_NEW_BODY_ALIGN(nothrow) \
+ ENSURE_LSAN_INITED; \
+ GET_STACK_TRACE_MALLOC; \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
+ void* res = lsan_memalign((uptr)align, size, stack); \
+ if (!nothrow && UNLIKELY(!res)) \
+ ReportOutOfMemory(size, &stack); \
return res;
#define OPERATOR_DELETE_BODY\
diff --git a/compiler-rt/lib/memprof/memprof_new_delete.cpp b/compiler-rt/lib/memprof/memprof_new_delete.cpp
index cae5de301367a..6def879c959c1 100644
--- a/compiler-rt/lib/memprof/memprof_new_delete.cpp
+++ b/compiler-rt/lib/memprof/memprof_new_delete.cpp
@@ -14,6 +14,7 @@
#include "memprof_allocator.h"
#include "memprof_internal.h"
#include "memprof_stack.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
#include "interception/interception.h"
@@ -38,6 +39,8 @@ enum class align_val_t : size_t {};
return res;
#define OPERATOR_NEW_BODY_ALIGN(type, nothrow) \
GET_STACK_TRACE_MALLOC; \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
void *res = memprof_memalign((uptr)align, size, &stack, type); \
if (!nothrow && UNLIKELY(!res)) \
ReportOutOfMemory(size, &stack); \
diff --git a/compiler-rt/lib/msan/msan_new_delete.cpp b/compiler-rt/lib/msan/msan_new_delete.cpp
index 7daa55474b7da..177e02e8044cc 100644
--- a/compiler-rt/lib/msan/msan_new_delete.cpp
+++ b/compiler-rt/lib/msan/msan_new_delete.cpp
@@ -11,9 +11,10 @@
// Interceptors for operators new and delete.
//===----------------------------------------------------------------------===//
-#include "msan.h"
#include "interception/interception.h"
+#include "msan.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
#if MSAN_REPLACE_OPERATORS_NEW_AND_DELETE
@@ -38,13 +39,17 @@ namespace std {
ReportOutOfMemory(size, &stack); \
} \
return res
-# define OPERATOR_NEW_BODY_ALIGN(nothrow) \
- GET_MALLOC_STACK_TRACE; \
- void *res = msan_memalign((uptr)align, size, &stack); \
- if (!nothrow && UNLIKELY(!res)) { \
- GET_FATAL_STACK_TRACE_IF_EMPTY(&stack); \
- ReportOutOfMemory(size, &stack); \
- } \
+# define OPERATOR_NEW_BODY_ALIGN(nothrow) \
+ GET_MALLOC_STACK_TRACE; \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) { \
+ GET_FATAL_STACK_TRACE_IF_EMPTY(&stack); \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
+ } \
+ void* res = msan_memalign((uptr)align, size, &stack); \
+ if (!nothrow && UNLIKELY(!res)) { \
+ GET_FATAL_STACK_TRACE_IF_EMPTY(&stack); \
+ ReportOutOfMemory(size, &stack); \
+ } \
return res;
INTERCEPTOR_ATTRIBUTE
diff --git a/compiler-rt/lib/nsan/nsan_new_delete.cpp b/compiler-rt/lib/nsan/nsan_new_delete.cpp
index f203a583f2c44..0101e19d202c0 100644
--- a/compiler-rt/lib/nsan/nsan_new_delete.cpp
+++ b/compiler-rt/lib/nsan/nsan_new_delete.cpp
@@ -15,6 +15,7 @@
#include "nsan.h"
#include "nsan_allocator.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
#include <stddef.h>
@@ -36,6 +37,11 @@ enum class align_val_t : size_t {};
} \
return res
#define OPERATOR_NEW_BODY_ALIGN(nothrow) \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) { \
+ BufferedStackTrace stack; \
+ GET_FATAL_STACK_TRACE_IF_EMPTY(&stack); \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
+ } \
void *res = nsan_memalign((uptr)align, size); \
if (!nothrow && UNLIKELY(!res)) { \
BufferedStackTrace stack; \
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_allocator_checks.h b/compiler-rt/lib/sanitizer_common/sanitizer_allocator_checks.h
index 1cc3992c4c9fa..af3f3a278c2a5 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_allocator_checks.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_allocator_checks.h
@@ -57,6 +57,12 @@ inline bool CheckPosixMemalignAlignment(uptr alignment) {
(alignment % sizeof(void *)) == 0;
}
+// Checks the std::align_val_t argument of aligned operator new, verifies that
+// the alignment is a power of two.
+inline bool CheckAlignedNewAlignment(uptr alignment) {
+ return alignment != 0 && IsPowerOfTwo(alignment);
+}
+
// Returns true if calloc(size, n) call overflows on size*n calculation.
inline bool CheckForCallocOverflow(uptr size, uptr n) {
if (!size)
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
index 0b52796836c96..706b38fc7cdd4 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_handler.h
@@ -13,10 +13,12 @@
// throwing / nothrow exhaustion policies that compose with it. Each
// sanitizer's operator new wrapper supplies two small lambdas:
//
-// * Alloc — invokes the sanitizer's internal allocator, returning
-// nullptr on OOM (never aborting on OOM). Other detected
-// failure modes (e.g. invalid alignment) should abort with
-// a diagnostic.
+// * Alloc — invokes the sanitizer's internal allocator. Returns
+// nullptr, rather than aborting, only when storage cannot
+// be obtained: every nullptr return is treated as a
+// storage failure by the std::get_new_handler() loop.
+// Other detected failures (e.g. invalid alignment) must
+// abort with a diagnostic.
//
// * OnExhausted — invokes the sanitizer's "abort with diagnostic"
// handler (e.g. asan's ReportOutOfMemory + Die()).
@@ -51,8 +53,8 @@ new_handler get_new_handler() noexcept;
namespace __sanitizer {
// Runs std::get_new_handler() per [new.delete.single]/3+/4 until the
-// allocation succeeds or the chain is exhausted. Returns the allocated
-// pointer on success, nullptr if the handler chain is exhausted.
+// allocation succeeds (returns the allocated pointer) or the chain is
+// exhausted (returns nullptr).
//
// NOTE: Exceptions thrown by Alloc or std::new_handler callbacks escape this
// function. Callers that need to convert exceptions to a nullptr return
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
index 66dd83f2fddc5..a81426bc17c74 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_new_operators.inc
@@ -11,9 +11,9 @@
// All eight variants compose the same per-call setup (a sanitizer-specific
// stack-trace acquisition) with one of the two chain-handling templates from
// sanitizer_new_handler.h and two sanitizer-supplied lambdas: an Alloc that
-// invokes the sanitizer's internal allocator (returning nullptr on failure)
-// and an OnExhausted that calls the sanitizer's "abort with diagnostic"
-// handler.
+// invokes the sanitizer's internal allocator (returning nullptr only when
+// storage cannot be obtained) and an OnExhausted that calls the sanitizer's
+// "abort with diagnostic" handler.
//
// This file is included after the consuming TU has defined the prerequisite
// macros listed below. Names reference symbols (e.g. `stack`, `size`) that
@@ -40,9 +40,10 @@
// SANITIZER_NEW_ALIGNED(size, align)
// SANITIZER_NEW_ARRAY_ALIGNED(size, align)
// The sanitizer's four internal alloc helpers. Each must return nullptr
-// on OOM (so the chain-handling templates can drive the
-// std::get_new_handler() loop). Other failure modes (e.g. invalid
-// alignment) should cause the helper to abort with a diagnostic.
+// only when storage cannot be obtained: every nullptr return is treated
+// as a storage failure by the std::get_new_handler() loop. Other failure
+// modes (e.g. invalid alignment) must cause the helper to abort with a
+// diagnostic.
//
//===----------------------------------------------------------------------===//
diff --git a/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp b/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp
index fc44a5221b5b0..6314348e5d4c5 100644
--- a/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp
+++ b/compiler-rt/lib/tsan/rtl/tsan_new_delete.cpp
@@ -12,6 +12,7 @@
//===----------------------------------------------------------------------===//
#include "interception/interception.h"
#include "sanitizer_common/sanitizer_allocator.h"
+#include "sanitizer_common/sanitizer_allocator_checks.h"
#include "sanitizer_common/sanitizer_allocator_report.h"
#include "sanitizer_common/sanitizer_internal_defs.h"
#include "tsan_interceptors.h"
@@ -43,19 +44,23 @@ DECLARE_REAL(void, free, void *ptr)
invoke_malloc_hook(p, size); \
return p;
-#define OPERATOR_NEW_BODY_ALIGN(mangled_name, nothrow) \
- if (in_symbolizer()) \
- return InternalAlloc(size, nullptr, (uptr)align); \
- void *p = 0; \
- { \
- SCOPED_INTERCEPTOR_RAW(mangled_name, size); \
- p = user_memalign(thr, pc, (uptr)align, size); \
- if (!nothrow && UNLIKELY(!p)) { \
- GET_STACK_TRACE_FATAL(thr, pc); \
- ReportOutOfMemory(size, &stack); \
- } \
- } \
- invoke_malloc_hook(p, size); \
+#define OPERATOR_NEW_BODY_ALIGN(mangled_name, nothrow) \
+ if (in_symbolizer()) \
+ return InternalAlloc(size, nullptr, (uptr)align); \
+ void* p = 0; \
+ { \
+ SCOPED_INTERCEPTOR_RAW(mangled_name, size); \
+ if (UNLIKELY(!CheckAlignedNewAlignment((uptr)align))) { \
+ GET_STACK_TRACE_FATAL(thr, pc); \
+ ReportInvalidAllocationAlignment((uptr)align, &stack); \
+ } \
+ p = user_memalign(thr, pc, (uptr)align, size); \
+ if (!nothrow && UNLIKELY(!p)) { \
+ GET_STACK_TRACE_FATAL(thr, pc); \
+ ReportOutOfMemory(size, &stack); \
+ } \
+ } \
+ invoke_malloc_hook(p, size); \
return p;
SANITIZER_INTERFACE_ATTRIBUTE
diff --git a/compiler-rt/test/dfsan/invalid_aligned_new.cpp b/compiler-rt/test/dfsan/invalid_aligned_new.cpp
new file mode 100644
index 0000000000000..1018615224d53
--- /dev/null
+++ b/compiler-rt/test/dfsan/invalid_aligned_new.cpp
@@ -0,0 +1,43 @@
+// Invalid alignment is a caller error, not a failure to obtain storage. DFSan
+// must report it, even with allocator_may_return_null=1, rather than return
+// nullptr. DFSan's operator new overrides serve uninstrumented code, so the
+// allocation happens in an uninstrumented object linked into the program.
+
+// RUN: %clangxx_dfsan -fno-sanitize=dataflow -std=c++17 -DLIB -c %s -o %t-lib.o
+// RUN: echo 'fun:AllocAligned=uninstrumented' > %t.abilist
+// RUN: echo 'fun:AllocAligned=discard' >> %t.abilist
+// RUN: %clangxx_dfsan -std=c++17 -fsanitize-ignorelist=%t.abilist %s %t-lib.o -o %t
+// RUN: env DFSAN_OPTIONS=allocator_may_return_null=1 not %run %t new 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: env DFSAN_OPTIONS=allocator_may_return_null=1 not %run %t new-nothrow 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: env DFSAN_OPTIONS=allocator_may_return_null=1 not %run %t new 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: env DFSAN_OPTIONS=allocator_may_return_null=1 not %run %t new-nothrow 3 2>&1 | FileCheck %s -DALIGN=3
+
+#include <cstddef>
+
+extern "C" void *AllocAligned(bool nothrow, std::size_t alignment);
+
+#ifdef LIB
+# include <new>
+
+extern "C" void *AllocAligned(bool nothrow, std::size_t alignment) {
+ const auto align = static_cast<std::align_val_t>(alignment);
+ return nothrow ? ::operator new(16, align, std::nothrow)
+ : ::operator new(16, align);
+}
+#else
+# include <cassert>
+# include <cstdio>
+# include <cstdlib>
+# include <cstring>
+
+int main(int argc, char **argv) {
+ assert(argc == 3);
+ const bool nothrow = std::strcmp(argv[1], "new-nothrow") == 0;
+ assert(nothrow || std::strcmp(argv[1], "new") == 0);
+ void *p = AllocAligned(nothrow, std::strtoull(argv[2], nullptr, 0));
+ std::fprintf(stderr, "allocation unexpectedly returned %p\n", p);
+ return 1;
+}
+#endif
+
+// CHECK: ERROR: DataflowSanitizer: invalid allocation alignment: [[ALIGN]],
diff --git a/compiler-rt/test/memprof/TestCases/invalid_aligned_new.cpp b/compiler-rt/test/memprof/TestCases/invalid_aligned_new.cpp
new file mode 100644
index 0000000000000..5bc237c693c09
--- /dev/null
+++ b/compiler-rt/test/memprof/TestCases/invalid_aligned_new.cpp
@@ -0,0 +1,34 @@
+// Invalid alignment is a caller error, not a failure to obtain storage.
+// MemProf must report it, even with allocator_may_return_null=1, rather than
+// return nullptr.
+
+// RUN: %clangxx_memprof -O0 -std=c++17 %s -o %t
+// RUN: %env_memprof_opts=log_path=stderr:allocator_may_return_null=1 not %run %t new 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_memprof_opts=log_path=stderr:allocator_may_return_null=1 not %run %t new-nothrow 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_memprof_opts=log_path=stderr:allocator_may_return_null=1 not %run %t new 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: %env_memprof_opts=log_path=stderr:allocator_may_return_null=1 not %run %t new-nothrow 3 2>&1 | FileCheck %s -DALIGN=3
+
+#include <cassert>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <new>
+
+int main(int argc, char **argv) {
+ assert(argc == 3);
+ const auto alignment =
+ static_cast<std::align_val_t>(std::strtoull(argv[2], nullptr, 0));
+
+ void *p;
+ if (std::strcmp(argv[1], "new") == 0)
+ p = ::operator new(16, alignment);
+ else if (std::strcmp(argv[1], "new-nothrow") == 0)
+ p = ::operator new(16, alignment, std::nothrow);
+ else
+ assert(false);
+
+ std::fprintf(stderr, "allocation unexpectedly returned %p\n", p);
+ return 1;
+}
+
+// CHECK: ERROR: MemProfiler: invalid allocation alignment: [[ALIGN]],
diff --git a/compiler-rt/test/nsan/invalid_aligned_new.cpp b/compiler-rt/test/nsan/invalid_aligned_new.cpp
new file mode 100644
index 0000000000000..e431024b56e8d
--- /dev/null
+++ b/compiler-rt/test/nsan/invalid_aligned_new.cpp
@@ -0,0 +1,34 @@
+// Invalid alignment is a caller error, not a failure to obtain storage. NSan
+// must report it, even with allocator_may_return_null=1, rather than return
+// nullptr.
+
+// RUN: %clangxx_nsan -O0 %s -o %t
+// RUN: env NSAN_OPTIONS=allocator_may_return_null=1 not %run %t new 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: env NSAN_OPTIONS=allocator_may_return_null=1 not %run %t new-nothrow 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: env NSAN_OPTIONS=allocator_may_return_null=1 not %run %t new 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: env NSAN_OPTIONS=allocator_may_return_null=1 not %run %t new-nothrow 3 2>&1 | FileCheck %s -DALIGN=3
+
+#include <cassert>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <new>
+
+int main(int argc, char **argv) {
+ assert(argc == 3);
+ const auto alignment =
+ static_cast<std::align_val_t>(std::strtoull(argv[2], nullptr, 0));
+
+ void *p;
+ if (std::strcmp(argv[1], "new") == 0)
+ p = ::operator new(16, alignment);
+ else if (std::strcmp(argv[1], "new-nothrow") == 0)
+ p = ::operator new(16, alignment, std::nothrow);
+ else
+ assert(false);
+
+ std::fprintf(stderr, "allocation unexpectedly returned %p\n", p);
+ return 1;
+}
+
+// CHECK: ERROR: NumericalStabilitySanitizer: invalid allocation alignment: [[ALIGN]],
diff --git a/compiler-rt/test/sanitizer_common/TestCases/invalid_aligned_new.cpp b/compiler-rt/test/sanitizer_common/TestCases/invalid_aligned_new.cpp
new file mode 100644
index 0000000000000..b47d0ca0c2c6d
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/invalid_aligned_new.cpp
@@ -0,0 +1,55 @@
+// Invalid alignment is a caller error, not a failure to obtain storage. The
+// sanitizer must report it, even with allocator_may_return_null=1, rather than
+// return nullptr or call std::new_handler.
+
+// RUN: %clangxx -O0 -std=c++17 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-array 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-nothrow 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-array-nothrow 0 2>&1 | FileCheck %s -DALIGN=0
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-array 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-nothrow 3 2>&1 | FileCheck %s -DALIGN=3
+// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new-array-nothrow 3 2>&1 | FileCheck %s -DALIGN=3
+
+// ubsan does not replace operator new. ASan and LSan do not override aligned
+// operator new on Darwin, nor ASan on Windows (MSVC).
+// UNSUPPORTED: ubsan, (asan || lsan) && darwin, target={{.*windows-msvc.*}}
+// REQUIRES: stable-runtime
+
+#include <cassert>
+#include <cstdio>
+#include <cstdlib>
+#include <cstring>
+#include <new>
+
+static void NewHandler() {
+ std::fputs("new_handler called\n", stderr);
+ std::abort();
+}
+
+int main(int argc, char **argv) {
+ assert(argc == 3);
+ std::set_new_handler(NewHandler);
+ const auto alignment =
+ static_cast<std::align_val_t>(std::strtoull(argv[2], nullptr, 0));
+
+ void *p;
+ if (std::strcmp(argv[1], "new") == 0)
+ p = ::operator new(16, alignment);
+ else if (std::strcmp(argv[1], "new-array") == 0)
+ p = ::operator new[](16, alignment);
+ else if (std::strcmp(argv[1], "new-nothrow") == 0)
+ p = ::operator new(16, alignment, std::nothrow);
+ else if (std::strcmp(argv[1], "new-array-nothrow") == 0)
+ p = ::operator new[](16, alignment, std::nothrow);
+ else
+ assert(false);
+
+ std::fprintf(stderr, "allocation unexpectedly returned %p\n", p);
+ return 1;
+}
+
+// CHECK-NOT: new_handler called
+// CHECK: ERROR: {{.*}}Sanitizer: invalid allocation alignment: [[ALIGN]],
+// CHECK: SUMMARY: {{.*}}Sanitizer: invalid-allocation-alignment
>From 5860ba43f5962906f7640dd72530435dfd5b496b Mon Sep 17 00:00:00 2001
From: "Justin T. Gibbs" <gibbs at scsiguy.com>
Date: Fri, 25 Sep 2026 10:20:17 -0700
Subject: [PATCH 2/3] [sanitizer_common] Add unit tests for operator new
failure handling
Test `RunNewHandlerChain`, `NewImplThrowing`, `NewImplNothrow` and
`InvokeOnExhausted` with fake allocation and exhaustion callbacks. The
tests cover handler retries, chain exhaustion, exceptions thrown by a
`std::new_handler`, and the `allocator_may_return_null` exhaustion
policy for throwing and nothrow `operator new`. Sanitizers that adopt
the framework then only need to test their own wiring.
The tests build only where the header includes `<new>` and supports
exceptions, which excludes Windows.
Assisted-by: Claude Opus 5.5
---
.../lib/sanitizer_common/tests/CMakeLists.txt | 1 +
.../tests/sanitizer_new_handler_test.cpp | 202 ++++++++++++++++++
2 files changed, 203 insertions(+)
create mode 100644 compiler-rt/lib/sanitizer_common/tests/sanitizer_new_handler_test.cpp
diff --git a/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt b/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
index bf3e32c3f7786..ce5a1019144aa 100644
--- a/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
+++ b/compiler-rt/lib/sanitizer_common/tests/CMakeLists.txt
@@ -34,6 +34,7 @@ set(SANITIZER_UNITTESTS
sanitizer_mac_test.cpp
sanitizer_module_uuid_size.cpp
sanitizer_mutex_test.cpp
+ sanitizer_new_handler_test.cpp
sanitizer_nolibc_test.cpp
sanitizer_posix_test.cpp
sanitizer_printf_test.cpp
diff --git a/compiler-rt/lib/sanitizer_common/tests/sanitizer_new_handler_test.cpp b/compiler-rt/lib/sanitizer_common/tests/sanitizer_new_handler_test.cpp
new file mode 100644
index 0000000000000..39a4f557856a8
--- /dev/null
+++ b/compiler-rt/lib/sanitizer_common/tests/sanitizer_new_handler_test.cpp
@@ -0,0 +1,202 @@
+//===-- sanitizer_new_handler_test.cpp ------------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Tests for the operator new failure handling in sanitizer_new_handler.h,
+// driven by fake allocation and exhaustion callbacks.
+//
+//===----------------------------------------------------------------------===//
+#include "sanitizer_common/sanitizer_platform.h"
+
+#if !SANITIZER_WINDOWS && defined(__cpp_exceptions)
+
+# include <stdio.h>
+# include <stdlib.h>
+
+# include <new>
+# include <stdexcept>
+
+# include "gtest/gtest.h"
+# include "sanitizer_common/sanitizer_new_handler.h"
+
+namespace __sanitizer {
+namespace {
+
+int handler_calls;
+// Number of calls after which CountingHandler uninstalls itself.
+int handler_budget;
+
+void CountingHandler() {
+ if (++handler_calls >= handler_budget)
+ std::set_new_handler(nullptr);
+}
+
+void ThrowingHandler() { throw std::runtime_error("handler"); }
+
+// Ends the retry loop by throwing std::bad_alloc while staying installed.
+void BadAllocHandler() {
+ ++handler_calls;
+ throw std::bad_alloc();
+}
+
+void ReportExhausted() {
+ fprintf(stderr, "exhausted after %d handler call(s)\n", handler_calls);
+ abort();
+}
+
+// Exhaustion callbacks must not return; throwing records the failure without
+// killing the test binary.
+struct UnexpectedExhaustion {};
+
+void UnexpectedExhausted() {
+ ADD_FAILURE() << "unexpected exhaustion";
+ throw UnexpectedExhaustion();
+}
+
+class NewHandlerTest : public ::testing::Test {
+ protected:
+ // The new_handler and allocator_may_return_null are process-wide; restore
+ // them so other tests in this binary are unaffected.
+ void SetUp() override {
+ saved_handler_ = std::get_new_handler();
+ saved_may_return_null_ = AllocatorMayReturnNull();
+ handler_calls = 0;
+ handler_budget = 1;
+ attempts_ = 0;
+ }
+
+ void TearDown() override {
+ std::set_new_handler(saved_handler_);
+ SetAllocatorMayReturnNull(saved_may_return_null_);
+ }
+
+ // Returns an allocation callback that fails `failures` times, then succeeds.
+ auto Alloc(int failures) {
+ return [this, failures]() -> void* {
+ return ++attempts_ > failures ? &storage_ : nullptr;
+ };
+ }
+
+ // "Always" means the first ~million attempts. Eventually succeeding makes an
+ // infinite retry loop fail with a clear diagnostic instead of timing out.
+ static constexpr int kAlwaysFail = 1 << 20;
+ int attempts_;
+ char storage_;
+
+ private:
+ std::new_handler saved_handler_;
+ bool saved_may_return_null_;
+};
+
+TEST_F(NewHandlerTest, SuccessSkipsHandler) {
+ std::set_new_handler(CountingHandler);
+ EXPECT_EQ(&storage_, RunNewHandlerChain(Alloc(0)));
+ EXPECT_EQ(1, attempts_);
+ EXPECT_EQ(0, handler_calls);
+}
+
+TEST_F(NewHandlerTest, RetriesAfterEachHandlerCall) {
+ handler_budget = 10;
+ std::set_new_handler(CountingHandler);
+ EXPECT_EQ(&storage_, RunNewHandlerChain(Alloc(3)));
+ EXPECT_EQ(4, attempts_);
+ EXPECT_EQ(3, handler_calls);
+}
+
+TEST_F(NewHandlerTest, NoHandlerReturnsNull) {
+ std::set_new_handler(nullptr);
+ EXPECT_EQ(nullptr, RunNewHandlerChain(Alloc(kAlwaysFail)));
+ EXPECT_EQ(1, attempts_);
+}
+
+TEST_F(NewHandlerTest, ExhaustedChainReturnsNull) {
+ handler_budget = 2;
+ std::set_new_handler(CountingHandler);
+ EXPECT_EQ(nullptr, RunNewHandlerChain(Alloc(kAlwaysFail)));
+ EXPECT_EQ(3, attempts_);
+ EXPECT_EQ(2, handler_calls);
+}
+
+TEST_F(NewHandlerTest, ThrowingThrowsBadAllocOnExhaustionWhenMayReturnNull) {
+ SetAllocatorMayReturnNull(true);
+ std::set_new_handler(CountingHandler);
+ EXPECT_THROW((void)NewImplThrowing(Alloc(kAlwaysFail), UnexpectedExhausted),
+ std::bad_alloc);
+ EXPECT_EQ(1, handler_calls);
+}
+
+TEST_F(NewHandlerTest, ThrowingReportsExhaustion) {
+ SetAllocatorMayReturnNull(false);
+ std::set_new_handler(CountingHandler);
+ EXPECT_DEATH((void)NewImplThrowing(Alloc(kAlwaysFail), ReportExhausted),
+ "exhausted after 1 handler call");
+}
+
+TEST_F(NewHandlerTest, ThrowingPropagatesHandlerException) {
+ std::set_new_handler(ThrowingHandler);
+ for (bool may_return_null : {false, true}) {
+ SetAllocatorMayReturnNull(may_return_null);
+ EXPECT_THROW((void)NewImplThrowing(Alloc(kAlwaysFail), UnexpectedExhausted),
+ std::runtime_error);
+ }
+}
+
+TEST_F(NewHandlerTest, ThrowingPropagatesHandlerBadAlloc) {
+ std::set_new_handler(BadAllocHandler);
+ for (bool may_return_null : {false, true}) {
+ SetAllocatorMayReturnNull(may_return_null);
+ handler_calls = attempts_ = 0;
+ EXPECT_THROW((void)NewImplThrowing(Alloc(kAlwaysFail), UnexpectedExhausted),
+ std::bad_alloc);
+ EXPECT_EQ(1, attempts_);
+ EXPECT_EQ(1, handler_calls);
+ EXPECT_EQ(&BadAllocHandler, std::get_new_handler());
+ }
+}
+
+TEST_F(NewHandlerTest, NothrowReturnsNullOnExhaustionWhenMayReturnNull) {
+ SetAllocatorMayReturnNull(true);
+ std::set_new_handler(CountingHandler);
+ EXPECT_EQ(nullptr, NewImplNothrow(Alloc(kAlwaysFail), UnexpectedExhausted));
+ EXPECT_EQ(1, handler_calls);
+}
+
+TEST_F(NewHandlerTest, NothrowReportsExhaustion) {
+ SetAllocatorMayReturnNull(false);
+ std::set_new_handler(CountingHandler);
+ EXPECT_DEATH((void)NewImplNothrow(Alloc(kAlwaysFail), ReportExhausted),
+ "exhausted after 1 handler call");
+}
+
+TEST_F(NewHandlerTest, NothrowSwallowsHandlerException) {
+ std::set_new_handler(ThrowingHandler);
+ for (bool may_return_null : {false, true}) {
+ SetAllocatorMayReturnNull(may_return_null);
+ EXPECT_EQ(nullptr, NewImplNothrow(Alloc(kAlwaysFail), UnexpectedExhausted));
+ }
+}
+
+TEST_F(NewHandlerTest, NothrowReturnsNullOnHandlerBadAlloc) {
+ std::set_new_handler(BadAllocHandler);
+ for (bool may_return_null : {false, true}) {
+ SetAllocatorMayReturnNull(may_return_null);
+ handler_calls = attempts_ = 0;
+ EXPECT_EQ(nullptr, NewImplNothrow(Alloc(kAlwaysFail), UnexpectedExhausted));
+ EXPECT_EQ(1, attempts_);
+ EXPECT_EQ(1, handler_calls);
+ EXPECT_EQ(&BadAllocHandler, std::get_new_handler());
+ }
+}
+
+TEST_F(NewHandlerTest, ReturningExhaustionCallbackDies) {
+ EXPECT_DEATH(InvokeOnExhausted([] {}), "OnExhausted callable returned");
+}
+
+} // namespace
+} // namespace __sanitizer
+
+#endif // !SANITIZER_WINDOWS && defined(__cpp_exceptions)
>From f37dc6c54e4da581ad40f72b3033640a80700371 Mon Sep 17 00:00:00 2001
From: "Justin T. Gibbs" <gibbs at scsiguy.com>
Date: Sat, 30 May 2026 16:21:52 -0700
Subject: [PATCH 3/3] [asan] Adopt sanitizer_common operator-new framework
This completes the series that makes AddressSanitizer's `operator new`
conform to [new.delete.single]/3-4 (google/sanitizers#295), building on
#196413, #200719, #201151, #202816, #213077, #225847 and #226832.
ASan's eight `operator new` overrides now use the shared framework in
`sanitizer_new_operators.inc`. On allocation failure they call the
current `std::new_handler` until allocation succeeds or the chain is
exhausted. After that:
- `allocator_may_return_null=0` (default): report out-of-memory and
abort once the handler chain is exhausted. Programs that install a
`std::new_handler` now see it run first; previously ASan never called
it.
- `allocator_may_return_null=1`: throwing `new` throws `std::bad_alloc`
(aborts if the runtime is built without exceptions, e.g. on Windows);
nothrow `new` returns `nullptr`.
This resolves the long-standing TODO to throw `std::bad_alloc` instead
of dying on OOM.
Other changes:
- The `asan_new*` helpers pass `may_return_null=true`, so storage
failures reach the framework as `nullptr`.
- An oversized `operator new` request now reports `out-of-memory` after
the handler chain, instead of `allocation-size-too-big`.
- The nothrow `new`/`delete` overloads are `noexcept`, matching the
`<new>` declarations the framework header brings in.
- Correct the `allocator_may_return_null` description and add a release
note.
Tests cover all eight overloads. Coverage that applies to any adopting
sanitizer is added as sanitizer_common tests rather than ASan-specific
ones, gated by a new `operator-new-framework` lit feature that this
commit enables for ASan. The existing `allocator_returns_null` and
`max_allocation_size` tests use the feature to expect `std::bad_alloc`
from adopting tools and an abort from the rest. New ASan tests cover the
unaligned nothrow overloads, including on Windows, where
sanitizer_common tests do not run. Handler exceptions and the exhaustion
policy are covered by the framework unit tests in #226832.
Assisted-by: Claude Opus 5.5
---
compiler-rt/lib/asan/asan_allocator.cpp | 18 +++--
compiler-rt/lib/asan/asan_new_delete.cpp | 77 ++++++-------------
.../lib/sanitizer_common/sanitizer_flags.inc | 8 +-
.../TestCases/nothrow_new_default_aborts.cpp | 33 ++++++++
.../TestCases/nothrow_new_returns_null.cpp | 26 +++++++
.../nothrow_new_single_returns_null.cpp | 25 ++++++
.../Linux/allocator_returns_null_std.cpp | 6 +-
.../TestCases/allocator_returns_null.cpp | 33 ++++++--
.../TestCases/max_allocation_size.cpp | 36 +++++++--
.../TestCases/new_handler_invocation.cpp | 43 +++++++++++
...nothrow_new_aligned_array_returns_null.cpp | 29 +++++++
...othrow_new_aligned_single_returns_null.cpp | 28 +++++++
.../TestCases/throw_bad_alloc_aligned.cpp | 38 +++++++++
.../throw_bad_alloc_aligned_single.cpp | 36 +++++++++
.../TestCases/throw_bad_alloc_oversize.cpp | 35 +++++++++
.../test/sanitizer_common/lit.common.cfg.py | 5 ++
llvm/docs/ReleaseNotes.md | 7 ++
17 files changed, 407 insertions(+), 76 deletions(-)
create mode 100644 compiler-rt/test/asan/TestCases/nothrow_new_default_aborts.cpp
create mode 100644 compiler-rt/test/asan/TestCases/nothrow_new_returns_null.cpp
create mode 100644 compiler-rt/test/asan/TestCases/nothrow_new_single_returns_null.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/new_handler_invocation.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_array_returns_null.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_single_returns_null.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned_single.cpp
create mode 100644 compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_oversize.cpp
diff --git a/compiler-rt/lib/asan/asan_allocator.cpp b/compiler-rt/lib/asan/asan_allocator.cpp
index b75a8ea09e34a..fd3275df30105 100644
--- a/compiler-rt/lib/asan/asan_allocator.cpp
+++ b/compiler-rt/lib/asan/asan_allocator.cpp
@@ -1207,18 +1207,24 @@ uptr asan_malloc_usable_size(const void* ptr, uptr pc, uptr bp) {
namespace {
void* asan_new(uptr size, BufferedStackTrace* stack, bool array) {
- return SetErrnoOnNull(instance.Allocate(size, /*alignment=*/0, stack,
- array ? FROM_NEW_BR : FROM_NEW,
- /*can_fill=*/true));
+ // Return nullptr, not abort, when storage cannot be obtained so the operator
+ // new framework can run the std::get_new_handler() loop and then apply the
+ // allocator_may_return_null flag.
+ return SetErrnoOnNull(instance.AllocateImpl(
+ size, /*alignment=*/0, stack, array ? FROM_NEW_BR : FROM_NEW,
+ /*can_fill=*/true, /*may_return_null=*/true));
}
void* asan_new_aligned(uptr size, uptr alignment, BufferedStackTrace* stack,
bool array) {
if (UNLIKELY(!CheckAlignedNewAlignment(alignment)))
ReportInvalidAllocationAlignment(alignment, stack);
- return SetErrnoOnNull(instance.Allocate(size, alignment, stack,
- array ? FROM_NEW_BR : FROM_NEW,
- /*can_fill=*/true));
+ // Return nullptr, not abort, when storage cannot be obtained so the operator
+ // new framework can run the std::get_new_handler() loop and then apply the
+ // allocator_may_return_null flag.
+ return SetErrnoOnNull(instance.AllocateImpl(
+ size, alignment, stack, array ? FROM_NEW_BR : FROM_NEW,
+ /*can_fill=*/true, /*may_return_null=*/true));
}
void asan_delete(void* ptr, BufferedStackTrace* stack, bool array) {
diff --git a/compiler-rt/lib/asan/asan_new_delete.cpp b/compiler-rt/lib/asan/asan_new_delete.cpp
index 04f8082de3148..a17838b56d58d 100644
--- a/compiler-rt/lib/asan/asan_new_delete.cpp
+++ b/compiler-rt/lib/asan/asan_new_delete.cpp
@@ -18,6 +18,7 @@
#include "asan_report.h"
#include "asan_stack.h"
#include "interception/interception.h"
+#include "sanitizer_common/sanitizer_new_handler.h"
// C++ operators can't have dllexport attributes on Windows. We export them
// anyway by passing extra -export flags to the linker, which is exactly that
@@ -51,51 +52,17 @@ using namespace __asan;
// This code has issues on OSX.
// See https://github.com/google/sanitizers/issues/131.
-// Fake std::nothrow_t and std::align_val_t to avoid including <new>.
-namespace std {
-struct nothrow_t {};
-enum class align_val_t : size_t {};
-} // namespace std
-
-// TODO(alekseyshl): throw std::bad_alloc instead of dying on OOM.
-// For local pool allocation, align to SHADOW_GRANULARITY to match asan
-// allocator behavior.
-#define OPERATOR_NEW_BODY \
- GET_STACK_TRACE_MALLOC; \
- void* res = asan_new(size, &stack); \
- if (UNLIKELY(!res)) \
- ReportOutOfMemory(size, &stack); \
- return res
-#define OPERATOR_NEW_BODY_NOTHROW \
- GET_STACK_TRACE_MALLOC; \
- return asan_new(size, &stack)
-#define OPERATOR_NEW_BODY_ARRAY \
- GET_STACK_TRACE_MALLOC; \
- void* res = asan_new_array(size, &stack); \
- if (UNLIKELY(!res)) \
- ReportOutOfMemory(size, &stack); \
- return res
-#define OPERATOR_NEW_BODY_ARRAY_NOTHROW \
- GET_STACK_TRACE_MALLOC; \
- return asan_new_array(size, &stack)
-#define OPERATOR_NEW_BODY_ALIGN \
- GET_STACK_TRACE_MALLOC; \
- void* res = asan_new_aligned(size, static_cast<uptr>(align), &stack); \
- if (UNLIKELY(!res)) \
- ReportOutOfMemory(size, &stack); \
- return res
-#define OPERATOR_NEW_BODY_ALIGN_NOTHROW \
- GET_STACK_TRACE_MALLOC; \
- return asan_new_aligned(size, static_cast<uptr>(align), &stack)
-#define OPERATOR_NEW_BODY_ALIGN_ARRAY \
- GET_STACK_TRACE_MALLOC; \
- void* res = asan_new_array_aligned(size, static_cast<uptr>(align), &stack); \
- if (UNLIKELY(!res)) \
- ReportOutOfMemory(size, &stack); \
- return res
-#define OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW \
- GET_STACK_TRACE_MALLOC; \
- return asan_new_array_aligned(size, static_cast<uptr>(align), &stack)
+// Plug ASan's stack tracing, OOM reporting, and allocation helpers into the
+// shared operator new framework.
+#define SANITIZER_NEW_STACK_TRACE GET_STACK_TRACE_MALLOC
+#define SANITIZER_NEW_REPORT_OOM(size) ReportOutOfMemory(size, &stack)
+#define SANITIZER_NEW(size) asan_new(size, &stack)
+#define SANITIZER_NEW_ARRAY(size) asan_new_array(size, &stack)
+#define SANITIZER_NEW_ALIGNED(size, align) \
+ asan_new_aligned(size, static_cast<uptr>(align), &stack)
+#define SANITIZER_NEW_ARRAY_ALIGNED(size, align) \
+ asan_new_array_aligned(size, static_cast<uptr>(align), &stack)
+#include "sanitizer_common/sanitizer_new_operators.inc"
// On OS X it's not enough to just provide our own 'operator new' and
// 'operator delete' implementations, because they're going to be in the
@@ -110,11 +77,11 @@ void* operator new(size_t size) { OPERATOR_NEW_BODY; }
CXX_OPERATOR_ATTRIBUTE
void* operator new[](size_t size) { OPERATOR_NEW_BODY_ARRAY; }
CXX_OPERATOR_ATTRIBUTE
-void* operator new(size_t size, std::nothrow_t const&) {
+void* operator new(size_t size, std::nothrow_t const&) NOEXCEPT {
OPERATOR_NEW_BODY_NOTHROW;
}
CXX_OPERATOR_ATTRIBUTE
-void* operator new[](size_t size, std::nothrow_t const&) {
+void* operator new[](size_t size, std::nothrow_t const&) NOEXCEPT {
OPERATOR_NEW_BODY_ARRAY_NOTHROW;
}
CXX_OPERATOR_ATTRIBUTE
@@ -126,12 +93,13 @@ void* operator new[](size_t size, std::align_val_t align) {
OPERATOR_NEW_BODY_ALIGN_ARRAY;
}
CXX_OPERATOR_ATTRIBUTE
-void* operator new(size_t size, std::align_val_t align, std::nothrow_t const&) {
+void* operator new(size_t size, std::align_val_t align,
+ std::nothrow_t const&) NOEXCEPT {
OPERATOR_NEW_BODY_ALIGN_NOTHROW;
}
CXX_OPERATOR_ATTRIBUTE
void* operator new[](size_t size, std::align_val_t align,
- std::nothrow_t const&) {
+ std::nothrow_t const&) NOEXCEPT {
OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW;
}
@@ -177,9 +145,11 @@ void operator delete(void* ptr) NOEXCEPT { OPERATOR_DELETE_BODY; }
CXX_OPERATOR_ATTRIBUTE
void operator delete[](void* ptr) NOEXCEPT { OPERATOR_DELETE_BODY_ARRAY; }
CXX_OPERATOR_ATTRIBUTE
-void operator delete(void* ptr, std::nothrow_t const&) { OPERATOR_DELETE_BODY; }
+void operator delete(void* ptr, std::nothrow_t const&) NOEXCEPT {
+ OPERATOR_DELETE_BODY;
+}
CXX_OPERATOR_ATTRIBUTE
-void operator delete[](void* ptr, std::nothrow_t const&) {
+void operator delete[](void* ptr, std::nothrow_t const&) NOEXCEPT {
OPERATOR_DELETE_BODY_ARRAY;
}
CXX_OPERATOR_ATTRIBUTE
@@ -199,12 +169,13 @@ void operator delete[](void* ptr, std::align_val_t align) NOEXCEPT {
OPERATOR_DELETE_BODY_ALIGN_ARRAY;
}
CXX_OPERATOR_ATTRIBUTE
-void operator delete(void* ptr, std::align_val_t align, std::nothrow_t const&) {
+void operator delete(void* ptr, std::align_val_t align,
+ std::nothrow_t const&) NOEXCEPT {
OPERATOR_DELETE_BODY_ALIGN;
}
CXX_OPERATOR_ATTRIBUTE
void operator delete[](void* ptr, std::align_val_t align,
- std::nothrow_t const&) {
+ std::nothrow_t const&) NOEXCEPT {
OPERATOR_DELETE_BODY_ALIGN_ARRAY;
}
CXX_OPERATOR_ATTRIBUTE
diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc b/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
index 5f449907f6011..0a8c21b68ee18 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_flags.inc
@@ -83,8 +83,12 @@ COMMON_FLAG(
"detect_leaks=false, or if __lsan_do_leak_check() is called before the "
"handler has a chance to run.")
COMMON_FLAG(bool, allocator_may_return_null, false,
- "If false, the allocator will crash instead of returning 0 on "
- "out-of-memory.")
+ "If false (default), the allocator aborts on out-of-memory. If "
+ "true, allocation functions that can report failure return null "
+ "instead (for example, malloc and nothrow operator new). Throwing "
+ "operator new throws std::bad_alloc where the runtime supports "
+ "it (for example, AddressSanitizer built with exceptions) and "
+ "aborts otherwise.")
COMMON_FLAG(bool, print_summary, true,
"If false, disable printing error summaries in addition to error "
"reports.")
diff --git a/compiler-rt/test/asan/TestCases/nothrow_new_default_aborts.cpp b/compiler-rt/test/asan/TestCases/nothrow_new_default_aborts.cpp
new file mode 100644
index 0000000000000..f35b8c7bea92e
--- /dev/null
+++ b/compiler-rt/test/asan/TestCases/nothrow_new_default_aborts.cpp
@@ -0,0 +1,33 @@
+// With allocator_may_return_null=false (default), nothrow operator new
+// aborts on OOM. The handler chain runs (per [new.delete.single]/4); on
+// chain exhaustion the runtime emits the asan diagnostic and Die()s rather
+// than returning nullptr.
+
+// RUN: %clangxx_asan -O0 %s -o %t
+// RUN: not %run %t 2>&1 | FileCheck %s
+
+// REQUIRES: stable-runtime
+
+#include <cstdio>
+#include <new>
+
+static const size_t kHugeSize =
+#if __LP64__ || defined(_WIN64)
+ (1ULL << 40) + 1;
+#else
+ (3UL << 30) + 1;
+#endif
+
+int main() {
+ // No new_handler installed -> chain exhausts immediately -> default flag
+ // selects the abort path.
+ char *p = new (std::nothrow) char[kHugeSize];
+ fprintf(stderr, "FAIL: allocation unexpectedly returned %p\n", p);
+ return 0;
+}
+
+// Linux's secondary mmap fails first (out of memory) and Windows's
+// kMaxAllowedMallocSize check trips first (requested allocation size); both
+// prove the default-flag abort path was taken.
+// CHECK: AddressSanitizer: {{out of memory|requested allocation size}}
+// CHECK: ABORTING
diff --git a/compiler-rt/test/asan/TestCases/nothrow_new_returns_null.cpp b/compiler-rt/test/asan/TestCases/nothrow_new_returns_null.cpp
new file mode 100644
index 0000000000000..19e73ca3c58d7
--- /dev/null
+++ b/compiler-rt/test/asan/TestCases/nothrow_new_returns_null.cpp
@@ -0,0 +1,26 @@
+// Per [new.delete.single]/4, nothrow operator new must return nullptr on
+// allocation failure after running the new_handler chain. Opt-in via
+// allocator_may_return_null=1; the default-flag abort case is covered by
+// nothrow_new_default_aborts.cpp.
+
+// RUN: %clangxx_asan -O0 %s -o %t
+// RUN: %env_asan_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// REQUIRES: stable-runtime
+
+#include <cstdio>
+#include <new>
+
+static const size_t kHugeSize =
+#if __LP64__ || defined(_WIN64)
+ (1ULL << 40) + 1;
+#else
+ (3UL << 30) + 1;
+#endif
+
+int main() {
+ char *p = new (std::nothrow) char[kHugeSize];
+ fprintf(stderr, "nothrow returned %s\n", p ? "non-null" : "null");
+ // CHECK: nothrow returned null
+ return 0;
+}
diff --git a/compiler-rt/test/asan/TestCases/nothrow_new_single_returns_null.cpp b/compiler-rt/test/asan/TestCases/nothrow_new_single_returns_null.cpp
new file mode 100644
index 0000000000000..23c264c618437
--- /dev/null
+++ b/compiler-rt/test/asan/TestCases/nothrow_new_single_returns_null.cpp
@@ -0,0 +1,25 @@
+// Single-object nothrow operator new must return nullptr on allocation
+// failure (OPERATOR_NEW_BODY_NOTHROW). Opt-in via allocator_may_return_null=1.
+
+// RUN: %clangxx_asan -O0 %s -o %t
+// RUN: %env_asan_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// REQUIRES: stable-runtime
+
+#include <cstdio>
+#include <new>
+
+struct alignas(1) Huge {
+#if __LP64__ || defined(_WIN64)
+ char data[(1ULL << 40) + 1];
+#else
+ char data[(3UL << 30) + 1];
+#endif
+};
+
+int main() {
+ Huge *p = new (std::nothrow) Huge;
+ fprintf(stderr, "nothrow returned %s\n", p ? "non-null" : "null");
+ // CHECK: nothrow returned null
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/Linux/allocator_returns_null_std.cpp b/compiler-rt/test/sanitizer_common/TestCases/Linux/allocator_returns_null_std.cpp
index 812cf049f2a9b..8bdc244443a56 100644
--- a/compiler-rt/test/sanitizer_common/TestCases/Linux/allocator_returns_null_std.cpp
+++ b/compiler-rt/test/sanitizer_common/TestCases/Linux/allocator_returns_null_std.cpp
@@ -27,4 +27,8 @@ int main(int argc, char **argv) {
}
// CHECK: #{{[0-9]+.*}}allocator_returns_null_std.cpp
-// CHECK: {{SUMMARY: .*Sanitizer: allocation-size-too-big.*allocator_returns_null_std.cpp.*}} in main
+// std::vector::resize uses throwing operator new[]. Tools using the operator
+// new framework return nullptr from their allocator so std::get_new_handler()
+// runs first; the chain-exhausted abort then emits "out-of-memory" rather than
+// the in-place "allocation-size-too-big" emitted by other tools.
+// CHECK: {{SUMMARY: .*Sanitizer: (allocation-size-too-big|out-of-memory).*allocator_returns_null_std.cpp.*}} in main
diff --git a/compiler-rt/test/sanitizer_common/TestCases/allocator_returns_null.cpp b/compiler-rt/test/sanitizer_common/TestCases/allocator_returns_null.cpp
index 6343d87bc5c5d..01480882abd9f 100644
--- a/compiler-rt/test/sanitizer_common/TestCases/allocator_returns_null.cpp
+++ b/compiler-rt/test/sanitizer_common/TestCases/allocator_returns_null.cpp
@@ -28,14 +28,22 @@
// RUN: | FileCheck %s --check-prefix=CHECK-NULL
// RUN: %env_tool_opts=allocator_may_return_null=0 not %run %t new 2>&1 \
// RUN: | FileCheck %s --check-prefix=CHECK-nCRASH
-// RUN: %env_tool_opts=allocator_may_return_null=1 not %run %t new 2>&1 \
-// RUN: | FileCheck %s --check-prefix=CHECK-nCRASH-OOM
+// flag=1 + throwing new: tools with the operator-new-framework feature throw
+// bad_alloc, which the test catches and converts to CHECK-NULL; other tools
+// abort inside operator new.
+// RUN: %if operator-new-framework %{ %env_tool_opts=allocator_may_return_null=1 %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-NULL %}
+// RUN: %if !operator-new-framework %{ %env_tool_opts=allocator_may_return_null=1 not %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-nCRASH-OOM %}
// RUN: %env_tool_opts=allocator_may_return_null=0 not %run %t new-nothrow 2>&1 \
// RUN: | FileCheck %s --check-prefix=CHECK-nnCRASH
// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t new-nothrow 2>&1 \
// RUN: | FileCheck %s --check-prefix=CHECK-NULL
// TODO(alekseyshl): win32 is disabled due to failing errno tests, fix it there.
+// Windows asan would also fail the flag=1 + new cell above: its runtime is
+// built without exceptions and never throws bad_alloc, so the throwing form
+// always falls back to ReportOutOfMemory + Die(). Re-enabling this test on
+// Windows requires excluding Windows from the %if operator-new-framework
+// dispatch.
// UNSUPPORTED: ubsan, target={{.*windows-msvc.*}}
// UNSUPPORTED: rtsan
@@ -81,7 +89,14 @@ int main(int argc, char **argv) {
assert(*t == 42);
free(t);
} else if (!strcmp(action, "new")) {
- x = operator new(kMaxAllowedMallocSizePlusOne);
+ try {
+ x = operator new(kMaxAllowedMallocSizePlusOne);
+ assert(0 && "throwing operator new returned without throwing -- "
+ "violates [basic.stc.dynamic.allocation]/3");
+ } catch (const std::bad_alloc &) {
+ x = nullptr;
+ errno = ENOMEM;
+ }
} else if (!strcmp(action, "new-nothrow")) {
x = operator new(kMaxAllowedMallocSizePlusOne, std::nothrow);
} else {
@@ -112,13 +127,17 @@ int main(int argc, char **argv) {
// CHECK-mrCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.*allocator_returns_null.cpp.*}} in main
// CHECK-nCRASH: new:
// CHECK-nCRASH: #{{[0-9]+.*}}allocator_returns_null.cpp
-// CHECK-nCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.*allocator_returns_null.cpp.*}} in main
+// Tools using the operator new framework return nullptr from their allocator
+// so std::get_new_handler() runs first; the chain-exhausted abort then emits
+// "out-of-memory" rather than the in-place "allocation-size-too-big" emitted
+// by other tools. Same alternation applies to CHECK-nnCRASH.
+// CHECK-nCRASH: {{SUMMARY: .*Sanitizer: (allocation-size-too-big|out-of-memory).*allocator_returns_null.cpp.*}} in main
// CHECK-nCRASH-OOM: new:
-// CHECK-nCRASH-O#{{[0-9]+.*}}allocator_returns_null.cpp
+// CHECK-nCRASH-OOM: #{{[0-9]+.*}}allocator_returns_null.cpp
// CHECK-nCRASH-OOM: {{SUMMARY: .*Sanitizer: out-of-memory.*allocator_returns_null.cpp.*}} in main
// CHECK-nnCRASH: new-nothrow:
// CHECK-nnCRASH: #{{[0-9]+.*}}allocator_returns_null.cpp
-// CHECK-nnCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.*allocator_returns_null.cpp.*}} in main
+// CHECK-nnCRASH: {{SUMMARY: .*Sanitizer: (allocation-size-too-big|out-of-memory).*allocator_returns_null.cpp.*}} in main
-// CHECK-NULL: {{malloc|calloc|calloc-overflow|realloc|realloc-after-malloc|new-nothrow}}
+// CHECK-NULL: {{malloc|calloc|calloc-overflow|realloc|realloc-after-malloc|new-nothrow|new}}
// CHECK-NULL: errno: 12, x: 0
diff --git a/compiler-rt/test/sanitizer_common/TestCases/max_allocation_size.cpp b/compiler-rt/test/sanitizer_common/TestCases/max_allocation_size.cpp
index 864d05f87e4c6..77ea50841dcc6 100644
--- a/compiler-rt/test/sanitizer_common/TestCases/max_allocation_size.cpp
+++ b/compiler-rt/test/sanitizer_common/TestCases/max_allocation_size.cpp
@@ -28,8 +28,11 @@
// RUN: | FileCheck %s --check-prefix=CHECK-NULL
// RUN: %env_tool_opts=max_allocation_size_mb=2:allocator_may_return_null=0 \
// RUN: not %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-nCRASH
-// RUN: %env_tool_opts=max_allocation_size_mb=2:allocator_may_return_null=1 \
-// RUN: not %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-nCRASH-OOM
+// flag=1 + throwing new: tools with the operator-new-framework feature throw
+// bad_alloc, which allocate() catches and converts to CHECK-NULL; other tools
+// abort inside operator new.
+// RUN: %if operator-new-framework %{ %env_tool_opts=max_allocation_size_mb=2:allocator_may_return_null=1 %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-NULL %}
+// RUN: %if !operator-new-framework %{ %env_tool_opts=max_allocation_size_mb=2:allocator_may_return_null=1 not %run %t new 2>&1 | FileCheck %s --check-prefix=CHECK-nCRASH-OOM %}
// RUN: %env_tool_opts=max_allocation_size_mb=2:allocator_may_return_null=0 \
// RUN: not %run %t new-nothrow 2>&1 \
// RUN: | FileCheck %s --check-prefix=CHECK-nnCRASH
@@ -41,6 +44,11 @@
// RUN: %run %t strndup 2>&1 | FileCheck %s --check-prefix=CHECK-NULL
// win32 is disabled due to failing errno tests.
+// Windows asan would also fail the flag=1 + new cell above: its runtime is
+// built without exceptions and never throws bad_alloc, so the throwing form
+// always falls back to ReportOutOfMemory + Die(). Re-enabling this test on
+// Windows requires excluding Windows from the %if operator-new-framework
+// dispatch.
// UNSUPPORTED: ubsan, target={{.*windows-msvc.*}}
// Symbolizer needs to allocated memory when reporting.
@@ -72,8 +80,18 @@ static void *allocate(const char *Action, size_t Size) {
free(P);
return nullptr;
}
- if (!strcmp(Action, "new"))
- return ::operator new(Size);
+ if (!strcmp(Action, "new")) {
+ try {
+ void *p = ::operator new(Size);
+ assert(p != nullptr &&
+ "throwing operator new returned nullptr without throwing -- "
+ "violates [basic.stc.dynamic.allocation]/3");
+ return p;
+ } catch (const std::bad_alloc &) {
+ errno = ENOMEM;
+ return nullptr;
+ }
+ }
if (!strcmp(Action, "new-nothrow"))
return ::operator new(Size, std::nothrow);
if (!strcmp(Action, "strndup")) {
@@ -138,18 +156,22 @@ int main(int Argc, char **Argv) {
// CHECK-mrCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.* in allocate}}
// CHECK-nCRASH: new:
// CHECK-nCRASH: #{{[0-9]+.*}}max_allocation_size.cpp
-// CHECK-nCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.* in allocate}}
+// Tools using the operator new framework return nullptr from their allocator
+// so std::get_new_handler() runs first; the chain-exhausted abort then emits
+// "out-of-memory" rather than the in-place "allocation-size-too-big" emitted
+// by other tools. Same alternation applies to CHECK-nnCRASH.
+// CHECK-nCRASH: {{SUMMARY: .*Sanitizer: (allocation-size-too-big|out-of-memory).* in allocate}}
// CHECK-nCRASH-OOM: new:
// CHECK-nCRASH-OOM: #{{[0-9]+.*}}max_allocation_size.cpp
// CHECK-nCRASH-OOM: {{SUMMARY: .*Sanitizer: out-of-memory.* in allocate}}
// CHECK-nnCRASH: new-nothrow:
// CHECK-nnCRASH: #{{[0-9]+.*}}max_allocation_size.cpp
-// CHECK-nnCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.* in allocate}}
+// CHECK-nnCRASH: {{SUMMARY: .*Sanitizer: (allocation-size-too-big|out-of-memory).* in allocate}}
// CHECK-sCRASH: strndup:
// CHECK-sCRASH: #{{[0-9]+.*}}max_allocation_size.cpp
// CHECK-sCRASH: {{SUMMARY: .*Sanitizer: allocation-size-too-big.*}}
-// CHECK-NULL: {{malloc|calloc|calloc-overflow|realloc|realloc-after-malloc|new-nothrow|strndup}}
+// CHECK-NULL: {{malloc|calloc|calloc-overflow|realloc|realloc-after-malloc|new-nothrow|new|strndup}}
// CHECK-NULL: errno: 12, P: 0
//
// CHECK-NOTNULL-NOT: P: 0
diff --git a/compiler-rt/test/sanitizer_common/TestCases/new_handler_invocation.cpp b/compiler-rt/test/sanitizer_common/TestCases/new_handler_invocation.cpp
new file mode 100644
index 0000000000000..27d923d62e84c
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/new_handler_invocation.cpp
@@ -0,0 +1,43 @@
+// Throwing operator new must invoke std::new_handler before throwing
+// std::bad_alloc, per [new.delete.single]/3 (and /4 for the nothrow form).
+
+// RUN: %clangxx -O0 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=0 %run %t 2>&1 | FileCheck %s
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// UNSUPPORTED: target={{.*windows.*}}
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+static const size_t kHugeSize =
+#if __LP64__ || defined(_WIN64)
+ (1ULL << 40) + 1;
+#else
+ (3UL << 30) + 1;
+#endif
+
+static int handler_calls = 0;
+
+static void my_handler() {
+ ++handler_calls;
+ fprintf(stderr, "handler call %d\n", handler_calls);
+ // Break the loop. A real handler would free memory and return; this
+ // allocation is unrecoverable so we throw to terminate.
+ throw std::bad_alloc();
+}
+
+int main() {
+ std::set_new_handler(my_handler);
+ try {
+ char *p = new char[kHugeSize];
+ fprintf(stderr, "FAIL: allocation unexpectedly returned %p\n", p);
+ } catch (const std::bad_alloc &) {
+ fprintf(stderr, "caught bad_alloc after %d handler call(s)\n",
+ handler_calls);
+ }
+ // CHECK: handler call 1
+ // CHECK: caught bad_alloc after 1 handler call(s)
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_array_returns_null.cpp b/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_array_returns_null.cpp
new file mode 100644
index 0000000000000..76417d9d7615d
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_array_returns_null.cpp
@@ -0,0 +1,29 @@
+// Aligned array nothrow operator new must return nullptr on allocation
+// failure (OPERATOR_NEW_BODY_ALIGN_ARRAY_NOTHROW). Opt-in via
+// allocator_may_return_null=1.
+
+// RUN: %clangxx -O0 -std=c++17 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// ASan does not override aligned operator new on Darwin or Windows (MSVC).
+// UNSUPPORTED: asan && (darwin || target={{.*windows-msvc.*}})
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+struct alignas(64) HugeAligned {
+#if __LP64__ || defined(_WIN64)
+ char data[(1ULL << 40) + 1];
+#else
+ char data[(3UL << 30) + 1];
+#endif
+};
+
+int main() {
+ HugeAligned *p = new (std::nothrow) HugeAligned[1];
+ fprintf(stderr, "nothrow aligned array returned %s\n",
+ p ? "non-null" : "null");
+ // CHECK: nothrow aligned array returned null
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_single_returns_null.cpp b/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_single_returns_null.cpp
new file mode 100644
index 0000000000000..bbdf6900775c4
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/nothrow_new_aligned_single_returns_null.cpp
@@ -0,0 +1,28 @@
+// Aligned single-object nothrow operator new must return nullptr on
+// allocation failure (OPERATOR_NEW_BODY_ALIGN_NOTHROW). Opt-in via
+// allocator_may_return_null=1.
+
+// RUN: %clangxx -O0 -std=c++17 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// ASan does not override aligned operator new on Darwin or Windows (MSVC).
+// UNSUPPORTED: asan && (darwin || target={{.*windows-msvc.*}})
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+struct alignas(64) HugeAligned {
+#if __LP64__ || defined(_WIN64)
+ char data[(1ULL << 40) + 1];
+#else
+ char data[(3UL << 30) + 1];
+#endif
+};
+
+int main() {
+ HugeAligned *p = new (std::nothrow) HugeAligned;
+ fprintf(stderr, "nothrow aligned returned %s\n", p ? "non-null" : "null");
+ // CHECK: nothrow aligned returned null
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned.cpp b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned.cpp
new file mode 100644
index 0000000000000..196f99c153c9b
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned.cpp
@@ -0,0 +1,38 @@
+// Throwing aligned operator new must throw std::bad_alloc on allocation
+// failure, just like the unaligned form. Opt-in via allocator_may_return_null=1.
+
+// RUN: %clangxx -O0 -std=c++17 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// UNSUPPORTED: target={{.*windows.*}}
+// ASan does not override aligned operator new on Darwin.
+// UNSUPPORTED: asan && darwin
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+static const size_t kHugeSize =
+#if __LP64__ || defined(_WIN64)
+ (1ULL << 40) + 1;
+#else
+ (3UL << 30) + 1;
+#endif
+
+struct alignas(64) Aligned {
+ char data[1];
+};
+
+int main() {
+ bool caught = false;
+ try {
+ Aligned *p = new Aligned[kHugeSize];
+ fprintf(stderr, "FAIL: allocation unexpectedly returned %p\n", p);
+ } catch (const std::bad_alloc &) {
+ caught = true;
+ }
+ if (caught)
+ fprintf(stderr, "caught bad_alloc\n");
+ // CHECK: caught bad_alloc
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned_single.cpp b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned_single.cpp
new file mode 100644
index 0000000000000..76d45316abc8f
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_aligned_single.cpp
@@ -0,0 +1,36 @@
+// Aligned single-object throwing operator new must throw std::bad_alloc on
+// allocation failure (OPERATOR_NEW_BODY_ALIGN). Opt-in via
+// allocator_may_return_null=1.
+
+// RUN: %clangxx -O0 -std=c++17 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// UNSUPPORTED: target={{.*windows.*}}
+// ASan does not override aligned operator new on Darwin.
+// UNSUPPORTED: asan && darwin
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+struct alignas(64) HugeAligned {
+#if __LP64__ || defined(_WIN64)
+ char data[(1ULL << 40) + 1];
+#else
+ char data[(3UL << 30) + 1];
+#endif
+};
+
+int main() {
+ bool caught = false;
+ try {
+ HugeAligned *p = new HugeAligned;
+ fprintf(stderr, "FAIL: allocation unexpectedly returned %p\n", p);
+ } catch (const std::bad_alloc &) {
+ caught = true;
+ }
+ if (caught)
+ fprintf(stderr, "caught bad_alloc\n");
+ // CHECK: caught bad_alloc
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_oversize.cpp b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_oversize.cpp
new file mode 100644
index 0000000000000..dc863e52a8bdd
--- /dev/null
+++ b/compiler-rt/test/sanitizer_common/TestCases/throw_bad_alloc_oversize.cpp
@@ -0,0 +1,35 @@
+// Throwing operator new must throw std::bad_alloc on allocation failure
+// (here triggered by an oversize request) rather than aborting. Opt-in via
+// allocator_may_return_null=1.
+
+// RUN: %clangxx -O0 %s -o %t
+// RUN: %env_tool_opts=allocator_may_return_null=1 %run %t 2>&1 | FileCheck %s
+
+// Windows ASan can't throw bad_alloc; see
+// sanitizer_common/sanitizer_new_handler.h.
+// UNSUPPORTED: target={{.*windows.*}}
+// REQUIRES: operator-new-framework, stable-runtime
+
+#include <cstdio>
+#include <new>
+
+static const size_t kHugeSize =
+#if __LP64__ || defined(_WIN64)
+ (1ULL << 40) + 1;
+#else
+ (3UL << 30) + 1;
+#endif
+
+int main() {
+ bool caught = false;
+ try {
+ char *p = new char[kHugeSize];
+ fprintf(stderr, "FAIL: allocation unexpectedly returned %p\n", p);
+ } catch (const std::bad_alloc &) {
+ caught = true;
+ }
+ if (caught)
+ fprintf(stderr, "caught bad_alloc\n");
+ // CHECK: caught bad_alloc
+ return 0;
+}
diff --git a/compiler-rt/test/sanitizer_common/lit.common.cfg.py b/compiler-rt/test/sanitizer_common/lit.common.cfg.py
index 5614229d9a126..3e3ec7546f375 100644
--- a/compiler-rt/test/sanitizer_common/lit.common.cfg.py
+++ b/compiler-rt/test/sanitizer_common/lit.common.cfg.py
@@ -39,6 +39,11 @@
config.available_features.add(config.tool_name)
+# Tools whose operator new overrides use the shared framework in
+# sanitizer_common/sanitizer_new_operators.inc.
+if config.tool_name in ["asan"]:
+ config.available_features.add("operator-new-framework")
+
if (
config.target_os == "Linux"
and config.tool_name == "lsan"
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 41b2fa83d380f..d2b652d504412 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -366,6 +366,13 @@ Makes programs 10x faster by doing Special New Thing.
### Changes to Sanitizers
+* AddressSanitizer's `operator new` now calls the current
+ `std::new_handler` on allocation failure, as the C++ standard
+ requires. With `allocator_may_return_null=1`, throwing `operator new`
+ throws `std::bad_alloc` instead of aborting (on runtimes built with
+ exceptions), and nothrow `operator new` returns `nullptr`. By default
+ it still aborts once the handler chain is exhausted.
+
### Other Changes
* `cas::ObjectStore::getMemoryBuffer()` was documented as returning a buffer
More information about the llvm-commits
mailing list