[compiler-rt] [sanitizer_common] Return nullptr on ASan allocator commitment error (PR #119753)

[Mike Hommey via llvm-commits]

https://github.com/glandium created https://github.com/llvm/llvm-project/pull/119753

None

>From 287625f921cf8fd22b9c757d9df0ab63e5fee7f5 Mon Sep 17 00:00:00 2001
From: Mike Hommey <mh at glandium.org>
Date: Fri, 13 Dec 2024 05:03:45 +0900
Subject: [PATCH] [sanitizer_common] Return nullptr on ASan allocator
 commitment error

---
 compiler-rt/lib/sanitizer_common/sanitizer_win.cpp | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp b/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
index 1eef16fbde3e7d..fd0f989ee392bb 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_win.cpp
@@ -167,7 +167,7 @@ static void *ReturnNullptrOnOOMOrDie(uptr size, const char *mem_type,
 
   // Assumption: VirtualAlloc is the last system call that was invoked before
   //   this method.
-  // VirtualAlloc emits one of 2 error codes when running out of memory
+  // VirtualAlloc emits one of 3 error codes when running out of memory
   // 1. ERROR_NOT_ENOUGH_MEMORY:
   //  There's not enough memory to execute the command
   // 2. ERROR_INVALID_PARAMETER:
@@ -176,12 +176,12 @@ static void *ReturnNullptrOnOOMOrDie(uptr size, const char *mem_type,
   //  (the `lpMaximumApplicationAddress` field within the `SystemInfo` struct).
   //  This does not seem to be officially documented, but is corroborated here:
   //  https://stackoverflow.com/questions/45833674/why-does-virtualalloc-fail-for-lpaddress-greater-than-0x6ffffffffff
-
-  // Note - It's possible that 'ERROR_COMMITMENT_LIMIT' needs to be handled here
-  // as well. It is currently not handled due to the lack of a reproducer that
-  // induces the error code.
+  // 3. ERROR_COMMITMENT_LIMIT:
+  //  VirtualAlloc will return this if e.g. the pagefile is too small to commit
+  //  the requested amount of memory.
   if (last_error == ERROR_NOT_ENOUGH_MEMORY ||
-      last_error == ERROR_INVALID_PARAMETER)
+      last_error == ERROR_INVALID_PARAMETER ||
+      last_error == ERROR_COMMITMENT_LIMIT)
     return nullptr;
   ReportMmapFailureAndDie(size, mem_type, mmap_type, last_error);
 }