[PATCH] D121399: Fixed a roll-over on size_t in getNewUninitMemBuffer()

[Dmitry Vassiliev via Phabricator via llvm-commits]

slydiman created this revision.
slydiman added reviewers: kazu, abhina.sreeskantharajan, serge-sans-paille, MaskRay.
slydiman added a project: LLVM.
Herald added subscribers: dexonsmith, hiraditya.
Herald added a project: All.
slydiman requested review of this revision.
Herald added a subscriber: llvm-commits.

This patch fixes a roll-over on size_t in WritableMemoryBuffer::getNewUninitMemBuffer().


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D121399

Files:
  llvm/lib/Support/MemoryBuffer.cpp
  llvm/unittests/Support/MemoryBufferTest.cpp


Index: llvm/unittests/Support/MemoryBufferTest.cpp
===================================================================
--- llvm/unittests/Support/MemoryBufferTest.cpp
+++ llvm/unittests/Support/MemoryBufferTest.cpp
@@ -219,6 +219,11 @@
   EXPECT_NE(nullptr, Four.get());
   for (size_t i = 0; i < 123; ++i)
     EXPECT_EQ(0, Four->getBufferStart()[0]);
+
+  // uninitialized buffer with rollover size
+  OwningBuffer Five(
+      WritableMemoryBuffer::getNewUninitMemBuffer(SIZE_MAX, "huge"));
+  EXPECT_EQ(nullptr, Five.get());
 }
 
 void MemoryBufferTest::testGetOpenFileSlice(bool Reopen) {
Index: llvm/lib/Support/MemoryBuffer.cpp
===================================================================
--- llvm/lib/Support/MemoryBuffer.cpp
+++ llvm/lib/Support/MemoryBuffer.cpp
@@ -286,6 +286,8 @@
   StringRef NameRef = BufferName.toStringRef(NameBuf);
   size_t AlignedStringLen = alignTo(sizeof(MemBuffer) + NameRef.size() + 1, 16);
   size_t RealLen = AlignedStringLen + Size + 1;
+  if (RealLen <= Size) // Check for rollover.
+    return nullptr;
   char *Mem = static_cast<char*>(operator new(RealLen, std::nothrow));
   if (!Mem)
     return nullptr;


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D121399.414450.patch
Type: text/x-patch
Size: 1160 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20220310/29e7f21c/attachment.bin>