[llvm] acaf403 - [SampleProfile] Fix UB in Demangler invocation. (#137659)

[via llvm-commits]

Author: Krzysztof Pszeniczny
Date: 2025-04-28T19:28:56+02:00
New Revision: acaf403c6397dc0fcd8f0988bd057b4d5ee2460f

URL: https://github.com/llvm/llvm-project/commit/acaf403c6397dc0fcd8f0988bd057b4d5ee2460f
DIFF: https://github.com/llvm/llvm-project/commit/acaf403c6397dc0fcd8f0988bd057b4d5ee2460f.diff

LOG: [SampleProfile] Fix UB in Demangler invocation. (#137659)

Currently the backing buffer of a `std::vector<char>` is passed[1] to
`Demangler.getFunctionBaseName`. However, deeply inside the call stack
`OutputBuffer::grow` will call[2] `std::realloc` if it needs to grow the
buffer, leading to UB.

The demangler APIs specify[3] that "`Buf` and `N` behave like the second
and third parameters to `__cxa_demangle`" and the docs for the latter
say[4] that the output buffer must be allocated with `malloc` (but can
also be `NULL` and will then be realloced accordingly).

Note: PR #135863 changed this from a stack array to a `std::vector` and
increased the size to 65K, but this can still lead to a crash if the
demangled name is longer than that - yes, I'm surprised that a >65K-long
function name happens in practice...

[1]:
https://github.com/llvm/llvm-project/blob/d7e631c7cd6d9c13b9519991ec6becf08bc6b8aa/llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp#L744
[2]:
https://github.com/llvm/llvm-project/blob/d7e631c7cd6d9c13b9519991ec6becf08bc6b8aa/llvm/include/llvm/Demangle/Utility.h#L50
[3]:
https://github.com/llvm/llvm-project/blob/d7e631c7cd6d9c13b9519991ec6becf08bc6b8aa/llvm/include/llvm/Demangle/Demangle.h#L92-L93
[4]:
https://gcc.gnu.org/onlinedocs/libstdc++/libstdc++-html-USERS-4.3/a01696.html

Added: 
    

Modified: 
    llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp

Removed: 
    


################################################################################
diff  --git a/llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp b/llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp
index 963c321772d6e..093a39eb4b5d7 100644
--- a/llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp
+++ b/llvm/lib/Transforms/IPO/SampleProfileMatcher.cpp
@@ -737,14 +737,16 @@ bool SampleProfileMatcher::functionMatchesProfileHelper(
     auto FunctionName = FName.str();
     if (Demangler.partialDemangle(FunctionName.c_str()))
       return std::string();
-    constexpr size_t MaxBaseNameSize = 65536;
-    std::vector<char> BaseNameBuf(MaxBaseNameSize, 0);
-    size_t BaseNameSize = MaxBaseNameSize;
-    char *BaseNamePtr =
-        Demangler.getFunctionBaseName(BaseNameBuf.data(), &BaseNameSize);
-    return (BaseNamePtr && BaseNameSize)
-               ? std::string(BaseNamePtr, BaseNameSize)
-               : std::string();
+    size_t BaseNameSize = 0;
+    // The demangler API follows the __cxa_demangle one, and thus needs a
+    // pointer that originates from malloc (or nullptr) and the caller is
+    // responsible for free()-ing the buffer.
+    char *BaseNamePtr = Demangler.getFunctionBaseName(nullptr, &BaseNameSize);
+    std::string Result = (BaseNamePtr && BaseNameSize)
+                             ? std::string(BaseNamePtr, BaseNameSize)
+                             : std::string();
+    free(BaseNamePtr);
+    return Result;
   };
   auto IRBaseName = GetBaseName(IRFunc.getName());
   auto ProfBaseName = GetBaseName(ProfFunc.stringRef());