[compiler-rt] [sanitizer] Fail __sanitizer_symbolize_demangle instead of returning input (PR #66006)

[Vitaly Buka via llvm-commits]

https://github.com/vitalybuka created https://github.com/llvm/llvm-project/pull/66006:

LLVMSymbolizer::DemangleName returns the same input if it can't demangle.
We can't tell if this is already demangled or format is unsupported.

Internally DemangleName uses nonMicrosoftDemangle which can report a failure.


>From 5aba4cd74fb4ded346e944f79f7dc456db36e725 Mon Sep 17 00:00:00 2001
From: Vitaly Buka <vitalybuka at google.com>
Date: Sat, 9 Sep 2023 13:15:33 -0700
Subject: [PATCH] [sanitizer] Fail __sanitizer_symbolize_demangle instead of
 returning input

LLVMSymbolizer::DemangleName returns the same input if it can't demangle.
We can't tell if this is already demangled or format is unsupported.

Internally DemangleName uses nonMicrosoftDemangle which can report a failure.
---
 .../lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp b/compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp
index 6ca4b9835bf3acd..f6dac78337068b5 100644
--- a/compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp
+++ b/compiler-rt/lib/sanitizer_common/symbolizer/sanitizer_symbolize.cpp
@@ -17,6 +17,7 @@
 
 #include "llvm/DebugInfo/Symbolize/DIPrinter.h"
 #include "llvm/DebugInfo/Symbolize/Symbolize.h"
+#include "llvm/Demangle/Demangle.h"
 
 static llvm::symbolize::LLVMSymbolizer *Symbolizer = nullptr;
 static bool Demangle = true;
@@ -117,8 +118,9 @@ void __sanitizer_symbolize_flush() {
 
 bool __sanitizer_symbolize_demangle(const char *Name, char *Buffer,
                                    int MaxLength) {
-  std::string Result =
-      llvm::symbolize::LLVMSymbolizer::DemangleName(Name, nullptr);
+  std::string Result;
+  if (!llvm::nonMicrosoftDemangle(Name, Result))
+    return false;
   return __sanitizer::internal_snprintf(Buffer, MaxLength, "%s",
                                         Result.c_str()) < MaxLength;
 }