[PATCH] D116020: [clang][#52782] Bail on incomplete parameter type in stdcall name mangling

[Markus Böck via Phabricator via cfe-commits]

zero9178 created this revision.
zero9178 added reviewers: mstorsjo, rnk.
zero9178 requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

stdcall name mangling requires a suffix with the number equal to the sum of the byte count of all parameter types. In the case of a function prototype that has a parameter type of an incomplete type it is impossible to get the size of the type. While such a function is not callable or able to be defined in the TU, it may still be mangled when generating debug info, which would previously lead to a crash. 
This patch fixes that by simply bailing out of the loop and using the so far accumulated byte count. This matches GCCs behaviour as well: https://github.com/gcc-mirror/gcc/blob/bc8d6c60137f8bbf173b86ddf31b15d7ba2a33dd/gcc/config/i386/winnt.c#L203

Fixes https://github.com/llvm/llvm-project/issues/52782


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D116020

Files:
  clang/lib/AST/Mangle.cpp
  clang/test/CodeGen/pr52782-stdcall-func-decl.cpp


Index: clang/test/CodeGen/pr52782-stdcall-func-decl.cpp
===================================================================
--- /dev/null
+++ clang/test/CodeGen/pr52782-stdcall-func-decl.cpp
@@ -0,0 +1,11 @@
+// RUN: %clang_cc1 -triple i686-w64-windows-gnu -o - -emit-llvm -debug-info-kind=constructor %s
+
+#define NS_IMETHOD_(type) type __stdcall
+
+enum nsresult {};
+
+class NotNull;
+
+class nsICanvasRenderingContextInternal {
+  NS_IMETHOD_(nsresult) InitializeWithDrawTarget(NotNull);
+} nsTBaseHashSet;
Index: clang/lib/AST/Mangle.cpp
===================================================================
--- clang/lib/AST/Mangle.cpp
+++ clang/lib/AST/Mangle.cpp
@@ -225,11 +225,17 @@
   if (const CXXMethodDecl *MD = dyn_cast<CXXMethodDecl>(FD))
     if (!MD->isStatic())
       ++ArgWords;
-  for (const auto &AT : Proto->param_types())
+  for (const auto &AT : Proto->param_types()) {
+    // If an argument type is incomplete there is no way to get its size to
+    // correctly encode into the mangling scheme.
+    // Follow GCCs behaviour by simply breaking out of the loop.
+    if (AT->isIncompleteType())
+      break;
     // Size should be aligned to pointer size.
     ArgWords +=
         llvm::alignTo(ASTContext.getTypeSize(AT), TI.getPointerWidth(0)) /
         TI.getPointerWidth(0);
+  }
   Out << ((TI.getPointerWidth(0) / 8) * ArgWords);
 }
 


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D116020.395384.patch
Type: text/x-patch
Size: 1371 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20211220/e65549ee/attachment-0001.bin>