[PATCH] Microsoft mangling of return type deducing functions/lambdas that return local types

[Faisal Vali]

faisalv added you to the CC list for the revision "Microsoft mangling of return type deducing functions/lambdas that return local types".

Hi rnk, majnemer,

The following code causes clang to enter into an infinite recursion on windows 7 while attempting to microsoft-mangle the following lambda's operator function type:


  auto L = [] () {
     struct M {};
     return M{};
  };  

  auto X = L();


I have kludged together a quick and dirty patch to identify where the problem lies, so that those who understand the microsoft-mangling process better can either help me refine it - or if they prefer - modify it to make it commit-worthy and commit it.


 

http://llvm-reviews.chandlerc.com/D3221

Files:
  lib/AST/MicrosoftMangle.cpp

Index: lib/AST/MicrosoftMangle.cpp
===================================================================
--- lib/AST/MicrosoftMangle.cpp
+++ lib/AST/MicrosoftMangle.cpp
@@ -13,6 +13,7 @@
 
 #include "clang/AST/Mangle.h"
 #include "clang/AST/ASTContext.h"
+#include "clang/AST/ASTLambda.h"
 #include "clang/AST/Attr.h"
 #include "clang/AST/CXXInheritance.h"
 #include "clang/AST/CharUnits.h"
@@ -1549,9 +1550,38 @@
     Out << '@';
   } else {
     QualType ResultType = Proto->getReturnType();
+    bool AlreadyMangledReturnType = false;
     if (ResultType->isVoidType())
       ResultType = ResultType.getUnqualifiedType();
-    mangleType(ResultType, Range, QMM_Result);
+    else if (ResultType->getContainedAutoType()) {
+      if (ResultType->isRecordType()) {
+        CXXRecordDecl *RD = ResultType->getAsCXXRecordDecl();
+        // Is this type defined within the function/lambda itself and 
+        // deduced as the return type? i.e
+        //   auto fun() { struct M{}; return M{}; } 
+        // If so, prevent infinite recursion, by mangling this here as a temporary fix me.
+        if (FunctionDecl *FD = dyn_cast<FunctionDecl>(RD->getDeclContext())) {
+          while (FD) {
+            FD = FD->getFirstDecl();
+            if (FD == D) {
+              Out << '?';
+              mangleQualifiers(ResultType.getLocalQualifiers(), false);
+              mangleSourceName("<fixme-nested-local-tag>");
+              AlreadyMangledReturnType = true;
+              break;
+            }
+            FD = dyn_cast<FunctionDecl>(getLambdaAwareParentOfDeclContext(FD));
+          }
+        }
+      } else if (ResultType->isUndeducedType()) {
+        Out << '?';
+        mangleQualifiers(ResultType.getLocalQualifiers(), false);
+        mangleSourceName("<fixme-undeduced-auto-return-type>");
+        AlreadyMangledReturnType = true;
+      }
+    }
+    if (!AlreadyMangledReturnType)  
+      mangleType(ResultType, Range, QMM_Result);
   }
 
   // <argument-list> ::= X # void
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D3221.1.patch
Type: text/x-patch
Size: 2003 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20140330/cc241cce/attachment.bin>