r235186 - [MSVC] Mimic MSVC whitespace collapse for incompatible token pasting

[Will Wilson]

Author: lantictac
Date: Fri Apr 17 07:43:57 2015
New Revision: 235186

URL: http://llvm.org/viewvc/llvm-project?rev=235186&view=rev
Log:
[MSVC] Mimic MSVC whitespace collapse for incompatible token pasting

In public MS headers for XAudio, clang would fail to generate a valid UUID due to the UUID components being combined with the '-' UUID separators. Clang would attempting to recover but would preserve the leading whitespace from the tokens after each failed paste leading to spaces creeping into the UUID and causing an error in the __declspace(uuid()) parsing.

Reference: Microsoft DirectX SDK (June 2010)\Include\XAudio2.h(51)

Resolves http://llvm.org/pr23071

Modified:
    cfe/trunk/lib/Lex/TokenLexer.cpp
    cfe/trunk/test/Preprocessor/macro_paste_msextensions.c

Modified: cfe/trunk/lib/Lex/TokenLexer.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/TokenLexer.cpp?rev=235186&r1=235185&r2=235186&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/TokenLexer.cpp (original)
+++ cfe/trunk/lib/Lex/TokenLexer.cpp Fri Apr 17 07:43:57 2015
@@ -521,6 +521,13 @@ bool TokenLexer::Lex(Token &Tok) {
 /// are more ## after it, chomp them iteratively.  Return the result as Tok.
 /// If this returns true, the caller should immediately return the token.
 bool TokenLexer::PasteTokens(Token &Tok) {
+  // MSVC: If previous token was pasted, this must be a recovery from an invalid
+  // paste operation. Ignore spaces before this token to mimic MSVC output.
+  // Required for generating valid UUID strings in some MS headers.
+  if (PP.getLangOpts().MicrosoftExt && (CurToken >= 2) &&
+      Tokens[CurToken - 2].is(tok::hashhash))
+    Tok.clearFlag(Token::LeadingSpace);
+  
   SmallString<128> Buffer;
   const char *ResultTokStrPtr = nullptr;
   SourceLocation StartLoc = Tok.getLocation();

Modified: cfe/trunk/test/Preprocessor/macro_paste_msextensions.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/macro_paste_msextensions.c?rev=235186&r1=235185&r2=235186&view=diff
==============================================================================
--- cfe/trunk/test/Preprocessor/macro_paste_msextensions.c (original)
+++ cfe/trunk/test/Preprocessor/macro_paste_msextensions.c Fri Apr 17 07:43:57 2015
@@ -32,3 +32,10 @@ nested(baz)  rise of the dead tokens
 bar(q)
 
 // CHECK: abc(baz(q))
+
+
+#define str(x) #x
+#define collapse_spaces(a, b, c, d) str(a ## - ## b ## - ## c ## d)
+collapse_spaces(1a, b2, 3c, d4)
+
+// CHECK: "1a-b2-3cd4"