[PATCH] Prevent macro argument expansion when the LHS of concatenation is empty

[Harald van Dijk]

Hi all,

As mentioned in http://llvm.org/bugs/show_bug.cgi?id=12767, and what I 
again reported as http://llvm.org/bugs/show_bug.cgi?id=15661, when 
concatenating x ## y, argument expansion for y should be suppressed even 
when x is empty. Does this approach look okay? All preprocessor tests 
pass, a few other tests were already failing for me and continue to fail 
the same way, there are no new failures.

This is my first contribution to clang, I am not yet familiar with the 
process. I should not need to do more than send this patch to the list, 
correct?

Cheers,
Harald
-------------- next part --------------
Author: Harald van Dijk <harald at gigawatt.nl>
Date:   Wed Apr 10 22:03:15 2013 +0200

    Prevent expansion of y in x ## y when x is empty
    
    When x is empty, x ## is suppressed, and when y gets expanded, the fact that it follows ## is not
    available in the macro expansion result. The macro definition can be checked instead, the ## will
    be available there regardless of what x expands to.
    
    PR12767

diff --git a/lib/Lex/TokenLexer.cpp b/lib/Lex/TokenLexer.cpp
index 5b41fe9..580d211 100644
--- a/lib/Lex/TokenLexer.cpp
+++ b/lib/Lex/TokenLexer.cpp
@@ -244,9 +244,11 @@ void TokenLexer::ExpandFunctionArguments() {
 
     // Otherwise, this is a use of the argument.  Find out if there is a paste
     // (##) operator before or after the argument.
-    bool PasteBefore =
+    bool NonEmptyPasteBefore =
       !ResultToks.empty() && ResultToks.back().is(tok::hashhash);
+    bool PasteBefore = i != 0 && Tokens[i-1].is(tok::hashhash);
     bool PasteAfter = i+1 != e && Tokens[i+1].is(tok::hashhash);
+    assert(!NonEmptyPasteBefore || PasteBefore);
 
     // In Microsoft mode, remove the comma before __VA_ARGS__ to ensure there
     // are no trailing commas if __VA_ARGS__ is empty.
@@ -314,7 +316,7 @@ void TokenLexer::ExpandFunctionArguments() {
       // that __VA_ARGS__ expands to multiple tokens, avoid a pasting error when
       // the expander trys to paste ',' with the first token of the __VA_ARGS__
       // expansion.
-      if (PasteBefore && ResultToks.size() >= 2 &&
+      if (NonEmptyPasteBefore && ResultToks.size() >= 2 &&
           ResultToks[ResultToks.size()-2].is(tok::comma) &&
           (unsigned)ArgNo == Macro->getNumArgs()-1 &&
           Macro->isVariadic()) {
@@ -350,7 +352,7 @@ void TokenLexer::ExpandFunctionArguments() {
       // case, we do not want the extra whitespace to be added.  For example,
       // we want ". ## foo" -> ".foo" not ". foo".
       if ((CurTok.hasLeadingSpace() || NextTokGetsSpace) &&
-          !PasteBefore)
+          !NonEmptyPasteBefore)
         ResultToks[ResultToks.size()-NumToks].setFlag(Token::LeadingSpace);
 
       NextTokGetsSpace = false;
@@ -371,10 +373,14 @@ void TokenLexer::ExpandFunctionArguments() {
     }
 
     // If this is on the RHS of a paste operator, we've already copied the
-    // paste operator to the ResultToks list.  Remove it.
-    assert(PasteBefore && ResultToks.back().is(tok::hashhash));
-    NextTokGetsSpace |= ResultToks.back().hasLeadingSpace();
-    ResultToks.pop_back();
+    // paste operator to the ResultToks list, unless the LHS was empty too.
+    // Remove it.
+    assert(PasteBefore);
+    if (NonEmptyPasteBefore) {
+      assert(ResultToks.back().is(tok::hashhash));
+      NextTokGetsSpace |= ResultToks.back().hasLeadingSpace();
+      ResultToks.pop_back();
+    }
 
     // If this is the __VA_ARGS__ token, and if the argument wasn't provided,
     // and if the macro had at least one real argument, and if the token before
diff --git a/test/Preprocessor/macro_paste_empty.c b/test/Preprocessor/macro_paste_empty.c
index 2e26f14..78feb19 100644
--- a/test/Preprocessor/macro_paste_empty.c
+++ b/test/Preprocessor/macro_paste_empty.c
@@ -1,6 +1,7 @@
 // RUN: %clang_cc1 -E %s | grep 'a:Y'
 // RUN: %clang_cc1 -E %s | grep 'b:Y'
 // RUN: %clang_cc1 -E %s | grep 'c:YY'
+// RUN: %clang_cc1 -E %s | grep 'd:FOO4'
 
 #define FOO(X) X ## Y
 a:FOO()
@@ -11,3 +12,5 @@ b:FOO2()
 #define FOO3(X) X ## Y ## X ## Y ## X ## X
 c:FOO3()
 
+#define FOO4(X, Y) X ## Y
+d:FOO4(,FOO4(,))