[cfe-commits] r134588 - in /cfe/trunk: lib/Lex/TokenLexer.cpp test/Preprocessor/macro_paste_hashhash.c

[Chris Lattner]

On Jul 6, 2011, at 8:40 PM, Argyrios Kyrtzidis wrote:

> Author: akirtzidis
> Date: Wed Jul  6 22:40:37 2011
> New Revision: 134588
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=134588&view=rev
> Log:
> When expanding macro arguments, treat '##' coming from an argument as a normal token.
> 
> e.g.
> 
> #define M(x) A x B
> M(##) // should expand to 'A ## B', not 'AB'

Thanks for fixing this Argyrios, one comment though:

>     // Otherwise, this is a use of the argument.  Find out if there is a paste
>     // (##) operator before or after the argument.
>     bool PasteBefore =
> -      !ResultToks.empty() && ResultToks.back().is(tok::hashhash);
> +      !ResultToks.empty() && ResultToks.back().is(tok::hashhash) &&
> +      // If the '##' came from expanding an argument,treat it as a normal token.
> +      SM.isBeforeInSourceLocationOffset(ResultToks.back().getLocation(),
> +                                        MacroStartSLocOffset);

This really shouldn't be using SM.isBeforeInSourceLocationOffset.  The way we handle hashhash elsewhere is to turn it into tok::other when it shouldn't cause token pasting.  For example:

      // Turn ## into 'unknown' to avoid # ## # from looking like a paste
      // operator.
      if (Result.is(tok::hashhash))
        Result.setKind(tok::unknown);

Can you rework this to have the argument expansion logic strip tok::hashhash's from arguments instead of using the SM.isBeforeInSourceLocationOffset query?

-Chris


>     bool PasteAfter = i+1 != e && Tokens[i+1].is(tok::hashhash);
> 
>     // If it is not the LHS/RHS of a ## operator, we must pre-expand the
> @@ -215,7 +220,6 @@
>         ResultToks.append(ResultArgToks, ResultArgToks+NumToks);
> 
>         if(InstantiateLocStart.isValid()) {
> -          SourceManager &SM = PP.getSourceManager();
>           SourceLocation curInst =
>               getMacroExpansionLocation(CurTok.getLocation());
>           assert(curInst.isValid() &&
> @@ -264,7 +268,6 @@
>       ResultToks.append(ArgToks, ArgToks+NumToks);
> 
>       if(InstantiateLocStart.isValid()) {
> -        SourceManager &SM = PP.getSourceManager();
>         SourceLocation curInst =
>             getMacroExpansionLocation(CurTok.getLocation());
>         assert(curInst.isValid() &&
> @@ -371,6 +374,8 @@
>     return PPCache.Lex(Tok);
>   }
> 
> +  SourceManager &SM = PP.getSourceManager();
> +
>   // If this is the first token of the expanded result, we inherit spacing
>   // properties later.
>   bool isFirstToken = CurToken == 0;
> @@ -382,7 +387,10 @@
> 
>   // If this token is followed by a token paste (##) operator, paste the tokens!
>   // Note that ## is a normal token when not expanding a macro.
> -  if (!isAtEnd() && Tokens[CurToken].is(tok::hashhash) && Macro) {
> +  if (!isAtEnd() && Tokens[CurToken].is(tok::hashhash) && Macro &&
> +      // If the '##' came from expanding an argument,treat it as a normal token.
> +      SM.isBeforeInSourceLocationOffset(Tokens[CurToken].getLocation(),
> +                                        MacroStartSLocOffset)) {
>     // When handling the microsoft /##/ extension, the final token is
>     // returned by PasteTokens, not the pasted token.
>     if (PasteTokens(Tok))
> @@ -391,7 +399,6 @@
>     TokenIsFromPaste = true;
>   }
> 
> -  SourceManager &SM = PP.getSourceManager();
>   // The token's current location indicate where the token was lexed from.  We
>   // need this information to compute the spelling of the token, but any
>   // diagnostics for the expanded token should appear as if they came from
> 
> Modified: cfe/trunk/test/Preprocessor/macro_paste_hashhash.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/macro_paste_hashhash.c?rev=134588&r1=134587&r2=134588&view=diff
> ==============================================================================
> --- cfe/trunk/test/Preprocessor/macro_paste_hashhash.c (original)
> +++ cfe/trunk/test/Preprocessor/macro_paste_hashhash.c Wed Jul  6 22:40:37 2011
> @@ -1,7 +1,11 @@
> -// RUN: %clang_cc1 -E %s | grep '^"x ## y";$'
> +// RUN: %clang_cc1 -E %s | FileCheck %s
> #define hash_hash # ## # 
> #define mkstr(a) # a 
> #define in_between(a) mkstr(a) 
> #define join(c, d) in_between(c hash_hash d) 
> +// CHECK: "x ## y";
> join(x, y);
> 
> +#define FOO(x) A x B
> +// CHECK: A ## B;
> +FOO(##);
> 
> 
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits