r184968 - Match MSVC's handling of commas during macro argument expansion

[João Matos]

Nice to see this finally fixed!

On Wed, Jun 26, 2013 at 6:16 PM, Reid Kleckner <reid at kleckner.net> wrote:

> Author: rnk
> Date: Wed Jun 26 12:16:08 2013
> New Revision: 184968
>
> URL: http://llvm.org/viewvc/llvm-project?rev=184968&view=rev
> Log:
> Match MSVC's handling of commas during macro argument expansion
>
> This allows clang to parse the type_traits header in Visual Studio 2012,
> which is included widely in practice.
>
> This is a rework of r163022 by João Matos.  The original patch broke
> preprocessing of gtest headers, which this patch addresses.
>
> Patch by Will Wilson!
>
> Modified:
>     cfe/trunk/include/clang/Lex/Token.h
>     cfe/trunk/lib/Lex/PPMacroExpansion.cpp
>     cfe/trunk/lib/Lex/TokenLexer.cpp
>     cfe/trunk/test/Preprocessor/microsoft-ext.c
>
> Modified: cfe/trunk/include/clang/Lex/Token.h
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Lex/Token.h?rev=184968&r1=184967&r2=184968&view=diff
>
> ==============================================================================
> --- cfe/trunk/include/clang/Lex/Token.h (original)
> +++ cfe/trunk/include/clang/Lex/Token.h Wed Jun 26 12:16:08 2013
> @@ -77,7 +77,8 @@ public:
>      NeedsCleaning = 0x08,  // Contained an escaped newline or trigraph.
>      LeadingEmptyMacro = 0x10, // Empty macro exists before this token.
>      HasUDSuffix = 0x20,    // This string or character literal has a
> ud-suffix.
> -    HasUCN = 0x40          // This identifier contains a UCN.
> +    HasUCN = 0x40,         // This identifier contains a UCN.
> +    IgnoredComma = 0x80,   // This comma is not a macro argument
> separator (MS).
>    };
>
>    tok::TokenKind getKind() const { return (tok::TokenKind)Kind; }
>
> Modified: cfe/trunk/lib/Lex/PPMacroExpansion.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPMacroExpansion.cpp?rev=184968&r1=184967&r2=184968&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/PPMacroExpansion.cpp (original)
> +++ cfe/trunk/lib/Lex/PPMacroExpansion.cpp Wed Jun 26 12:16:08 2013
> @@ -458,7 +458,12 @@ MacroArgs *Preprocessor::ReadFunctionLik
>          }
>        } else if (Tok.is(tok::l_paren)) {
>          ++NumParens;
> -      } else if (Tok.is(tok::comma) && NumParens == 0) {
> +      } else if (Tok.is(tok::comma) && NumParens == 0 &&
> +                 !(Tok.getFlags() & Token::IgnoredComma)) {
> +        // In Microsoft-compatibility mode, single commas from nested
> macro
> +        // expansions should not be considered as argument separators. We
> test
> +        // for this with the IgnoredComma token flag above.
> +
>          // Comma ends this argument if there are more fixed arguments
> expected.
>          // However, if this is a variadic macro, and this is part of the
>          // variadic part, then the comma is just an argument token.
>
> Modified: cfe/trunk/lib/Lex/TokenLexer.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/TokenLexer.cpp?rev=184968&r1=184967&r2=184968&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/TokenLexer.cpp (original)
> +++ cfe/trunk/lib/Lex/TokenLexer.cpp Wed Jun 26 12:16:08 2013
> @@ -278,6 +278,14 @@ void TokenLexer::ExpandFunctionArguments
>          unsigned NumToks = MacroArgs::getArgLength(ResultArgToks);
>          ResultToks.append(ResultArgToks, ResultArgToks+NumToks);
>
> +        // In Microsoft-compatibility mode, we follow MSVC's preprocessing
> +        // behavior by not considering single commas from nested macro
> +        // expansions as argument separators. Set a flag on the token so
> we can
> +        // test for this later when the macro expansion is processed.
> +        if (PP.getLangOpts().MicrosoftMode && NumToks == 1 &&
> +            ResultToks.back().is(tok::comma))
> +          ResultToks.back().setFlag(Token::IgnoredComma);
> +
>          // If the '##' came from expanding an argument, turn it into
> 'unknown'
>          // to avoid pasting.
>          for (unsigned i = FirstResult, e = ResultToks.size(); i != e;
> ++i) {
>
> Modified: cfe/trunk/test/Preprocessor/microsoft-ext.c
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/microsoft-ext.c?rev=184968&r1=184967&r2=184968&view=diff
>
> ==============================================================================
> --- cfe/trunk/test/Preprocessor/microsoft-ext.c (original)
> +++ cfe/trunk/test/Preprocessor/microsoft-ext.c Wed Jun 26 12:16:08 2013
> @@ -22,3 +22,15 @@ class GMOCK_ACTION_CLASS_(name, value_pa
>  ACTION_TEMPLATE(InvokeArgument,
>                  HAS_1_TEMPLATE_PARAMS(int, k),
>                  AND_2_VALUE_PARAMS(p0, p1));
> +
> +// This tests compatibility with behaviour needed for type_traits in
> VS2012
> +// Test based on _VARIADIC_EXPAND_0X macros in xstddef of VS2012
> +#define _COMMA ,
> +
> +#define MAKER(_arg1, _comma, _arg2)
>      \
> +  void func(_arg1 _comma _arg2) {}
> +#define MAKE_FUNC(_makerP1, _makerP2, _arg1, _comma, _arg2)
>      \
> +  _makerP1##_makerP2(_arg1, _comma, _arg2)
> +
> +MAKE_FUNC(MAK, ER, int a, _COMMA, int b);
> +// CHECK: void func(int a , int b) {}
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>



-- 
João Matos
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130626/a5ebf49d/attachment.html>