r200787 - Fix whitespace handling in empty macro expansions

[Richard Smith]

On Tue, Feb 4, 2014 at 11:18 AM, Justin Bogner <mail at justinbogner.com>wrote:

> Author: bogner
> Date: Tue Feb  4 13:18:35 2014
> New Revision: 200787
>
> URL: http://llvm.org/viewvc/llvm-project?rev=200787&view=rev
> Log:
> Fix whitespace handling in empty macro expansions
>
> When a macro expansion does not result in any tokens, and the macro name
> is preceded by whitespace, the whitespace should be passed to the first
> token that follows the macro expansion. Similarly when a macro expansion
> ends with a placemarker token, and that placemarker token is preceded by
> whitespace. This worked already for top-level macro expansions, but is
> now extended to also work for nested macro expansions.
>
> Patch by Harald van Dijk!
>
> Modified:
>     cfe/trunk/lib/Lex/TokenLexer.cpp
>     cfe/trunk/test/Preprocessor/macro_space.c
>
> Modified: cfe/trunk/lib/Lex/TokenLexer.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/TokenLexer.cpp?rev=200787&r1=200786&r2=200787&view=diff
>
> ==============================================================================
> --- cfe/trunk/lib/Lex/TokenLexer.cpp (original)
> +++ cfe/trunk/lib/Lex/TokenLexer.cpp Tue Feb  4 13:18:35 2014
> @@ -477,9 +477,13 @@ bool TokenLexer::Lex(Token &Tok) {
>    if (isFirstToken) {
>      Tok.setFlagValue(Token::StartOfLine , AtStartOfLine);
>      Tok.setFlagValue(Token::LeadingSpace, HasLeadingSpace);
> -    AtStartOfLine = false;
> -    HasLeadingSpace = false;
> +  } else {
> +    // If this is not the first token, we may still need to pass through
> +    // leading whitespace if we've expanded a macro.
> +    if (HasLeadingSpace) Tok.setFlag(Token::LeadingSpace);
>

Do we need to do this for AtStartOfLine too? What happens if we have a
macro at the start of a line that expands to no tokens, within a
TokenLexer? (I guess this can't happen within a macro expansion, since no
tokens there are ever at the start of a line after line-splicing, but we
use TokenLexers in other cases too).


>    }
> +  AtStartOfLine = false;
> +  HasLeadingSpace = false;
>
>    // Handle recursive expansion!
>    if (!Tok.isAnnotation() && Tok.getIdentifierInfo() != 0) {
>
> Modified: cfe/trunk/test/Preprocessor/macro_space.c
> URL:
> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/macro_space.c?rev=200787&r1=200786&r2=200787&view=diff
>
> ==============================================================================
> --- cfe/trunk/test/Preprocessor/macro_space.c (original)
> +++ cfe/trunk/test/Preprocessor/macro_space.c Tue Feb  4 13:18:35 2014
> @@ -1,6 +1,36 @@
>  // RUN: %clang_cc1 -E %s | FileCheck --strict-whitespace %s
>
> -#define XX
> -! XX,
> +#define FOO1()
> +#define FOO2(x)x
> +#define FOO3(x) x
> +#define FOO4(x)x x
> +#define FOO5(x) x x
> +#define FOO6(x) [x]
> +#define FOO7(x) [ x]
> +#define FOO8(x) [x ]
>
> -// CHECK: {{^}}! ,{{$}}
> +#define TEST(FOO,x) FOO <FOO()> < FOO()> <FOO ()> <FOO( )> <FOO() >
> <FOO()x> <FOO() x> < FOO()x>
> +
> +TEST(FOO1,)
> +// CHECK: FOO1 <> < > <> <> < > <> < > < >
> +
> +TEST(FOO2,)
> +// CHECK: FOO2 <> < > <> <> < > <> < > < >
> +
> +TEST(FOO3,)
> +// CHECK: FOO3 <> < > <> <> < > <> < > < >
> +
> +TEST(FOO4,)
> +// CHECK: FOO4 < > < > < > < > < > < > < > < >
> +
> +TEST(FOO5,)
> +// CHECK: FOO5 < > < > < > < > < > < > < > < >
> +
> +TEST(FOO6,)
> +// CHECK: FOO6 <[]> < []> <[]> <[]> <[] > <[]> <[] > < []>
> +
> +TEST(FOO7,)
> +// CHECK: FOO7 <[ ]> < [ ]> <[ ]> <[ ]> <[ ] > <[ ]> <[ ] > < [ ]>
> +
> +TEST(FOO8,)
> +// CHECK: FOO8 <[ ]> < [ ]> <[ ]> <[ ]> <[ ] > <[ ]> <[ ] > < [ ]>
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20140204/a2d7befc/attachment.html>