r367530 - [Preprocessor] Always discard body of #define if we failed to parse it

[Hans Wennborg via cfe-commits]

Merged to release_90 in r367681.

On Thu, Aug 1, 2019 at 11:09 AM Ilya Biryukov via cfe-commits
<cfe-commits at lists.llvm.org> wrote:
>
> Author: ibiryukov
> Date: Thu Aug  1 02:10:37 2019
> New Revision: 367530
>
> URL: http://llvm.org/viewvc/llvm-project?rev=367530&view=rev
> Log:
> [Preprocessor] Always discard body of #define if we failed to parse it
>
> Summary:
> Preivously we would only discard it if we failed to parse parameter lists.
> If we do not consume the body, parser sees tokens inside directive. In
> turn, this leads to spurious diagnostics and a crash in TokenBuffer, see
> the added tests.
>
> Reviewers: sammccall
>
> Reviewed By: sammccall
>
> Subscribers: cfe-commits
>
> Tags: #clang
>
> Differential Revision: https://reviews.llvm.org/D65517
>
> Added:
>     cfe/trunk/test/Preprocessor/stringize_skipped.c
> Modified:
>     cfe/trunk/lib/Lex/PPDirectives.cpp
>     cfe/trunk/unittests/Tooling/Syntax/TokensTest.cpp
>
> Modified: cfe/trunk/lib/Lex/PPDirectives.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/PPDirectives.cpp?rev=367530&r1=367529&r2=367530&view=diff
> ==============================================================================
> --- cfe/trunk/lib/Lex/PPDirectives.cpp (original)
> +++ cfe/trunk/lib/Lex/PPDirectives.cpp Thu Aug  1 02:10:37 2019
> @@ -33,6 +33,7 @@
>  #include "clang/Lex/Token.h"
>  #include "clang/Lex/VariadicMacroSupport.h"
>  #include "llvm/ADT/ArrayRef.h"
> +#include "llvm/ADT/ScopeExit.h"
>  #include "llvm/ADT/SmallString.h"
>  #include "llvm/ADT/SmallVector.h"
>  #include "llvm/ADT/STLExtras.h"
> @@ -2399,6 +2400,13 @@ MacroInfo *Preprocessor::ReadOptionalMac
>    Token Tok;
>    LexUnexpandedToken(Tok);
>
> +  // Ensure we consume the rest of the macro body if errors occur.
> +  auto _ = llvm::make_scope_exit([&]() {
> +    // The flag indicates if we are still waiting for 'eod'.
> +    if (CurLexer->ParsingPreprocessorDirective)
> +      DiscardUntilEndOfDirective();
> +  });
> +
>    // Used to un-poison and then re-poison identifiers of the __VA_ARGS__ ilk
>    // within their appropriate context.
>    VariadicMacroScopeGuard VariadicMacroScopeGuard(*this);
> @@ -2420,12 +2428,8 @@ MacroInfo *Preprocessor::ReadOptionalMac
>    } else if (Tok.is(tok::l_paren)) {
>      // This is a function-like macro definition.  Read the argument list.
>      MI->setIsFunctionLike();
> -    if (ReadMacroParameterList(MI, LastTok)) {
> -      // Throw away the rest of the line.
> -      if (CurPPLexer->ParsingPreprocessorDirective)
> -        DiscardUntilEndOfDirective();
> +    if (ReadMacroParameterList(MI, LastTok))
>        return nullptr;
> -    }
>
>      // If this is a definition of an ISO C/C++ variadic function-like macro (not
>      // using the GNU named varargs extension) inform our variadic scope guard
>
> Added: cfe/trunk/test/Preprocessor/stringize_skipped.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Preprocessor/stringize_skipped.c?rev=367530&view=auto
> ==============================================================================
> --- cfe/trunk/test/Preprocessor/stringize_skipped.c (added)
> +++ cfe/trunk/test/Preprocessor/stringize_skipped.c Thu Aug  1 02:10:37 2019
> @@ -0,0 +1,5 @@
> +// RUN: %clang_cc1 -fsyntax-only -verify %s
> +// Ensure we see the error from PP and do not see errors from the parser.
> +
> +// expected-error at +1{{'#' is not followed by a macro parameter}}
> +#define INVALID() #B 10+10
>
> Modified: cfe/trunk/unittests/Tooling/Syntax/TokensTest.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/unittests/Tooling/Syntax/TokensTest.cpp?rev=367530&r1=367529&r2=367530&view=diff
> ==============================================================================
> --- cfe/trunk/unittests/Tooling/Syntax/TokensTest.cpp (original)
> +++ cfe/trunk/unittests/Tooling/Syntax/TokensTest.cpp Thu Aug  1 02:10:37 2019
> @@ -298,6 +298,21 @@ file './input.cpp'
>    spelled tokens:
>      <empty>
>    no mappings.
> +)"},
> +      // Should not crash on errors inside '#define' directives. Error is that
> +      // stringification (#B) does not refer to a macro parameter.
> +      {
> +          R"cpp(
> +a
> +#define MACRO() A #B
> +)cpp",
> +          R"(expanded tokens:
> +  a
> +file './input.cpp'
> +  spelled tokens:
> +    a # define MACRO ( ) A # B
> +  mappings:
> +    ['#'_1, '<eof>'_9) => ['<eof>'_1, '<eof>'_1)
>  )"}};
>    for (auto &Test : TestCases)
>      EXPECT_EQ(collectAndDump(Test.first), Test.second)
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits