[clang-tools-extra] r361007 - [Lex] Allow to consume tokens while preprocessing

[Ilya Biryukov via cfe-commits]

Author: ibiryukov
Date: Fri May 17 02:32:05 2019
New Revision: 361007

URL: http://llvm.org/viewvc/llvm-project?rev=361007&view=rev
Log:
[Lex] Allow to consume tokens while preprocessing

Summary:
By adding a hook to consume all tokens produced by the preprocessor.
The intention of this change is to make it possible to consume the
expanded tokens without re-runnig the preprocessor with minimal changes
to the preprocessor and minimal performance penalty when preprocessing
without recording the tokens.

The added hook is very low-level and reconstructing the expanded token
stream requires more work in the client code, the actual algorithm to
collect the tokens using this hook can be found in the follow-up change.

Reviewers: rsmith

Reviewed By: rsmith

Subscribers: eraman, nemanjai, kbarton, jsji, riccibruno, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D59885

Modified:
    clang-tools-extra/trunk/clang-tidy/modernize/UseTrailingReturnTypeCheck.cpp

Modified: clang-tools-extra/trunk/clang-tidy/modernize/UseTrailingReturnTypeCheck.cpp
URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/clang-tidy/modernize/UseTrailingReturnTypeCheck.cpp?rev=361007&r1=361006&r2=361007&view=diff
==============================================================================
--- clang-tools-extra/trunk/clang-tidy/modernize/UseTrailingReturnTypeCheck.cpp (original)
+++ clang-tools-extra/trunk/clang-tidy/modernize/UseTrailingReturnTypeCheck.cpp Fri May 17 02:32:05 2019
@@ -175,7 +175,9 @@ classifyToken(const FunctionDecl &F, Pre
   Token End;
   End.setKind(tok::eof);
   SmallVector<Token, 2> Stream{Tok, End};
-  PP.EnterTokenStream(Stream, false);
+
+  // FIXME: do not report these token to Preprocessor.TokenWatcher.
+  PP.EnterTokenStream(Stream, false, /*IsReinject=*/false);
   while (true) {
     Token T;
     PP.Lex(T);