[PATCH] D59885: [Lex] Allow to consume tokens while preprocessing

[Richard Smith - zygoloid via Phabricator via cfe-commits]

rsmith added a comment.

In D59885#1496734 <https://reviews.llvm.org/D59885#1496734>, @ilya-biryukov wrote:

> Overall, the performance cost in empty-callback case seems to be lower than `5%`. This is significant, but hopefully acceptable.


5% is a lot to pay for something that we won't be using at all during normal preprocessing / compilation. But I think we can simplify this a bit, and that might help.



================
Comment at: clang/lib/Lex/PPCaching.cpp:64
   ExitCachingLexMode();
-  Lex(Result);
+  Lex(Result, Report);
 
----------------
This change seems redundant: `LexLevel` is always `1` here, so this token would never be reported anyway. And with that gone I think you can remove the `Report` parameter entirely.


================
Comment at: clang/lib/Lex/Preprocessor.cpp:892
   do {
     switch (CurLexerKind) {
     case CLK_Lexer:
----------------
Doesn't this always set `Report` to the same value as `IsNewToken`? (The only case we set `Report` to `false` is when we call `CachingLex` and it sets `IsNewToken` to `false`, and `CachingLex` can't be recursively called twice, so its recursive call to `Lex` can't set `Report` to `false`..)


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D59885/new/

https://reviews.llvm.org/D59885