[PATCH] D59885: [Lex] Allow to consume tokens while preprocessing

[Richard Smith - zygoloid via Phabricator via cfe-commits]

rsmith accepted this revision.
rsmith marked an inline comment as done.
rsmith added a comment.
This revision is now accepted and ready to land.

Thanks, LGTM!



================
Comment at: clang/lib/Lex/Pragma.cpp:370
   // Push the tokens onto the stack.
-  EnterTokenStream(TokArray, PragmaToks.size(), true, true);
+  EnterTokenStream(TokArray, PragmaToks.size(), true, true,
+                   /*IsReinject*/ false);
----------------
ilya-biryukov wrote:
> rsmith wrote:
> > I think this case is a reinjection; we've copied some tokens inside __pragma out into a duplicate position in the token stream. But I guess it doesn't matter because the tokens never escape the outer Lex function anyway.
> Yeah, my logic is that it's not a re-injection in the sense that they were never the phase 4 tokens before.
Yeah, I think you're right. As a weird example, if we had a

    #pragma clang produce_token x

pragma that just produced the given token, it'd be reasonable for it to leave the `Reinjected` flag alone, and it should come out as `false` in this case.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D59885/new/

https://reviews.llvm.org/D59885