[clang] abb9eb2 - [Lex] Handle repl_input_end in Preprocessor::LexTokensUntilEOF()

[Jonas Hahnfeld via cfe-commits]

Author: Jonas Hahnfeld
Date: 2023-10-05T12:09:14+02:00
New Revision: abb9eb2778dc68ca3bc2ca786632e65682741469

URL: https://github.com/llvm/llvm-project/commit/abb9eb2778dc68ca3bc2ca786632e65682741469
DIFF: https://github.com/llvm/llvm-project/commit/abb9eb2778dc68ca3bc2ca786632e65682741469.diff

LOG: [Lex] Handle repl_input_end in Preprocessor::LexTokensUntilEOF()

This fixes many unit tests when trying to enable IncrementalExtensions
by default for testing purposes.

Differential Revision: https://reviews.llvm.org/D158415

Added: 
    

Modified: 
    clang/lib/Lex/Preprocessor.cpp

Removed: 
    


################################################################################
diff  --git a/clang/lib/Lex/Preprocessor.cpp b/clang/lib/Lex/Preprocessor.cpp
index b82dd1fe49c7040..c16dd75fbbd3f53 100644
--- a/clang/lib/Lex/Preprocessor.cpp
+++ b/clang/lib/Lex/Preprocessor.cpp
@@ -1002,7 +1002,8 @@ void Preprocessor::LexTokensUntilEOF(std::vector<Token> *Tokens) {
   while (1) {
     Token Tok;
     Lex(Tok);
-    if (Tok.isOneOf(tok::unknown, tok::eof, tok::eod))
+    if (Tok.isOneOf(tok::unknown, tok::eof, tok::eod,
+                    tok::annot_repl_input_end))
       break;
     if (Tokens != nullptr)
       Tokens->push_back(Tok);