[clang] [clang-repl] Address error recovery fixing infinite loop while parsing (PR #127569)

[Aaron Ballman via cfe-commits]

================
@@ -519,6 +519,14 @@ class Parser : public CodeCompletionHandler {
   bool SkipUntil(ArrayRef<tok::TokenKind> Toks,
                  SkipUntilFlags Flags = static_cast<SkipUntilFlags>(0));
 
+  /// Determine if the given token marks the end of the current partial
+  /// translation unit. In incremental (REPL) mode, this checks for
+  /// annot_repl_input_end. In normal compilation, this checks for EOF.
+  static bool isAtInputEnd(const Token &T, const LangOptions &LO) {
----------------
AaronBallman wrote:

Thinking out loud, but...

I think this is a red flag; needing to handle eof *and* the annotation token in the same place basically everywhere means every compilation pays the cost of clang-repl even if the repl bits aren't of value (e.g., for a regular compilation workflow). This isn't the first place we've had to go and stub this sort of change in, either.

Is this a sign that we can get rid of the annotation entirely and in the few places where repl behavior is different, use `LO.IncrementalExtensions && T.is(tok::eof)` in just those places?

https://github.com/llvm/llvm-project/pull/127569