[clang] [clang-repl] Address error recovery fixing infinite loop while parsing (PR #127569)

[Vassil Vassilev via cfe-commits]

vgvassilev wrote:

> Thanks for the reply.
> 
> > I suspect we could somehow invert the rule considering eof as a non-terminator when incremental extensions is on. In this case we will have to figure out the places where eof should be treated as a terminator...
> 
> So if i understand correctly, we could probably start with something like this
> 
>     1. Add a tiny predicate in Parser and use it where a loop means “stop at end of the current unit of input”:
> 
> 
> ```
> // Parser.h
> bool isChunkEnd(const Token &T) const {
>   if (getLangOpts().IncrementalExtensions)
>     return T.is(tok::annot_repl_input_end);
>   return T.is(tok::eof);
> }
> bool isNotChunkEnd(const Token &T) const { return !isChunkEnd(T); }
> ```
> 
>     2. Then update parser loops (compound stmt bodies, namespace/class bodies present in `clang/lib/Parse/*`)
> 
> 
> ```
> - while (Tok.isNot(tok::r_brace) && Tok.isNot(tok::eof)) {
> + while (Tok.isNot(tok::r_brace) && isNotChunkEnd(Tok)) {
>     ...
> }
> ```
> 
>     3. We wouldn't need to touch other `tok::eof` instances (Rewriter / Frontend / HTMLRewrite / tidy checks) I think. Only the ones inside `Parse` would need to be changed.
> 
> 
> This should work I think. Let me know if you think we could try this !

Nominally, yes.

https://github.com/llvm/llvm-project/pull/127569