[clang] [clang-format] Fix template parsing regression for unspaced user-defi… (PR #210630)

[via cfe-commits]

johnnyb2543 wrote:

@owenca I 

> > The lexer failed to correctly separate the operator keyword and the user defined literal ""_mag as described in the original bug.
> 
> `operator` and `""_mag` are already two separate tokens, and the bug report doesn't complain that there is no space between them.
> 
> > Since FormatTokenLexer::tryMergeUserDefinedLiteral did not properly merge the operator token and its suffix (e.g., _mag), the < token was evaluated as a comparison operator, inserting spaces both before and after the < token.
> 
> We should _not_ merge `operator` and `""_mag` into a single token as some users might want a space to separate them.
> 
> > * Updated FormatTokenLexer::tryMergeUserDefinedLiteral to ensure proper token merging for UDLs
> 
> The bug is in the token annotator itself, and we should leave the token lexer alone.

Thanks for the feedback. I restored FormatTokenLexer.cpp and I modified TokenAnnotator.cpp instead.

https://github.com/llvm/llvm-project/pull/210630