[clang-tools-extra] [clangd] Correctly identify the next token after the completion point (PR #69153)

[Younan Zhang via cfe-commits]

================
@@ -1491,6 +1491,46 @@ FuzzyFindRequest speculativeFuzzyFindRequestForCompletion(
   return CachedReq;
 }
 
+// This function is similar to Lexer::findNextToken(), but assumes
+// that the input SourceLocation is the completion point (which is
+// a case findNextToken() does not handle).
+std::optional<Token>
+findTokenAfterCompletionPoint(SourceLocation CompletionPoint,
+                              const SourceManager &SM,
+                              const LangOptions &LangOpts) {
+  SourceLocation Loc = CompletionPoint;
----------------
zyn0217 wrote:

I was about to comment that we should add an assertion here to reflect the contract. However, after going through the codes, it seems hard to tell the token kind under a source location without a lexer. :(

Also, do you think it's possible to make this function a static member of `Lexer` like `Lexer::findNextToken`? That way we might need a slight refactor to reuse some common logics.

https://github.com/llvm/llvm-project/pull/69153