[PATCH] D123289: [clangd][SymbolCollector] Introduce a cache for SymbolID generation and some cleanups

[Kadir Cetinkaya via Phabricator via cfe-commits]

kadircet added inline comments.


================
Comment at: clang-tools-extra/clangd/index/SymbolCollector.cpp:189
+  clang::Token Tok;
+  if (clang::Lexer::getRawToken(Loc, Tok, SM, LO))
+    return false;
----------------
sammccall wrote:
> You've changed this from tokenizing the file with a cache.
> If I'm reading your benchmark spreadsheet right, this is ~3% speedup.
> 
> I'm not sure this is significant (I imagine not), but you don't actually have to run the lexer here, since you already know what the string is going to be, it's enough to grab the buffer pointer, check that it starts with the text, check that the next character is not an identifier-continuer.
right, I actually did that at first, but it implies keeping the behaviour around unclean tokens broken, and there wasn't much of a win (delta was in the noise). I think because the expensive part is actually figuring out the fileid, and the lexing call can do it cheaply right now as it benefits from the single element cache.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D123289/new/

https://reviews.llvm.org/D123289