[PATCH] D39050: Add index-while-building support to Clang

[Argyrios Kyrtzidis via Phabricator via cfe-commits]

akyrtzi added a comment.

Hey Marc,

> The fact that both clang and clangd have to agree on the format so that index-while-building can be used seems to make it inherently not possible to extend

I don't think "not possible to extend" is quite correct, we can make it so that the format allows optional data to be recorded.

On the topic of recording the end-loc, I agree it's not much data overall, but it will be useful to examine the uses closely and to figure out whether it's really required and whether it is at the same time inadequate for other uses.

>   I changed my prototype so that the end-loc is not stored in the index but rather computed "on the fly" using SourceManager and Lexer only.

I assume you used SingleFileParseMode+SkipFunctionBodies for this, right ?

> For my little benchmark, I used the LLVM/Clang/Clangd code base which I queried for all references of "std" (the namespace) which is around 46K references in the index.

This is an interesting use case, and I can say we have some experience because Xcode has this functionality without requiring the end-loc for every reference.
So what it does is that it has a 'name' to look for (say 'foo' for  the variable foo) and if it finds the name in the location then it highlights, otherwise if it doesn't find it (e.g. because it is an implicit reference) then it points to the location but doesn't highlight something. The same thing happens for operator overloads (the operators get highlighted at the reference location).
For implicit references it's most likely there's nothing to highlight so the end-loc will most likely be empty anyway (or same as start-loc ?) to indicate an empty range.

> With SingleFileParseMode, I get several errors:

Good point, the parser definitely needs recovery improvements in C++.

> With SkipFunctionBodies alone, I can get the Decl* but FunctionDecl::getSourceRange() doesn't include the body

This seems strange, there's an EndRangeLoc field that should have been filled in, not exactly sure if it is a bug or omission.

Going back to the topic of what use cases end-loc covers, note that it still seems inadequate for peek-definition functionality. You can't set it to body-end loc (otherwise occurrences highlighting would highlight the whole body which I think is undesirable) and you still need to include doc-comments if they exist.


https://reviews.llvm.org/D39050