[cfe-dev] motivations for approaches to macros in Index/IndexingAction.cpp ?

[Argyrios Kyrtzidis via cfe-dev]

> On May 20, 2019, at 3:35 PM, James Widman <james.widman at gmail.com> wrote:
> 
> On Mon, May 20, 2019 at 6:32 PM Jan Korous <jkorous at apple.com <mailto:jkorous at apple.com>> wrote:
>> 
>> + Argyrios and Ben who might comment on current use of index-store.
>> + clangd-dev in case anyone is interested.
>> 
>>> On May 20, 2019, at 3:23 PM, James Widman <james.widman at gmail.com> wrote:
>>> 
>>> On Mon, May 20, 2019 at 5:32 PM Jan Korous <jkorous at apple.com> wrote:
>>>> 
>>>> Hi James,
>>>> 
>>>> The current implementation of this hashing function ignores some cases. We're in the process of upstreaming the whole index-while-building feature to LLVM project and decided to fix this by temporarily using trivial implementation - just hashing the full bitcode representation of index record. Eventually we need a new implementation that is both correct and performant - so far the idea was to explore the possibility of using the existing ODR violation hasher for this purpose.
>>> 
>>> Ok, so... I guess "correct and performant" means "yes, we do want to
>>> incorporate macro definitions & expansions in the future", in which
>>> case I'm looking forward to it! (:

Not including macros is only because before enabling them we wanted to get performance metrics to see what impact they will have.

>>> 
>>> separate topic, under "performance": it seems like it would be good if
>>> declarations & definitions of entities with external linkage were not
>>> represented as "occurrences", because currently, if I want to get the
>>> locations of all such decls, it seems like I need to visit all
>>> "occurrences", which seems like a lot of extra work if the user ends
>>> up visiting only two translation units. (And currently, doing this for
>>> an index-store for the clang & llvm source code takes about 11.5
>>> seconds on my macbook.)
>>> 
>>> In other words, upon loading the index-store, I need to know the
>>> locations of all external decls & defs for the whole program (because
>>> the user should be able to jump to any of those from the outset, from
>>> any TU), but it seems like loading for all *other* occurrences could
>>> be done more lazily.
> 
> On second thought, I guess you want to be able to jump to any
> namespace-scope decl from any file (or no file), i.e. you also want to
> load all decls with internal linkage on startup. (That would still be
> a significantly lighter load than "all occurrences" though.)

I’d recommend to take a look at using https://github.com/apple/indexstore-db <https://github.com/apple/indexstore-db> for queries based on the index-store. That one is even more lazily than you suggest in that during loading it only reads the section that contains the Symbol info + roles, and avoids reading the occurrences, and only reads rest of the record if it wants to find the source locations of specific occurrences (e.g. we know this record contains the ‘definition’ occurrence of this USR, so read the record to find out its source location).

> 
> 
>>> I don't know where this topic fits in the pipeline of design
>>> discussions for upstreaming index-while-building tho.
>>> 
>>> --James

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190520/7cc58f9a/attachment.html>