[cfe-dev] RFC: Upstreaming index-while-building

[Dmitri Gribenko via cfe-dev]

Hi Jan,

I'm very happy that you're picking up this work again!

Since clangd team and the Apple source tools team talked last time,
clangd became a lot more full-featured, and I think there's a lot of
overlap between index-while-building and indexing that is already
built in clangd in the open source repository.

I would like to suggest that we figure out a way to unify these
indexing implementations.  The value proposition for the community is
that there is no feature duplication.  The value proposition for you
is that index-while-building would be able to reuse the infrastructure
that clangd has already built.  For example, global code completion,
or the fast index that supports complex and fuzzy queries (Dex,
http://lists.llvm.org/pipermail/cfe-dev/2018-July/058487.html).

My strawman proposal is that index-while-building should use the same
data structures as clangd for representing symbols -- please take a
look at clang-tools-extra/clangd/index/Index.h.  It would be also
great if index-while-building could reuse the on-disk serialization
format for the symbol information --
clang-tools-extra/clangd/index/Serialization.h.

These are central to the indexing system, and I think we should reuse
them.  Doing so would allow you to reuse other indexing
infrastructure, and infrastructure build on top of indexing, like Dex.
I'm afraid if index-while-building does not speak the same data
structures for symbols, it is unlikely that the two implementations
will ever converge.

What do you think?

Dmitri