[cfe-dev] Adding indexing support to Clangd

[Marc-André Laperle via cfe-dev]

Hi!


Just a for a quick update: I'm in the process of making small patches for review, as you might have seen already. I also work in parallel on fixing bugs and polishing the different component that are not as ready, on the Github branch. I though maybe it would be useful to have more information of what's currently there so I added a README to the branch and will keep it up to date as I change things.
<https://raw.githubusercontent.com/MarkZ3/clang-tools-extra/79b0506f8afecb806a5acb763849147c1585df78/clangd/index/README>


<https://raw.githubusercontent.com/MarkZ3/clang-tools-extra/79b0506f8afecb806a5acb763849147c1585df78/clangd/index/README>

<https://raw.githubusercontent.com/MarkZ3/clang-tools-extra/IndexFunctionsPrototype/clangd/index/README>https://raw.githubusercontent.com/MarkZ3/clang-tools-extra/IndexFunctionsPrototype/clangd/index/README


I have also done some experimentation with LMDB and its been pretty positive, although I am not sure about the possibility of adding it as a dependency as of now.


Regards,

Marc-André<https://raw.githubusercontent.com/MarkZ3/clang-tools-extra/79b0506f8afecb806a5acb763849147c1585df78/clangd/index/README>


________________________________
From: Marc-André Laperle <marc-andre.laperle at ericsson.com>
Sent: Friday, September 8, 2017 4:39 PM
To: Ilya Biryukov
Cc: via cfe-dev; zeratul976 at hotmail.com; Doug Schaefer
Subject: Re: [cfe-dev] Adding indexing support to Clangd



On 2017-08-10 05:56 AM, Ilya Biryukov wrote:
One thing that is really important for us at Google is to come up with the index interface (for both querying and building an index) that would allow an alternative implementation that could scale for larger codebases.
That should not require significant changes to your design, merely extracting a few interface classes and figuring out the APIs should be easy.
We could figure it all out during the review process and I would highly encourage to start a review as early as possible.


It's good to know that you are interested in replacing the implementation. The higher level classes are not very polished right now so as I improve things, I will keep that in mind. For example, right now, the data storage component is not fully hidden from the indexing interface.

BTW, I started putting the indexing prototype on Github [1]. The lower level "storage" parts (ClangdIndexDataStorage, BTree) I think could be ready for review soon but with the discussions about Sqlite and LMDB, I'm not sure if that's the way to go yet. I'll have to research more LMDB especially and also in the light of the index-while-building proposal.
The file event handling, indexing operations and model are all prototype level so not ready for review yet (lots of TODOs and *long* methods) but they help to know how the pieces can fit together. The index-while-building proposal contains some good ideas on how to model things too and I think it makes sense to align on that. There are also some protocol parts in the prototype that could also be reviewed soon, if it's OK for some handlers to not do anything for a while.
I'll determine what the next steps should be and keep you posted.

[1] https://github.com/MarkZ3/clang-tools-extra/commits/IndexFunctionsPrototype

Regards,
Marc-André

We could add the index modification APIs during the review process as well.
The lack of header caching in the first implementation is fine. That's something we should iterate on later. That's a hard problem and it seems fine if we solve it separately.

Wish you a great vacation!

--
Regards,
Ilya Biryukov

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170922/490828d6/attachment.html>