[cfe-dev] [RFC] A C++ pseudo parser for tooling

[Sam McCall via cfe-dev]

On Tue, Nov 9, 2021 at 6:41 PM Demi Marie Obenour <demiobenour at gmail.com>
wrote:

> On 11/9/21 12:09 PM, Sam McCall wrote:
> > On Tue, Nov 9, 2021 at 4:38 PM Demi Marie Obenour via cfe-dev <
> > cfe-dev at lists.llvm.org> wrote:
> >
> >>>> * I think you are better off spending your time on optimizing the
> >> correct
> >>>> parser infrastructure.  I'm sure more can be done - particularly in
> >> terms
> >>>> of caching, persisting and resusing state (think like PCH and modules
> >> etc).
> >>>>
> >>> We have worked on projects over several years to improve these things
> >> (and
> >>> other aspects such as error-resilience). We agree there's more that can
> >> be
> >>> done, and will continue to work on this. We don't believe this approach
> >>> will get anywhere near a 100x latency improvement, which is what we're
> >>> looking for.
> >>
> >> What about pushing the state to a server?  Have a server that has the
> >> entire
> >> index, and keeps it up to date whenever a VCS commit is made to the main
> >> branch.
> >
> > We have this for clangd's index:
> https://clangd.llvm.org/guides/remote-index
> > It works great (try it out with LLVM!) but needing to deploy a server
> means
> > 90% of users won't ever touch it.
> >
> > (A shared repository of serialized ASTs *is* something we're considering
> in
> > a tightly controlled corp environment but the barriers are pretty huge:
> > size, security and it only works if everyone uses the same precise
> version
> > of the tool. And it only makes sense at all if you're sure you can
> download
> > 300MB in less than 30 seconds!)
> Indeed that is not going to be useful outside of an on-premises corporate
> environment with extremely fast network connectivity.  And the security
> considerations are stringent, especially since clangd is written in C++ and
> I am not sure it can be trusted with untrusted input.
>
> Could an index be persisted to disk, reloaded at startup

(Clangd does have this too, FWIW)


> and incrementally changed as the user edits?

User edits mean we need to parse new code.
Absent a pseudoparser, the only thing that parses code is clang.
Clang needs a full AST, so this step is slow.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20211109/47af0103/attachment.html>