[cfe-dev] [RFC] A C++ pseudo parser for tooling

[Haojian Wu via cfe-dev]

On Wed, Nov 10, 2021 at 8:14 AM David Blaikie via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> On Tue, Nov 9, 2021 at 8:38 PM Andrew Tomazos via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> On Tue, Nov 9, 2021 at 7:28 PM Sam McCall via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> On Tue, Nov 9, 2021 at 7:30 PM Demi Marie Obenour <demiobenour at gmail.com>
>>> wrote:
>>>
>>>> On 11/9/21 1:03 PM, Sam McCall wrote:
>>>> >> Could an index be persisted to disk, reloaded at startup
>>>> >
>>>> > (Clangd does have this too, FWIW)
>>>> >
>>>> >
>>>> >> and incrementally changed as the user edits?
>>>> >
>>>> > User edits mean we need to parse new code.
>>>> > Absent a pseudoparser, the only thing that parses code is clang.
>>>> > Clang needs a full AST, so this step is slow.
>>>>
>>>> What I meant is that clang could reparse only the code the user has
>>>> edited, rather than reparsing the entire source file from scratch.
>>>> I believe rust-analyzer does exactly this for Rust source, and my
>>>> understanding is that it is considered fast enough.
>>>
>>>
>>
>>> Yes, you could do this, but not with clang as the parser.
>>>
>>
>> If it were modified to efficiently persist the salient parser state
>> (including partial AST) at checkpoints during translation after every few
>> lines (see https://en.wikipedia.org/wiki/Persistent_data_structure), and
>> then able to serialize that history and reload from a checkpoint just above
>> a modification, then it wouldn't have to start from scratch every time.
>> Again, I would spend some time thinking about how pre-compiled headers and
>> modules work.
>>
>
> To be clear, the folks making this proposal are pretty closely connected
> to how PCH and modules work - I think it's fair to assume they've spent
> quite a while thinking about how these things work and could be applied.
>

Yeah, We've been spending years of effort on improving these things (and
will continue) and applying them in clangd -- clangd snapshots the initial
#includes of the main file (which is called preamble), and uses this
preamble to build the AST of the main file (the AST build with an
up-to-date/stale preamble is fast).
However, it doesn't solve all: initial build of preamble can be extremely
slow (tens of seconds, or minutes); cross-file refactorings might want to
edit an unopened file; there are use cases where it is impractical to
have a real compilation environment setup.
These are the areas we aim to improve with a pseudo parser.



> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20211110/1b7b01da/attachment-0001.html>