[cfe-dev] parsing without building an AST tree

[Manuel Klimek]

On Thu, Jul 19, 2012 at 5:14 AM, Anders Bakken <agbakken at gmail.com> wrote:
> Hi Manuel
>
> Well. We essentially provide a client/server setup where an editor can
> pass a location (file,offset) to the server and some options and the
> server can respond with various information. Most importantly
> references to this location (from all the files we've indexed) and
> whatever it refers to. This is to be able to do 21st century things
> like "follow symbol" and "find references" in Emacs since I'll never
> switch to an IDE. We need to be able to visit cursors I and ask them
> what they reference I guess. Not sure if this would be possible with
> the preprocess-only option. Likely not I guess. If you could point me
> at an example on how to do the preprocessing only I'd love to have a
> look.

I think for your use case you really need the fully type-resolved AST.
This also means that there is no faster way to do it than to parse the
C++ code. The way you can save time is by doing aggressive in-memory
caching of processed parts of the file, which is one thing Chandler is
planning to work on (we call that "clangd" for Clang daemon).

You can take a look at:
http://clang.llvm.org/docs/Tooling.html
to see the various possibilities you currently have to integrate with
clang here.

Cheers,
/Manuel

>
> If you want to take a look at the project it can be found here:
>
> https://github.com/Andersbakken/rtags
>
> thanks
>
> On Tue, Jul 17, 2012 at 2:11 AM, Manuel Klimek <klimek at google.com> wrote:
>> On Tue, Jul 17, 2012 at 10:07 AM, Anders Bakken <agbakken at gmail.com> wrote:
>>> Hi
>>>
>>> We're writing a clang-based tagger and while trying to improve the
>>> performance of our solution we came upon this paragraph:
>>
>> Not sure what your requirements for a "tagger" are, would be curious :)
>>
>>> "Elsa is not built as a stack of reusable libraries like clang is. It
>>> is very difficult to use part of Elsa without the whole front-end. For
>>> example, you cannot use Elsa to parse C/ObjC code without building an
>>> AST. You can do this in Clang and it is much faster than building an
>>> AST."
>>>
>>> from here: http://clang.llvm.org/comparison.html
>>>
>>> We've been using the C-api in clang-c/Index.h but if we could get
>>> better performance by using the C++ APIs directly we'd gladly do so
>>> (even if it might change or be harder to use).
>>>
>>> Is there an example or some documentation on how to do this somewhere possibly?
>>
>> You can use the clang preprocessor to tokenize if that's all you need.
>> Currently there's not really good docs around that, and I don't think
>> I have a really good example. I can get you some more ideas on how to
>> go about this if you say that preprocessor-only is what you need.
>>
>> Cheers,
>> /Manuel