[cfe-dev] parsing without building an AST tree

Manuel Klimek klimek at google.com
Thu Jul 19 01:18:03 PDT 2012

On Thu, Jul 19, 2012 at 5:14 AM, Anders Bakken <agbakken at gmail.com> wrote:
> Hi Manuel
> Well. We essentially provide a client/server setup where an editor can
> pass a location (file,offset) to the server and some options and the
> server can respond with various information. Most importantly
> references to this location (from all the files we've indexed) and
> whatever it refers to. This is to be able to do 21st century things
> like "follow symbol" and "find references" in Emacs since I'll never
> switch to an IDE. We need to be able to visit cursors I and ask them
> what they reference I guess. Not sure if this would be possible with
> the preprocess-only option. Likely not I guess. If you could point me
> at an example on how to do the preprocessing only I'd love to have a
> look.

I think for your use case you really need the fully type-resolved AST.
This also means that there is no faster way to do it than to parse the
C++ code. The way you can save time is by doing aggressive in-memory
caching of processed parts of the file, which is one thing Chandler is
planning to work on (we call that "clangd" for Clang daemon).

You can take a look at:
to see the various possibilities you currently have to integrate with
clang here.


> If you want to take a look at the project it can be found here:
> https://github.com/Andersbakken/rtags
> thanks
> On Tue, Jul 17, 2012 at 2:11 AM, Manuel Klimek <klimek at google.com> wrote:
>> On Tue, Jul 17, 2012 at 10:07 AM, Anders Bakken <agbakken at gmail.com> wrote:
>>> Hi
>>> We're writing a clang-based tagger and while trying to improve the
>>> performance of our solution we came upon this paragraph:
>> Not sure what your requirements for a "tagger" are, would be curious :)
>>> "Elsa is not built as a stack of reusable libraries like clang is. It
>>> is very difficult to use part of Elsa without the whole front-end. For
>>> example, you cannot use Elsa to parse C/ObjC code without building an
>>> AST. You can do this in Clang and it is much faster than building an
>>> AST."
>>> from here: http://clang.llvm.org/comparison.html
>>> We've been using the C-api in clang-c/Index.h but if we could get
>>> better performance by using the C++ APIs directly we'd gladly do so
>>> (even if it might change or be harder to use).
>>> Is there an example or some documentation on how to do this somewhere possibly?
>> You can use the clang preprocessor to tokenize if that's all you need.
>> Currently there's not really good docs around that, and I don't think
>> I have a really good example. I can get you some more ideas on how to
>> go about this if you say that preprocessor-only is what you need.
>> Cheers,
>> /Manuel

More information about the cfe-dev mailing list