[cfe-dev] Designing a clang incremental compiler server

[Reid Kleckner]

Hello cfe-dev,

I'm interning at Google for the summer, and the focus of my internship
is to improve C++ build times by caching the results of parsing
headers and persisting the results either on disk or in the memory of
a persistent compiler server process.

Pre-compiled headers already do this, but they generally only work for
projects that use the '#include "world.h"' header style.  My goal is
to either improve the existing pre-compiled header support so that
this restriction can be lifted, or do something else entirely.

I worked with the LLVM JIT last summer, but I'm unfamiliar with
clang's internals.  Therefore, I thought I would throw some starting
questions out to the clang community.

First and foremost, are people interested in this?  I'm pretty sure
the answer is going to be yes, but I thought I should ask.  :)

Does anyone have any design advice?  I'd like to hear from people who
worked on PCH about how to make this work.  It requires doing things
like making sure all symbols that were not macros on the first include
are still not macros when the header is re-included, and some other
stuff with operator overloading that I'm not familiar with yet.

Where should I do this development?  Depending on the invasiveness of
the changes I'm making, I think it would be best to either work on
trunk or in a branch on the public svn repo.

There's also this previous work on GCC that was done by Per Bothner at Apple:
ftp://gcc.gnu.org/pub/gcc/summit/2003/GCC%20Compile%20Server.pdf

It's from 2003.  I can't find my info about it on the web, so
presumably the effort was abandoned.  I also got the impression that
the implementation was somewhat fragile from the undo buffers that
were used to deal with unwanted AST mutations during compilation.  Can
anyone at Apple familiar with the effort share some advice that might
also apply to clang?

Cheers,
Reid