[cfe-dev] Compile Server?

[Douglas Gregor]

On Jan 16, 2011, at 5:14 PM, David Claughton wrote:

> Hi,
> 
> So, over the Christmas break I spent some time trying to get my head
> around the clang code base.  While looking around for a little project I
> could do to get me started, I happened upon this thread [1] from last
> summer which piqued my interest.
> 
> [1] http://lists.cs.uiuc.edu/pipermail/cfe-dev/2010-June/009473.html
> 
> The thread describes the idea of a compile-server and some ideas of how
> such a thing might be used to improve compile speeds.  However I had a
> poke around the source and cfe-commits and as far as I can tell nothing
> much has happened with the idea.
> 
> Is someone still actively looking at this, or planning on doing so?  If
> not, I wouldn't mind having a go at it myself.

I don't know of anyone planning to work on this.

> Mind you, I had a go this weekend at hacking in some (really ugly)
> socket code just to get a feel for it, and TBH I'm starting to wonder if
> an actual server is the best approach - it seems a tad over-engineered.
> 
> As I understand it, the goal is to present a single instance of clang
> with a list of compile jobs it needs to perform, allowing it to cache
> headers and intermediate results in memory.  IMHO the most obvious way
> of doing this is to simply read a list of job descriptions from a file.

As Reid notes, C/C++ make caching of intermediate results really tricky. However, you can perform some optimizations, e.g., detecting common sequences of prefix headers and automatically generating precompiled headers for them.

> The main (only?) reason of using a server process is because 'make' and
> other build tools do not call the compiler in this way, but call it
> repeatedly for each source file.  However I can't help but think that if
> some sort of "batch mode" was available and it allowed for significantly
> improved compile times, then people would find ways to make the various
> build tools work with it.


I think one of the other goals of a compiler server is to support interactive clients, such as an IDE. So, it's not that we have a list of compile jobs provided in advance: we have requests coming in, typically for some smallish set of files that are constantly changing, and we need to return results quickly.

libclang does some optimization of these cases internally; a compiler server could generalize those optimizations.

	- Doug