[cfe-dev] distributed clang patch

[Ted Kremenek]

On Jul 10, 2008, at 5:36 PM, Csaba Hruska wrote:

>  My main problem is that I cant imagine how we can support external  
> compilers without messing up the current design and without  
> reimplement the original distcc.

Having the option to use other compilers would obviously be only for a  
subset of the ASTConsumers.  For example, for the ASTConsumers that  
implement compilation, -fsyntax-only, etc., a fork and exec instead of  
running the equivalent Clang consumer seems like a reasonable solution.

> Distributed clang server's goal is support compilation natively  
> without executing external programs.

Don't conflate implementation design with project goals.  Performance  
and stability are the goals.  How this is accomplished can very well  
involve executing external programs.  Obviously we believe that much  
of the performance win will come from *not* doing this, but again  
supporting other compilers makes the system far more testable and  
encourages more adoption.

> Or if you think to support the client side, (new clang distcc client  
> with original distcc server case) then we have to reimplement  
> distcc's protocol, including ssl support.

I see no reason to be compatible with the original distcc server or  
its protocol.  I'm not fluent in the distributed computing design used  
by distcc, but I imagine there are more advanced techniques  
available.  Good support for load balancing, efficient use of the  
network, etc., are all things that we want to be able to tackle.

My goal is not to re-implement the current distcc, it's to build a  
better one that takes advantage of the features of Clang (be it the  
integrated preprocessor, its compilation features, serialization of  
ASTs, or whatever).  I'm a big believer in Clang's future, and having  
unrivaled distributed compilation performance is my hope hope for a  
clang-distcc.  I believe that having limited support to do compilation  
with other compilers is complementary to this goal.

> In this case its far much easy to patch the original distcc client  
> to include and use clang's preprocessor natively.
> I'd like to focus on clang specific stuff.

Absolutely.

> Ex: I can imagine an AST mege function what can support precompiled  
> headers, or can be used to store the already parsed and semachecked  
> headers and reuse it later from a Database.

I see these as advanced features.  Don't get me wrong; I'm excited  
about them too.

What I'm talking about is getting a good basic distcc working first.   
What has been implemented right now is a good start towards this goal,  
but the overall design needs to be more modular (e.g., using a library  
for ASTConsumers, etc.), the distributed computing core can go a lot  
further without worrying about things like PCH, etc.  Using the clang  
preprocessor even when the employed compiler is not Clang is a good  
incremental step that requires much of the  boilerplate stuff to be  
working, and allows experimentation with different performance knobs.   
I don't think what I'm talking about will take years of work; I just  
think it's the first logical step.