[cfe-dev] distributed clang patch
kremenek at apple.com
Thu Jul 10 17:54:25 PDT 2008
On Jul 10, 2008, at 5:36 PM, Csaba Hruska wrote:
> My main problem is that I cant imagine how we can support external
> compilers without messing up the current design and without
> reimplement the original distcc.
Having the option to use other compilers would obviously be only for a
subset of the ASTConsumers. For example, for the ASTConsumers that
implement compilation, -fsyntax-only, etc., a fork and exec instead of
running the equivalent Clang consumer seems like a reasonable solution.
> Distributed clang server's goal is support compilation natively
> without executing external programs.
Don't conflate implementation design with project goals. Performance
and stability are the goals. How this is accomplished can very well
involve executing external programs. Obviously we believe that much
of the performance win will come from *not* doing this, but again
supporting other compilers makes the system far more testable and
encourages more adoption.
> Or if you think to support the client side, (new clang distcc client
> with original distcc server case) then we have to reimplement
> distcc's protocol, including ssl support.
I see no reason to be compatible with the original distcc server or
its protocol. I'm not fluent in the distributed computing design used
by distcc, but I imagine there are more advanced techniques
available. Good support for load balancing, efficient use of the
network, etc., are all things that we want to be able to tackle.
My goal is not to re-implement the current distcc, it's to build a
better one that takes advantage of the features of Clang (be it the
integrated preprocessor, its compilation features, serialization of
ASTs, or whatever). I'm a big believer in Clang's future, and having
unrivaled distributed compilation performance is my hope hope for a
clang-distcc. I believe that having limited support to do compilation
with other compilers is complementary to this goal.
> In this case its far much easy to patch the original distcc client
> to include and use clang's preprocessor natively.
> I'd like to focus on clang specific stuff.
> Ex: I can imagine an AST mege function what can support precompiled
> headers, or can be used to store the already parsed and semachecked
> headers and reuse it later from a Database.
I see these as advanced features. Don't get me wrong; I'm excited
about them too.
What I'm talking about is getting a good basic distcc working first.
What has been implemented right now is a good start towards this goal,
but the overall design needs to be more modular (e.g., using a library
for ASTConsumers, etc.), the distributed computing core can go a lot
further without worrying about things like PCH, etc. Using the clang
preprocessor even when the employed compiler is not Clang is a good
incremental step that requires much of the boilerplate stuff to be
working, and allows experimentation with different performance knobs.
I don't think what I'm talking about will take years of work; I just
think it's the first logical step.
More information about the cfe-dev