[cfe-dev] Question about clang distcc

[Matthieu Monrocq]

On Sat, Jun 16, 2012 at 3:19 PM, Lyu Mitnick <mitnick.lyu at gmail.com> wrote:

> Hello Douglas,
>
> I have read all posted carefully. According to the discussion, what we
> can do better than
> original distcc are as follows:
>
> 1) The intermediate files passed over the network would be serialized AST
> 2) The intermediate files passed over the network would be LLVM IR
> 3) Centralized admin daemon
> 4) Use PCH
>
> To improve the issues above. We can extend the original distcc.
> However Chris Lattner
> mentioned the first mile-stone of clang distributed build project is
> re-implementing a new
> distcc.
>
> http://lists.cs.uiuc.edu/pipermail/cfe-dev/2008-April/001357.html
>
> I am wondering to know why and what is the desired design of clang
> distributed build
> project for cfe community.
>
> Mitnick
>
> >
> > No, this project has not been accomplished. I don't think there was any
> real progress on it since that discussion.
> >
> >        - Doug
>
>
We have been using distcc (on top of gcc) for a while at work, and it does
work, somewhat, but it has a big issue: preprocessing is a huge part of
compilation, not being able to get rid of it creates a bottleneck that
inhibits true scalability. Given that we are using 24 cores servers, we
could push to about 40/60 parallel compilations (interesting when an error
occurs in a header and the local machine has to compile those 40/60 files
locally to display the errors); any further and we would not observe any
significant progress in compilation time: the local machine became the
bottleneck.

We experimented for a time with a solution where we streamed the raw
unprocessed files and had a filtering in place to only push "local" files
and have replica on the distcc hosts for the 3rd party headers. It worked
quite well, not much gain but slightly faster... as long as the distcc host
had the up-to-date collection of 3rd party headers & the local directory
hierarchy was similar; had a few issues with it (maintenance) so we fell
back to the traditional distcc.

Honestly, we got much more performance boost from ccache than from distcc.

I am unsure how to work around the local preprocessing issue, and I am
afraid that no significant progress will be made as long as it stands in
the way. I would be glad to hear some folks have ideas to get around it,
these days I put more hope in a "persistent" process that would cache
various stages of the compilation pipeline (maybe using the daemon Chandler
was talking about ?).

-- Matthieu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120616/1294bead/attachment.html>