[cfe-dev] Distcc implementation details

Chris Lattner clattner at apple.com
Sun Mar 14 09:49:16 PDT 2010


On Mar 14, 2010, at 12:03 AM, Mike Miller wrote:
> Hi cfe-dev,
> 
> I'm planning to (attempt to) add in native distcc support to clang(per Chris Lattner's suggestion). The feature would work like this:

This sounds great to me!

> 3. If the socket connection succeeds, clang would send over the command sent by the user to the process, which would preprocess the source, and send it out to a "slave" node along with all compiler options(is it worth performing basic compression on the preprocessed source beforehand?).

It's hard to say and will probably depend on the configuration.  I'd start with not doing it and then experiment with adding it later.

> After the fork(), the parent clang process would attempt to reconnect to the server.

The forked server should beware of the potenial race condition when multiple clang's all start up at the same time, all notice there is no server process, and all fork themselves.  Only one should win.

> On the slave nodes, there would be multiple listening 'clangd' processes(presumably one per core, but this could be configurable) which would all be listening starting at some port(maybe 63000 for core 1, 63001 for core 2, etc.). The slave nodes would do everything from preprocessing to codegen - this means for the first version LTO wouldn't be supported.

Right, eventually it could just be one multithreaded server once clang and llvm are thread safe enough.

> In terms of actual implementation, I'm very new to the clang codebase, and have very little familiarity with it. After poking around a bit, it seems like a good place for my new code to be called from would be "Compilation *Driver::BuildCompilation(int argc, const char **argv)" in Driver.cpp. Does this seem like a good place to "inject" distcc code?

Daniel is the right one to answer this,

> Before I go off and (attempt to) implement this, I'd like to ask for any feedback about the implementation I've proposed. Am I doing something the "wrong way"? Would you do anything differently? Please, let me know any concerns you have - I'd hate to implement this and find out I did it the "wrong way" and have to re-implement all over again!

Sounds great to me!

-Chris



More information about the cfe-dev mailing list