[cfe-dev] [GSoC] Remote-compiling programs with Clang

[Denis Steckelmacher]

>Hi,

>>
>>For many years, Google has organized its famous Google Summer of Code program, sponsoring students to work on their favorite Free Software project during the summer. This year is not an exception and Google has announced its GSoC program a few weeks ago.
>>
>>In 2011, I had the chance to work on an OpenCL implementation used in Mesa3D and now known as Clover (my work concentrated on a software implementation and has now been largely rewritten to be able to use hardware acceleration when possible). This experience allowed me to use Clang as a library, and it did exactly what I wanted. Even though Clang was not the main part of my work, I read some documentation and learned how it is architectured at a high level.
>>
>>Since then, my Computer Science studies took all my time and I wasn't able to work on any public Free Software. My spare time was dedicated to pet projects and experimentation. C++ being my favorite programming language, I used it extensively for the past two years. My computer having a very slow 2x1.6 Ghz E-350 processor (and I have no other computer), I was able to see how C++ can be slow to compile.
>>
>>This made me dreaming of a fast way to compile C++. I did a bit of research and have found interesting things, mainly the well-known distcc and a LLVM presentation talking about using "modules" instead of includes to fasten C++ compilation. This exposes two means to fasten big C++ compilation :
>>
>>* Delegating the compilation to a more powerful machine (this delegation must be fast to be worth using)
>>* Avoiding to re-parse and re-compile files that don't change
>>
>>The first point is already well covered by distcc, except that distcc has some limitations that limit its use (for instance, it preprocesses the code on the local computer and then sends a huge amount of data on the network, thereby limiting the use of more than 2 or 3 remote computers). You can ask distcc not to preprocess the code on the local machine, but the locale machine and the remote one then need to run the exact same operating system and have all the include files at the same location.
>>
>>The second point was partly addressed by the presentation about modules : store information about the public interface of modules in a compiler-friendly form and avoid parsing thousands of header files.
>
>There is more about modules that keeping the public interface in a compiler-friendly form. The thing is, a module is (normally) completely independent; contrary to a header. If you include two headers A and B, then depending on which header goes first, the second is processed differently (because you already got some of the common dependencies, because A defines a macro that influences B parsing, because A defines a template class that may influence B parsing, ...); this does not happen with modules (at least, not with other languages), and therefore since each module is "stand-alone", instead of compiling a module each time it's included somewhere you only recompile it if itself or one of its dependencies changed.
>
>The implementation of the modules is already ongoing (I've seen Douglas Gregor and Argyrios K[...] working on this), but it's definitely not ready for prime time. I don't know of any deadline, either.
>
>Since modules are the way forward, and since they will radically change the behavior of compilation, do you really want to invest on a remote-compiler scheme now ?
>
>-- Matthieu

Hi,

First of all, I would like to thank everyone who have responded to my message for their good remarks, and particularly for their pointers to distcc-like tools. I will take a look at them.

The difference between modules and simple order-dependent include files interests me greatly. Where can I find more information about the development status of modules ? Is there already any repository where I can find some code to play with ?

Denis Steckelmacher.