[cfe-dev] clang windows: clang-cl support for /MP

[Reid Kleckner]

On Wed, May 20, 2015 at 5:47 AM, Domagoj Saric <
domagoj.saric at littleendian.com> wrote:

> On 20.5.2015. 13:09, Kim Gräsman wrote:
>
>> There are two levels of parallellization in Visual Studio/Visual C++;
>>
>> 1) Machine-wide setting of how many projects to build in parallel --
>> this informs Visual Studio/MSBuild
>> 2) Compiler setting (/MP) of how many source files to compile in
>> parallel -- this actually informs CL.EXE and causes it to launch child
>> processes
>>
>> See Bruce Dawson's treatise for the gory details:
>>
>> https://randomascii.wordpress.com/2014/03/22/make-vc-compiles-fast-through-parallel-compilation/
>>
>> Not sure how easy/desirable this behavior would be in clang-cl, but
>> from a principal standpoint I'd prefer this kind of thing in the build
>> system, not the compiler.
>
>
Given that cl implements /MP by simply launching more cl.exe subprocesses
that can oversaturate your system if MSBuild is building multiple projects
in parallel, I'm not inclined to complicate the driver with parallel
execution. If someone put together a particularly elegant and
low-maintenance burden patch, though, I think we could take it.

If Microsoft really wants to squeeze more parallelism out of VS builds,
they should expose the in-project compilation parallelism to the build
system and let MSBuild handle it.

If individual users want more parallel builds, I have to recommend that
they avoid MSBuild and try to hook VS up to ninja instead.

But from a 'practical standpoint', wouldn't a multithreaded compiler be the
> faster solution? e.g.:
>  - you'd parse precompiled and/or shared headers only once for all the
> sources being compiled simultaneously/in-parallel
>  - multithreaded LTO/LTCG 'for free'
>  - threads are cheaper than processes


When I was an intern, we thought this would be a good idea. We learned,
though, that at least for the frontend, nobody actually wants a
multi-threaded compiler. This would inevitably lead to threading bugs and
non-deterministic diagnostic output and compiler output. You really don't
want to use the AST for your last parse of a.h in your next compile or
you'll end up not emitting diagnostics that you would've otherwise, and
compilations stop being reproducible.

For basic compilation actions, you really want the safety that process
isolation gives you. Windows processes aren't as expensive as people think
they are, and on better optimized OSs they are pretty cheap.

Something like deterministic, multi-threaded LTO would be pretty cool,
though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20150520/9c8b0627/attachment.html>