[cfe-dev] RFC: Supporting clang-cl /MP
Alexandre Ganea via cfe-dev
cfe-dev at lists.llvm.org
Thu Apr 25 07:34:43 PDT 2019
With the recent support for Clang in Visual Studio 2019 , I thought we should revive and eventually land the /MP proposal I made a while ago .
As it stands now, building with clang-cl is much slower than MSVC when used along MSBuild, see timings in summary .
With this patch, the situation is greatly improved, clang-cl /MP performance is better in all cases .
When the MP option is active (default behavior) in a Visual Studio Project , MSBuild issues commands such as:
clang-cl /MP file-A.cpp file-B.cpp file-C.cpp
This creates a root clang-cl process, and then *sequentially* a cc1 sub-process for each CPP file in the list. Evidently this is slow and doesn't use the full potential on modern multi-cores machines.
A workaround is to increase the number of parallel builds , but that is sub-optimal and is still slower than MSVC.
The /MP proposal I made in  simply maintains a list of cc1 sub-processes for each hardware core (or for the number of cores provided as an option to /MP).
Waiting for termination in the root clang-cl process is done through a single OS primitive (see sys::WaitMany), which temporarily suspends the root process until at least one cc1 sub-process finishes. This ensure no cycles are wasted waiting, and avoids useless context-switches to the root process.
Most of the changes in the demo patch in  are: modernization of the process launching API (ProcessInfo), and support for sys::WaitMany, which I'd like to commit separately and incrementally.
Support for /MP is very small and mostly located in clang/trunk/lib/Driver/Compilation.cpp in the demo patch.
We could subsequently add a new flag -j to the regular clang driver, which mimics /MP. Further down the road, I'd like to discuss optional support for concrt multithreading which, from some preliminary testing, would be much faster than the current cc1 sub-process invocation, at least on Windows 10 1703+.
Feel free to comment here or in the demo patch.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the cfe-dev