[PATCH] D52193: RFC: [clang] Multithreaded compilation support

[Alexandre Ganea via Phabricator via cfe-commits]

aganea added a comment.

In https://reviews.llvm.org/D52193#1243456, @thakis wrote:

> ...and to reword this a bit: Clang taking a long time to start up in some configurations is a bug we should profile and fix :-)


This is time spent in `ntdll.dll` loading various low-level libraries like `kernel32.dll`, `KernelBase.dll`, `combase.dll` and so on. So at least 20ms just getting to main(). Then, about ~10ms spent loading `dbghelp.dll` while installing the exception handler in `llvm::RegisterHandler`. So a lot of wasted time. The easiest would be to just not invoke a new child process!

----

New timings without the child `clang-cl.exe` being invoked (hacked from https://reviews.llvm.org/D52411). The improvement **is significant**. Tested at r343846.

**Config 1:** Intel Xeon Haswell 6 cores / 12 HW threads, 3.5 GHz, 15M cache, 128 GB RAM, SSD 550 MB/s

__Ninja:__

| MSVC cl                                  | (37min 19sec)                             |
| clang-cl //(built with Clang)//          | //(in progress-will update a bit later)// |
| clang-cl no child //(built with Clang)// | //(in progress-will update a bit later)// |
|

**Config 2:** Intel Xeon Skylake 18 cores / 36 HW threads, x2 (Dual CPU), 72 HW threads total, 2.3 GHz, 24.75M cache, 128 GB RAM, NVMe SSD 4.6 GB/s

__Ninja:__

| MSVC cl                                      | (7min 33sec)    |
| clang-cl //(built with Clang)//              | (9min 2sec)     |
| **clang-cl no child //(built with Clang)//** | **(7min 9sec)** |
|

This asks whether the improvement will be of the same order, if invoking just one `clang-cl.exe` for the whole compilation process. A sort of local compilation-as-a-service.
In that scenario, Ninja could invoke `clang-cl.exe` and pass it all the files to be compiled, and let clang iterate and multi-thread internally. That could be quite a significant change, however the improvements //could be// important.
However that would probably break the concept behind //Goma //I suppose. Unless you split the invocations to precisely one `clang-cl` per available (remote) agent.


Repository:
  rC Clang

https://reviews.llvm.org/D52193