[PATCH] D69825: [Clang][Driver] Bypass cc1 process and re-enter driver main

Alexandre Ganea via Phabricator via cfe-commits cfe-commits at lists.llvm.org
Tue Nov 5 08:10:20 PST 2019


aganea added a comment.

Thanks for the feedback @Meinersbur!

This patch is mainly geared towards Windows users. I'm not expecting anything on Linux, you already have all the bells & whistles there :-) Although I definitely see improvements on my Linux box. Would the distro make a difference? Mine is:

  $ uname -a
  Linux (name edited) 5.0.0-29-generic #31~18.04.1-Ubuntu SMP Thu Sep 12 18:29:21 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux



In D69825#1733254 <https://reviews.llvm.org/D69825#1733254>, @Meinersbur wrote:

> This patch reduced the build time from 12 to 7 minutes? Sounds too good to be true.


Starting up and cooling down a large process like clang on Windows is very expensive. A fair amount of cpu time is spent in the clang driver, essentially in the kernel, in the range of 30 - 100 ms per process:
//(timings below are before this patch)//
F10630335: procmon-clang-cl.PNG <https://reviews.llvm.org/F10630335>

We don't have `fork()` on Windows meaning that the kernel needs to re-start a the cc1 process from scratch, allocate pages, remap the EXE, allocate heap, allocate TLS, start the CRT, etc. Also, `InitLLVM::InitLLVM` is expensive because it calls into `dbghelp.dll` and loads symbols - just that can sometimes take up to 10 ms per process (depending on the system load).
In addition, recent Windows builds have all sorts of kernel regressions related to process creation and virtual pages allocation. Bruce Dawson <https://randomascii.wordpress.com/2019/04/21/on2-in-createprocess/> has several blog entries about all this.
We're simply mitigating these issues. Upgrading the 36-core server to Windows 10 build 1903 will probably decrease the gap (12 min -> 7 min). However I would still expect users of Clang on pre-1903 builds for a few years from now.

I will fix the other issues you've mentioned.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D69825/new/

https://reviews.llvm.org/D69825





More information about the cfe-commits mailing list