[cfe-dev] [RFC] Embedding compilation database info in object files.

[David Chisnall]

On 19 Jul 2013, at 09:08, Manuel Klimek <klimek at google.com> wrote:

> Well, for example, we can spawn a process that writes to the compilation db in the background from the driver, and if we assume that C++ compilations are long compared to writing a line of text to a file, we get that:

So now we get the extra overhead of another fork / exec.  Lots more TLB churn, which is often a limiting factor on scalability on modern CPUs.

> a) finishing off the build will be independent of finishing writing to the compilation db, thus, the impact on the build is negligible

Other than having twice as many processes running...

> b) the compilation db will probably be finished writing before the build finishes (as each step probably finishes writing to the compilation db before the step finishes)

Yes, the 'probably' bit is fun here.  If your build system is doing the 'run the tool' step as a dependency of the build step, you end up with this sequence (for -j n):

n compilation tasks finish.
Build system starts tool.
Tool acquires lock on compilation database and runs.
n child processes of the compile sit waiting for the lock.
Tool finishes.
n child processes sequentially update compilation database

Even better, some of them will probably succeed, so not only do you have the wrong data in the compilation database, you have inconsistent and wrong data in the compilation database.  

David