[cfe-dev] Separate preprocess and compile: hack or feature?

[Boris Kolpackov via cfe-dev]

Hi Nico,

Nico Weber <thakis at chromium.org> writes:

> Most distributed build systems I know about end up writing their own custom
> preprocessor to very quickly discover which .h files are included by a cc
> file (you don't need a full preprocessor for getting just that, and so you
> can be faster than clang -E), and then send .h and .cc files to the server
> based on content hashes, so that you don't need to send the full
> preprocessed text, but can send source files before preprocessing.
> https://github.com/facebookarchive/warp was a somewhat recent example of
> this (but also, as you say, pump mode, and proprietary systems).

One property that these build systems rely on is a very controlled
environment (e.g., single compiler, all hosts have exactly the same
headers, etc). I would much rather trade some speed for using standard
and robust tooling.

Also, I saw it mentioned (I think in the pump's documentation) that
local preprocessing is a lot less of an issue on modern hardware. I
bet SSDs made quite a difference.


> Your thread mentions that you do this for -M / /showIncludes, but
> you can just do this as part of regular compilation – not sure why
> you need this in a separate process?

We do it this way to handle auto-generated headers.


Thanks for the feedback,
Boris