[cfe-dev] Separate preprocess and compile: some performance numbers

[Boris Kolpackov via cfe-dev]

Nico Weber <thakis at chromium.org> writes:

> Did you just build once for each number? If so, disk cache effects will
> make your measurements unreliable. I recommend doing 3 consecutive builds
> for each number and then recording the smallest of these 3 numbers.

I did 4 consecutive builds discarding the first one as well as re-doing
any significant outliers. Then averaging the last three.

Interestingly, in all the cases except the straight compilation (the first
column), all the measurements were pretty close without any outliers. In
the straight compilation case I had a significant (10% or more) outlier
for each compiler. If I didn't discard them, the other numbers would have
looked even better.

Boris