[cfe-dev] Performance degradation

[Daniel Dunbar]

I recently was going to update the performance timing page on the web
site to reflect the improvements in IRgen time and noticed that there
has been a substantial decrease in performance in other places since I
put the page up about 3 weeks ago. This encouraged me to start
building parts of what I hope will eventually become a general
infrastructure for monitoring performance over time.

One preliminary sample of the results is here:
  http://t1.minormatter.com/~ddunbar/utime-syntax-rev.pdf
This shows the user time to run -fsyntax-only over the Sketch
Objective-C app. The X axis is svn revision and the Y axis is time.
The red line tracks the minimum user time over all samples per
revision. The green line tracks the mean of the bottom 2/3 of samples;
this gives an indication of the reliability of the timing while still
discarding outliers. The number of samples per revision is adaptive,
and the resolution near TOT is low at the moment.

The high bit is that although I gave our lexer a nudge back in the
right direction, clang at -fsyntax-only has still slowed down
recently. In my timings for TOT versus r57900:
(1) clang -parse-noop of Cocoa.h has slowed down by ~4.1%.
(2) clang -fsyntax-only of Cocoa.h has slowed down by ~6.3%.

Some of this is expected, as we gain features and improve correctness,
but I imagine there are a number of other places where the performance
loss is unexpected and can be recovered.

This email is meant just to serve as a "heads up". I will hopefully
get some more clues about where other regressions occurred as more
data comes in and will fix / file / ask for help as appropriate.

 - Daniel