[cfe-dev] Performance degradation

[Daniel Dunbar]

Thanks for the link, I hadn't seen that yet. In fact I do have a
reference type run in a way because I also collect data on the stable
gcc version, and in a fully automated system I would have a few more
reference runs intended to check the current base performance of the
system (i.e., general performance tests not tests with a reference
version). However I do not intend to add a reference run in the same
vein as they are doing with chromium. This inflates the testing times
significantly and has other problems with maintenance.

Conceptually I think a better approach is to (a) try hard to gather
consistent data and (b) be able to estimate the reliability of the
data (this is not the same as having a reference run). In the end I
*want* consistent data; it is better to spend the extra testing time
getting accurate samples or getting more samples so that you end up
with consistent data (IMHO, of course).

The Web 2.0 style viewing infrastructure, on the other hand, is
something I would love to steal. :)

 - Daniel

On Thu, Nov 20, 2008 at 3:52 AM, Jean-Daniel Dupas
<devlists at shadowlab.org> wrote:
> Le 20 nov. 08 à 11:36, Daniel Dunbar a écrit :
>
> I recently was going to update the performance timing page on the web
> site to reflect the improvements in IRgen time and noticed that there
> has been a substantial decrease in performance in other places since I
> put the page up about 3 weeks ago. This encouraged me to start
> building parts of what I hope will eventually become a general
> infrastructure for monitoring performance over time.
>
> I saw an interesting idea in perf test automation on the Chromium blog
> http://blog.chromium.org/2008/11/putting-it-to-test.html
> Adding a reference build to discount variation in the test conditions. Maybe
> you can add one for clang test too (if you think it's worth).
>
>