Benchmarking some lld revisions

George Rimar via llvm-commits llvm-commits at lists.llvm.org
Sun May 28 23:38:01 PDT 2017


>Now that I have a workstation setup again for doing really stable
>benchmarks I decided to try a quick experiment and benchmark linking
>firefox with the last 1000 revisions or so.
>
>I used the last 1000 revision, regardless of what they changed. That
>way we can get an idea of the fluctuation to expect from measurement
>bias when unrelated things change.
>
>The results are attached. Data has 3 columns. The first one is the
>revision, the second one is the time and the third is the percentage
>std dev reported by perf over 10 runs.

Looks great ! 

>The plot was created with gnuplot:
>
>plot "data" using 1:2:($3 * $2 /100) title "time" with errorlines
>
>It is interesting to see that the variation from commit to commit is
>much bigger than the variation in a single commit, but the graph
>allows us to find some significant commits.

Data in text representation might have one more column: change from
previous revision in percents, that way even with text representation would
be easy to find significant commits probably.

> In this case, they were:
>
>r303689: The test has --buildit ... --build-id=none, and now the
>second one is used.
>
>r303925: Uses CachedHashStringRef again for comdat signatures.
>
>Cheers,
>Rafael
>
>P.S.: It would be truly awesome if someone could setup a bot that does
>this over various tests.

George.


More information about the llvm-commits mailing list