<div dir="ltr">When I re-ran the TSVC tests 100 times on a quieter machine, all the differences were less than 2%.</div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 21, 2016 at 12:43 PM, Kyle Butt <span dir="ltr"><<a href="mailto:kyle+llvm@iteratee.net" target="_blank">kyle+llvm@iteratee.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">iteratee added a comment.<br>
<br>
Key: for each run avg, median, stddev (stddev as a percent), 10th percentile.<br>
%Change: avg, median, 10th percentile. %Change is change in runtime, so negative percent is an improvement.<br>
<br>
Unchanged is listed first.<br>
<br>
I'm only showing the tests with a difference of more than 2 percent.<br>
I ran each of these benchmarks 50 times.<br>
<br>
The only significant results are in TSVC. I'll see how many benchmarks that is and maybe run them more times.<br>
<br>
test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test  7.014  7.012  0.290  (4.130%)    6.672<br>
test-suite :: MultiSource/Benchmarks/7zip/7zip-benchmark.test  7.021  7.011  0.199  (2.835%)    6.807<br>
% Change: -0.016 %Â Â Â -0.016 %Â Â Â Â 2.013 %<br>
test-suite :: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test    0.924  0.926  0.054  (5.828%)    0.857<br>
test-suite :: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test    0.914  0.907  0.042  (4.616%)    0.858<br>
% Change: -1.993 %Â Â Â -1.993 %Â Â Â Â 0.163 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Equivalencing-dbl/Equivalencing-dbl.test   1.976  1.964  0.109  (5.525%)    1.840<br>
test-suite :: MultiSource/Benchmarks/TSVC/Equivalencing-dbl/Equivalencing-dbl.test   2.000  1.959  0.122  (6.075%)    1.882<br>
% Change: -0.242 %Â Â Â -0.242 %Â Â Â Â 2.288 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Equivalencing-flt/Equivalencing-flt.test   1.246  1.233  0.091  (7.321%)    1.144<br>
test-suite :: MultiSource/Benchmarks/TSVC/Equivalencing-flt/Equivalencing-flt.test   1.239  1.219  0.066  (5.311%)    1.169<br>
% Change: -1.172 %Â Â Â -1.172 %Â Â Â Â 2.211 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Expansion-flt/Expansion-flt.test   1.745  1.751  0.096  (5.491%)    1.628<br>
test-suite :: MultiSource/Benchmarks/TSVC/Expansion-flt/Expansion-flt.test   1.732  1.706  0.095  (5.460%)    1.630<br>
% Change: -2.601 %Â Â Â -2.601 %Â Â Â Â 0.086 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/GlobalDataFlow-dbl/GlobalDataFlow-dbl.test  3.115  3.097  0.110  (3.546%)    2.982<br>
test-suite :: MultiSource/Benchmarks/TSVC/GlobalDataFlow-dbl/GlobalDataFlow-dbl.test  3.190  3.171  0.124  (3.872%)    3.048<br>
% Change: 2.392 %Â Â Â Â 2.392 % 2.220 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/InductionVariable-dbl/InductionVariable-dbl.test   3.691  3.673  0.177  (4.787%)    3.512<br>
test-suite :: MultiSource/Benchmarks/TSVC/InductionVariable-dbl/InductionVariable-dbl.test   3.685  3.667  0.181  (4.925%)    3.442<br>
% Change: -0.162 %Â Â Â -0.162 %Â Â Â Â -1.988 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/LinearDependence-dbl/LinearDependence-dbl.test    3.191  3.140  0.193  (6.044%)    2.978<br>
test-suite :: MultiSource/Benchmarks/TSVC/LinearDependence-dbl/LinearDependence-dbl.test    3.233  3.236  0.159  (4.908%)    3.051<br>
% Change: 3.070 %Â Â Â Â 3.070 % 2.478 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/LoopRerolling-dbl/LoopRerolling-dbl.test   3.557  3.524  0.166  (4.669%)    3.378<br>
test-suite :: MultiSource/Benchmarks/TSVC/LoopRerolling-dbl/LoopRerolling-dbl.test   3.577  3.556  0.118  (3.286%)    3.458<br>
% Change: 0.899 %Â Â Â Â 0.899 % 2.374 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/LoopRestructuring-flt/LoopRestructuring-flt.test   3.480  3.457  0.161  (4.639%)    3.292<br>
test-suite :: MultiSource/Benchmarks/TSVC/LoopRestructuring-flt/LoopRestructuring-flt.test   3.520  3.493  0.128  (3.642%)    3.382<br>
% Change: 1.053 %Â Â Â Â 1.053 % 2.721 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/NodeSplitting-dbl/NodeSplitting-dbl.test   3.411  3.379  0.176  (5.147%)    3.211<br>
test-suite :: MultiSource/Benchmarks/TSVC/NodeSplitting-dbl/NodeSplitting-dbl.test   3.469  3.459  0.168  (4.856%)    3.275<br>
% Change: 2.372 %Â Â Â Â 2.372 % 2.009 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt.test 2.399  2.354  0.156  (6.496%)    2.228<br>
test-suite :: MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt.test 2.415  2.415  0.099  (4.110%)    2.266<br>
% Change: 2.594 %Â Â Â Â 2.594 % 1.705 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Recurrences-flt/Recurrences-flt.test 3.480  3.449  0.142  (4.087%)    3.349<br>
test-suite :: MultiSource/Benchmarks/TSVC/Recurrences-flt/Recurrences-flt.test 3.409  3.412  0.110  (3.237%)    3.266<br>
% Change: -1.089 %Â Â Â -1.089 %Â Â Â Â -2.467 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Searching-dbl/Searching-dbl.test   2.778  2.749  0.137  (4.941%)    2.632<br>
test-suite :: MultiSource/Benchmarks/TSVC/Searching-dbl/Searching-dbl.test   2.825  2.824  0.135  (4.784%)    2.648<br>
% Change: 2.715 %Â Â Â Â 2.715 % 0.581 %<br>
test-suite :: MultiSource/Benchmarks/TSVC/Symbolics-flt/Symbolics-flt.test   1.002  1.002  0.077  (7.686%)    0.919<br>
test-suite :: MultiSource/Benchmarks/TSVC/Symbolics-flt/Symbolics-flt.test   1.041  1.043  0.069  (6.591%)    0.962<br>
% Change: 4.099 %Â Â Â Â 4.099 % 4.579 %<br>
<br>
<br>
<a href="http://reviews.llvm.org/D18223" rel="noreferrer" target="_blank">http://reviews.llvm.org/D18223</a><br>
<br>
<br>
<br>
</blockquote></div><br></div>