[llvm-dev] Compare test-suite benchmarks performance complied without TBAA, with default TBAA and with new TBAA struct path
Danil Malyshev via llvm-dev
llvm-dev at lists.llvm.org
Thu Apr 26 11:20:35 PDT 2018
Hello,
I was interested in how much Type-Based Alias Analysis helps to optimize code. For that purpose, I've compared
three sets of benchmarks: compiled without TBAA, compiled with a default TBAA metadata format, and compiled
with new TBAA metadata format.
As a set of benchmarks, I've used the LLVM test suite (http://llvm.org/docs/TestingGuide.html#test-suite-overview)
which has a lot of tests already identified as benchmarks. For statistical reliability, I've executed each test
at least 40 times. Many tests have a very short execution time so 40 times wasn't enough. For these, I increased
the number of repetitions. I've used two types of measurements: execution time and a number of CPU instructions
executed in user-mode. The first one is a common approach but its accuracy is not high and results may vary for
the same testing program. The second is exactly what a tested program did and it is very accurate: if a program
doesn't use random values, it will be almost the same amount for each run. But different instructions have
different complexity (for example, moving data from one register to another is much faster than moving from memory
to register) so we still need overall execution time for a complete picture. For details about testing, see
Appendix A.
### Some interesting results
5 tests compiled with new TBAA didn't passed verification (3 of them had segmentation faults). I think this is
because the new TBAA struct path isn't fully supported. In any case, all these tests should be investigated:
|-----------------------------------------------------------------|--------------------|
| Test name | Error |
|-----------------------------------------------------------------|--------------------|
| MultiSource/Applications/ClamAV/clamscan.test | different output |
| MultiSource/Benchmarks/DOE-ProxyApps-C/SimpleMOC/SimpleMOC.test | segmentation fault |
| MultiSource/Benchmarks/FreeBench/distray/distray.test | different output |
| MultiSource/Benchmarks/Olden/bh/bh.test | segmentation fault |
| SingleSource/Benchmarks/Misc-C++-EH/spirit.test | segmentation fault |
|-----------------------------------------------------------------|--------------------|
4 tests compiled without TBAA shows better results than compiled with TBAA. Probably some TBAA optimization passes
blocks more efficiently (as minimum in that cases) optimizations as the process continues. Anyway think these tests
should be investigated also, probably not so much like previous tests:
|------------------------------------------------------------|--------|--------|--------|--------|
|Test name | Execution time |CPU instructions |
| |Diff with No TBAA|Diff with No TBAA|
| |Default | New |Default | New |
| | TBAA,% | TBAA,% | TBAA,% | TBAA,% |
|------------------------------------------------------------|--------|--------|--------|--------|
|MultiSource/Benchmarks/DOE-ProxyApps-C/miniGMG/miniGMG.test | -1.11 | -1.15 | -4.48 | -4.48 |
|MultiSource/Benchmarks/VersaBench/beamformer/beamformer.test| -13.64 | -13.61 | -20.68 | -20.68 |
|MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg.test | -2.21 | -2.45 | -0.51 | -0.51 |
|SingleSource/Benchmarks/Misc-C++/Large/sphereflake.test | -2.45 | -3.45 | -2.41 | -3.45 |
|------------------------------------------------------------|--------|--------|--------|--------|
Typically, the execution time correlated to the number of executed CPU instructions. For the following tests,
however, that was not the case, as some of them have fewer instructions but longer execution time (or vice versa):
|-----------------------------------------------------------------|--------|--------|--------|--------|
|Test name | Execution time |CPU instructions |
| |Diff with No TBAA|Diff with No TBAA|
| |Default | New |Default | New |
| | TBAA,% | TBAA,% | TBAA,% | TBAA,% |
|-----------------------------------------------------------------|--------|--------|--------|--------|
|MultiSource/Benchmarks/7zip/7zip-benchmark.test | 0.51 | 0.20 | -0.74 | -0.74 |
|MultiSource/Benchmarks/DOE-ProxyApps-C++/miniFE/miniFE.test | 0.85 | 0.86 | -0.62 | -0.62 |
|MultiSource/Benchmarks/DOE-ProxyApps-C/Pathfinder/PathFinder.test| 0.73 | 0.82 | -1.09 | -1.09 |
|MultiSource/Benchmarks/FreeBench/pifft/pifft.test | -1.36 | -1.38 | 1.93 | 1.93 |
|MultiSource/Benchmarks/Ptrdist/anagram/anagram.test | 17.81 | 17.89 | -16.35 | -16.35 |
|SingleSource/Benchmarks/Shootout/Shootout-hash.test | -9.30 | -10.31 | 0.10 | 0.55 |
|SingleSource/Benchmarks/Shootout/Shootout-lists.test | -2.97 | -2.97 | 8.82 | 8.82 |
|-----------------------------------------------------------------|--------|--------|--------|--------|
There are many tests where TBAA helps with better optimizations. Several tests from that list have better
optimizations with new TBAA:
|-----------------------------------------------------------------|--------|--------|--------|--------|
|Test name | Execution time |CPU instructions |
| |Diff with No TBAA|Diff with No TBAA|
| |Default | New |Default | New |
| | TBAA,% | TBAA,% | TBAA,% | TBAA,% |
|-----------------------------------------------------------------|--------|--------|--------|--------|
|Bitcode/Benchmarks/Halide/blur/halide_blur.test | 239.61 | 239.62 | 413.65 | 413.65 |
|SingleSource/Benchmarks/Misc/himenobmtxpa.test | 64.58 | 64.97 | 219.74 | 219.74 |
|MultiSource/Benchmarks/TSVC/Equivalencing-flt/Equivalencing-flt.t| 46.74 | 47.04 | 48.01 | 48.01 |
|MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4.test | 41.32 | 41.57 | 54.97 | 54.97 |
|SingleSource/Benchmarks/Dhrystone/dry.test | 20.02 | 20.02 | 11.54 | 11.54 |
|SingleSource/Benchmarks/Dhrystone/fldry.test | 19.96 | 19.95 | 18.52 | 18.52 |
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-matrix.test | 17.43 | 17.47 | 14.73 | 14.73 |
|MultiSource/Benchmarks/MiBench/automotive-susan/automotive-susan.| 16.41 | 16.39 | 0 | 0 |
|MultiSource/Benchmarks/TSVC/Equivalencing-dbl/Equivalencing-dbl.t| 14.71 | 15.09 | 38.99 | 38.99 |
|MultiSource/Benchmarks/DOE-ProxyApps-C/miniAMR/miniAMR.test | 11.73 | 11.75 | 18.20 | 18.20 |
|MultiSource/Benchmarks/FreeBench/neural/neural.test | 7.20 | 7.29 | 8.15 | 8.15 |
|MultiSource/Benchmarks/DOE-ProxyApps-C++/PENNANT/PENNANT.test | 5.78 | 5.91 | 6.35 | 6.35 |
|MultiSource/Benchmarks/McCat/18-imp/imp.test | 5.30 | 5.24 | 2.73 | 2.73 |
|MultiSource/Benchmarks/MiBench/network-dijkstra/network-dijkstra.| 4.41 | 3.85 | 0 | 0 |
|SingleSource/Benchmarks/Misc/oourafft.test | 3.50 | 3.46 | 3.58 | 3.58 |
|MultiSource/Applications/JM/lencod/lencod.test | 2.67 | 2.30 | 3.03 | 3.03 |
|MultiSource/Benchmarks/MallocBench/espresso/espresso.test | 2.14 | 2.89 | 2.30 | 2.30 |
|MultiSource/Benchmarks/Olden/bisort/bisort.test | 2.04 | 2.12 | 3.59 | 3.59 |
|MultiSource/Benchmarks/DOE-ProxyApps-C++/CLAMR/CLAMR.test | 1.98 | 2.03 | 2.66 | 2.66 |
|MultiSource/Benchmarks/sim/sim.test | 1.89 | 1.85 | 2.06 | 2.06 |
|MultiSource/Applications/JM/ldecod/ldecod.test | 1.86 | 1.76 | 3.59 | 3.59 |
|MultiSource/Benchmarks/McCat/08-main/main.test | 1.73 | 1.74 | 3.46 | 3.46 |
|MultiSource/Benchmarks/mafft/pairlocalalign.test | 1.74 | 1.75 | 2.80 | 2.80 |
|SingleSource/Benchmarks/McGill/chomp.test | 1.29 | 3.32 | 8.75 | 8.75 |
|MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset.| 1.18 | 1.06 | 0.87 | 0.87 |
|MultiSource/Applications/sgefa/sgefa.test | 1.11 | 1.22 | 2.61 | 2.61 |
|MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test | 1.06 | 1.22 | 0.72 | 0.72 |
|MultiSource/Benchmarks/VersaBench/dbms/dbms.test | 1.05 | 1.28 | 2.00 | 2.00 |
|MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/HPCCG.test | 0.59 | 0.84 | 1.66 | 1.66 |
|MultiSource/Benchmarks/McCat/05-eks/eks.test | 0.53 | 1.04 | 2.28 | 2.28 |
|MultiSource/Applications/minisat/minisat.test | 0.90 | 0.76 | 1.49 | 1.81 |
|SingleSource/Benchmarks/CoyoteBench/fftbench.test | 0.44 | 0.52 | 1.10 | 1.35 |
|MultiSource/Benchmarks/Bullet/bullet.test | 0.36 | 0.47 | 0.33 | 2.34 |
|MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset.| 0.26 | 0.43 | 0.79 | 0.79 |
|MultiSource/Benchmarks/Olden/mst/mst.test | 0.21 | 0.40 | 0.69 | 0.69 |
|MultiSource/Benchmarks/Fhourstones-3.1/fhourstones3.1.test | 0.21 | 0.23 | 0 | 1.16 |
|MultiSource/Benchmarks/McCat/09-vor/vor.test | 0 | 0.59 | 2.47 | 2.47 |
|-----------------------------------------------------------------|--------|--------|--------|--------|
Full testing results are available in Appendix B.
## Appendix A: Testing details
### Environment and preparation
All testing was made in on a designated desktop computer with an Intel i7-4770 CPU and 16GB memory.
- Operation system: 64bit Ubuntu 16.04.1
- Linux core: 4.13.0-37
- test-suite SVN rev: 328330 2018-03-23 08:58:41 -0700 (Fri, 23 Mar 2018)
- LLVM SVN rev: 329926 2018-04-12 10:01:46 -0700 (Thu, 12 Apr 2018)
- clang SVN rev: 329924 2018-04-12 09:41:55 -0700 (Thu, 12 Apr 2018)
I followed these useful tips: https://www.llvm.org/docs/Benchmarking.html.
I used only a shell, no GUI, no other services, with 2 CPUs isolated from other processes and used only for
testing.
Test-suite source files were moved to a ramdisk (because some tests use some input files from the source
folder), and 3 builds were made in separate folders on the ramdisk with the following parameters:
1. No TBAA, CFLAGS and CXXFLAGS = " -O3 -relaxed-aliasing -static"
2. Default TBAA, CFLAGS and CXXFLAGS = " -O3 -static"
3. New TBAA, CFLAGS and CXXFLAGS = " -O3 -Xclang -new-struct-path-tbaa -static"
The following flag was specified for all the builds: -DTEST_SUITE_BENCHMARKING_ONLY=ON
### Execution
For each *.test file that was found in the first No TBAA folder I did the following:
1. Execute command line described in that file in section "RUN:" with redirect to a file stdout and stderr.
2. Execute verification programs described in sections "VERIFY: " with %o replaced to that file.
3. If verification was successful, the test from RUN section was executed repeatedly at least 40 times (if
the first execution time was less than one second, the number of repeats was 40/execution_time but not
more than 1000) with the following:
- the test was executed in shielded CPUs
- stdout and stderr was redirected to /dev/null
- execution time and the number of CPU instructions executed in user mode were collected by perf utility
4. Steps #2 and #3 was executed for the same test in "default" folder and "new-tbaa" folder.
After testing, the data was reviewed. Some tests with strange results like high differences were re-executed
several times with an extended number of repeats.
## Appendix B: Full testing results
|------------------------------------------------------------------------------|------|-----------|-----------|-----------|------|-----------|------|-----------|------|-----------|------|
| Test name |Passes| No TBAA | Default TBAA | New TBAA |
| | |Exec.Time | CPU | Execution time | CPU instructions | Execution time | CPU instructions |
| | | Seconds | instruct. | Seconds |Diff,%| |Diff,%| Seconds |Diff,%| |Diff,%|
|------------------------------------------------------------------------------|------|-----------|-----------|-----------|------|-----------|------|-----------|------|-----------|------|
|Bitcode/Benchmarks/Halide/bilateral_grid/halide_bilateral_grid.test | 40| 2.56518256|54828688896|2.552624706| 0.49|54828702561| 0|2.565364148| -0.01|54828691567| 0|
|Bitcode/Benchmarks/Halide/blur/halide_blur.test | 40|3.854254085|50498796318| 1.13491866|239.61| 9831357120|413.65|1.134860168|239.62| 9831359440|413.65|
|Bitcode/Benchmarks/Halide/local_laplacian/halide_local_laplacian.test | 40|2.071047626|38767595010|2.072455961| -0.07|38767396810| 0|2.068001419| 0.15|38767469541| 0|
|MultiSource/Applications/ALAC/decode/alacconvert-decode.test | 1000|0.010544763| 91421419|0.010553151| -0.08| 91421603| 0|0.010542988| 0.02| 91421603| 0|
|MultiSource/Applications/ALAC/encode/alacconvert-encode.test | 1000|0.023955411| 212117365|0.023962085| -0.03| 212117343| 0| 0.02398535| -0.12| 212117343| 0|
|MultiSource/Applications/ClamAV/clamscan.test | 161|0.176823794| 415432265| 0.17707974| -0.14| 410749619| 1.14|Wrong output |
|MultiSource/Applications/JM/ldecod/ldecod.test | 1000|0.035156961| 200210199|0.034515149| 1.86| 193262981| 3.59|0.034548374| 1.76| 193262982| 3.59|
|MultiSource/Applications/JM/lencod/lencod.test | 40|3.170994041|25664224656|3.088565998| 2.67|24909283720| 3.03|3.099617378| 2.3|24908920933| 3.03|
|MultiSource/Applications/SIBsim4/SIBsim4.test | 40|1.802025964|13873237620|1.789649065| 0.69|13871828929| 0.01|1.793373993| 0.48|13871828927| 0.01|
|MultiSource/Applications/SPASS/SPASS.test | 40|5.535440186|14013577342| 5.50070333| 0.63|14013577387| 0| 5.53306689| 0.04|14013577394| 0|
|MultiSource/Applications/aha/aha.test | 40|1.185788745| 9912983319| 1.18564303| 0.01| 9912983313| 0|1.184601587| 0.1| 9912983312| 0|
|MultiSource/Applications/d/make_dparser.test | 452|0.014109978| 116717348|0.014038429| 0.51| 116647843| 0.06|0.014039386| 0.5| 116662982| 0.05|
|MultiSource/Applications/hexxagon/hexxagon.test | 40|5.592275346|47824043731|5.605615652| -0.24|47824043204| 0|5.605680946| -0.24|47885303399| -0.13|
|MultiSource/Applications/kimwitu++/kc.test | 359|0.024159656| 72459109|0.024436261| -1.13| 72403695| 0.08|0.024352098| -0.79| 72403696| 0.08|
|MultiSource/Applications/lambda-0.1.3/lambda.test | 40|2.275253213|17103025337|2.274423014| 0.04|17103025924| 0|2.269417403| 0.26|17103026060| 0|
|MultiSource/Applications/minisat/minisat.test | 40|3.876793531|15327323253|3.842179375| 0.9|15102263467| 1.49|3.847563488| 0.76|15054335466| 1.81|
|MultiSource/Applications/obsequi/Obsequi.test | 40|1.128144503| 7768948914|1.130562094| -0.21| 7780133058| -0.14|1.130680714| -0.22| 7780133058| -0.14|
|MultiSource/Applications/oggenc/oggenc.test | 289|0.062699866| 436441370|0.062741241| -0.07| 436441373| 0|0.062885991| -0.3| 436441373| 0|
|MultiSource/Applications/sgefa/sgefa.test | 260|0.081885589| 554398988|0.080988188| 1.11| 540290140| 2.61|0.080895254| 1.22| 540290140| 2.61|
|MultiSource/Applications/siod/siod.test | 200|0.990491371|13171244108|0.996202976| -0.57|13171242336| 0|1.002628679| -1.21|13171242337| 0|
|MultiSource/Applications/spiff/spiff.test | 40|1.200894336| 9998849777|1.201319526| -0.04| 9998849824| 0|1.201652745| -0.06| 9998849822| 0|
|MultiSource/Applications/sqlite3/sqlite3.test | 40| 1.64155538|15773649278|1.640893436| 0.04|15719515133| 0.34|1.622134694| 1.2|15719515187| 0.34|
|MultiSource/Applications/viterbi/viterbi.test | 40|1.268640203|13778693803|1.269116281| -0.04|13778689618| 0|1.270443991| -0.14|13778689617| 0|
|MultiSource/Benchmarks/7zip/7zip-benchmark.test | 40|5.657674099|27688113001|5.629074283| 0.51|27893655301| -0.74|5.646465274| 0.2|27893655288| -0.74|
|MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000.test | 40|1.471894927| 5845153953|1.464805189| 0.48| 5856337655| -0.19|1.465630801| 0.43| 5856337654| -0.19|
|MultiSource/Benchmarks/ASC_Sequoia/AMGmk/AMGmk.test | 40|4.794886583|38013036990|4.579802023| 4.7|38512062691| -1.3|4.577213823| 4.76|38512062692| -1.3|
|MultiSource/Benchmarks/ASC_Sequoia/CrystalMk/CrystalMk.test | 40|2.942015001|26934039129|2.943005397| -0.03|26934039124| 0|2.942780883| -0.03|26934039124| 0|
|MultiSource/Benchmarks/ASC_Sequoia/IRSmk/IRSmk.test | 40|2.212075899| 8974161060|2.208563573| 0.16| 8974289110| 0| 2.20958891| 0.11| 8974289109| 0|
|MultiSource/Benchmarks/BitBench/drop3/drop3.test | 169|0.165502473| 1989074268|0.165868692| -0.22| 1989074282| 0|0.166100023| -0.36| 1989074282| 0|
|MultiSource/Benchmarks/BitBench/five11/five11.test | 40|1.108902765|11578404923|1.108417771| 0.04|11578404923| 0|1.109621078| -0.06|11578404923| 0|
|MultiSource/Benchmarks/BitBench/uudecode/uudecode.test | 325|0.051157985| 739996870|0.051196828| -0.08| 739996866| 0|0.051181051| -0.05| 739996866| 0|
|MultiSource/Benchmarks/BitBench/uuencode/uuencode.test | 2000|0.007618035| 95632612|0.007612137| 0.08| 95632608| 0|0.007624132| -0.08| 95632608| 0|
|MultiSource/Benchmarks/Bullet/bullet.test | 40|3.100232661|20411549376| 3.08908226| 0.36|20344332128| 0.33|3.085584196| 0.47|19944394342| 2.34|
|MultiSource/Benchmarks/DOE-ProxyApps-C++/CLAMR/CLAMR.test | 40|1.701780182| 7841108926|1.668697318| 1.98| 7638277453| 2.66|1.667994648| 2.03| 7638277450| 2.66|
|MultiSource/Benchmarks/DOE-ProxyApps-C++/HACCKernels/HACCKernels.test | 40| 1.48737964| 9896344890|1.486979924| 0.03| 9896344882| 0|1.487130984| 0.02| 9896344882| 0|
|MultiSource/Benchmarks/DOE-ProxyApps-C++/HPCCG/HPCCG.test | 65|0.526810604| 3453684292|0.523730614| 0.59| 3397415681| 1.66| 0.52241587| 0.84| 3397415680| 1.66|
|MultiSource/Benchmarks/DOE-ProxyApps-C++/PENNANT/PENNANT.test | 74|0.461503651| 3277466030|0.436302908| 5.78| 3081728827| 6.35|0.435748999| 5.91| 3081728827| 6.35|
|MultiSource/Benchmarks/DOE-ProxyApps-C++/miniFE/miniFE.test | 40|2.785365175|16967821186|2.761763971| 0.85|17073218934| -0.62|2.764480422| 0.76|17073218856| -0.62|
|MultiSource/Benchmarks/DOE-ProxyApps-C/Pathfinder/PathFinder.test | 40|1.859656743|11929189383|1.846241592| 0.73|12060634799| -1.09|1.844557322| 0.82|12060634799| -1.09|
|MultiSource/Benchmarks/DOE-ProxyApps-C/RSBench/rsbench.test | 89|0.349593699| 1895634860|0.348378196| 0.35| 1884479689| 0.59| 0.35004465| -0.13| 1894547420| 0.06|
|MultiSource/Benchmarks/DOE-ProxyApps-C/SimpleMOC/SimpleMOC.test | 40|1.354475041|12398092189|1.346655436| 0.58|12072876852| 2.69|Segmentation fault |
|MultiSource/Benchmarks/DOE-ProxyApps-C/XSBench/XSBench.test | 40|1.738973338|11478251410|1.739544013| -0.03|11478251408| 0|1.736277498| 0.16|11478251409| 0|
|MultiSource/Benchmarks/DOE-ProxyApps-C/miniAMR/miniAMR.test | 77|0.446020905| 2499357506|0.399211813| 11.73| 2114512829| 18.2|0.399135712| 11.75| 2114512827| 18.2|
|MultiSource/Benchmarks/DOE-ProxyApps-C/miniGMG/miniGMG.test | 40|4.463160117|39495727411|4.513092048| -1.11|41349450244| -4.48|4.514933929| -1.15|41349450244| -4.48|
|MultiSource/Benchmarks/Fhourstones-3.1/fhourstones3.1.test | 50|0.727607024| 3984827073| 0.72608465| 0.21| 3984827073| 0|0.725904042| 0.23| 3939143747| 1.16|
|MultiSource/Benchmarks/Fhourstones/fhourstones.test | 62|0.501330239| 3411226199|0.503241158| -0.38| 3411226236| 0|0.502318951| -0.2| 3411226234| 0|
|MultiSource/Benchmarks/FreeBench/analyzer/analyzer.test | 354| 0.04170324| 200209858|0.041814069| -0.27| 200209861| 0|0.041770605| -0.16| 200209862| 0|
|MultiSource/Benchmarks/FreeBench/distray/distray.test | 266|0.076093218| 433110094| 0.07548559| 0.8| 437230649| -0.94|Wrong output |
|MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow.test | 165|0.167456778| 1587336472|0.167386069| 0.04| 1587336483| 0|0.167529719| -0.04| 1587336482| 0|
|MultiSource/Benchmarks/FreeBench/mason/mason.test | 225|0.102074077| 732136575|0.102129407| -0.05| 732136575| 0| 0.10199898| 0.07| 732136575| 0|
|MultiSource/Benchmarks/FreeBench/neural/neural.test | 281|0.067091237| 485987030|0.062584854| 7.2| 449351840| 8.15| 0.06253224| 7.29| 449351840| 8.15|
|MultiSource/Benchmarks/FreeBench/pcompress2/pcompress2.test | 250|0.085234923| 381524686|0.085025746| 0.25| 380640232| 0.23|0.084940475| 0.35| 380640232| 0.23|
|MultiSource/Benchmarks/FreeBench/pifft/pifft.test | 319|0.053592126| 475969337|0.054333321| -1.36| 466941040| 1.93|0.054344607| -1.38| 466941039| 1.93|
|MultiSource/Benchmarks/MallocBench/cfrac/cfrac.test | 51|0.711857916| 5618701452|0.704370551| 1.06| 5578522523| 0.72|0.703280359| 1.22| 5578522522| 0.72|
|MultiSource/Benchmarks/MallocBench/espresso/espresso.test | 116|0.265408561| 1973459113|0.259836146| 2.14| 1929014137| 2.3|0.257958653| 2.89| 1929014137| 2.3|
|MultiSource/Benchmarks/MallocBench/gs/gs.test | 420|0.019941178| 161219695|0.019873082| 0.34| 160951613| 0.17|0.019994524| -0.27| 161049366| 0.11|
|MultiSource/Benchmarks/McCat/01-qbsort/qbsort.test | 374| 0.03600226| 346345056|0.035802963| 0.56| 345040872| 0.38|0.035781181| 0.62| 345040872| 0.38|
|MultiSource/Benchmarks/McCat/03-testtrie/testtrie.test | 532|0.004789364| 35791938|0.004788924| 0.01| 35791934| 0|0.004766156| 0.49| 35791934| 0|
|MultiSource/Benchmarks/McCat/04-bisect/bisect.test | 279|0.069007538| 186526057|0.069011136| -0.01| 186524033| 0| 0.06907746| -0.1| 186524033| 0|
|MultiSource/Benchmarks/McCat/05-eks/eks.test | 553| 0.00165538| 15570781|0.001646575| 0.53| 15223220| 2.28|0.001638383| 1.04| 15223220| 2.28|
|MultiSource/Benchmarks/McCat/08-main/main.test | 372| 0.03337232| 161304278|0.032804198| 1.73| 155911053| 3.46|0.032800796| 1.74| 155911053| 3.46|
|MultiSource/Benchmarks/McCat/09-vor/vor.test | 279|0.071138328| 306540065|0.071136569| 0| 299160437| 2.47|0.070723002| 0.59| 299160310| 2.47|
|MultiSource/Benchmarks/McCat/12-IOtest/iotest.test | 165|0.168151514| 1011901996|0.168855783| -0.42| 1011901997| 0|0.168294985| -0.09| 1011901997| 0|
|MultiSource/Benchmarks/McCat/17-bintr/bintr.test | 288|0.066522953| 156137162| 0.06641589| 0.16| 156137176| 0|0.066105489| 0.63| 156137176| 0|
|MultiSource/Benchmarks/McCat/18-imp/imp.test | 356|0.037911597| 413739243|0.036002649| 5.3| 402732049| 2.73|0.036023493| 5.24| 402732049| 2.73|
|MultiSource/Benchmarks/MiBench/automotive-basicmath/automotive-basicmath.test | 149|0.191687238| 1618428282|0.191739184| -0.03| 1618428285| 0|0.191648113| 0.02| 1618428284| 0|
|MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive-bitcount.test | 327| 0.04921461| 545726446|0.049273993| -0.12| 545726446| 0|0.049319623| -0.21| 545726446| 0|
|MultiSource/Benchmarks/MiBench/automotive-susan/automotive-susan.test | 407|0.027639707| 366507115|0.023744347| 16.41| 366507088| 0|0.023748187| 16.39| 366507088| 0|
|MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg.test | 554|0.001989594| 18819644| 0.00196219| 1.4| 18813962| 0.03|0.001975317| 0.72| 18813904| 0.03|
|MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame.test | 218|0.109014693| 911759808|0.108727547| 0.26| 904654896| 0.79|0.108545272| 0.43| 904608732| 0.79|
|MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset.test | 279|0.071121692| 495053202|0.070289502| 1.18| 490793738| 0.87|0.070373233| 1.06| 490794652| 0.87|
|MultiSource/Benchmarks/MiBench/network-dijkstra/network-dijkstra.test | 430|0.020567007| 176527194|0.019699038| 4.41| 176527198| 0|0.019804709| 3.85| 176527197| 0|
|MultiSource/Benchmarks/MiBench/network-patricia/network-patricia.test | 303| 0.06121928| 416943428|0.061165157| 0.09| 416944819| 0|0.061278774| -0.1| 416944819| 0|
|MultiSource/Benchmarks/MiBench/security-rijndael/security-rijndael.test | 433|0.018099047| 201247421|0.018235098| -0.75| 201247418| 0|0.018288239| -1.03| 201247418| 0|
|MultiSource/Benchmarks/MiBench/security-sha/security-sha.test | 499|0.009854432| 101220897|0.009849696| 0.05| 101220908| 0|0.009836555| 0.18| 101220908| 0|
|MultiSource/Benchmarks/MiBench/telecomm-CRC32/telecomm-CRC32.test | 186|0.142363463| 1198185188| 0.14242033| -0.04| 1198185192| 0|0.142398262| -0.02| 1198185192| 0|
|MultiSource/Benchmarks/MiBench/telecomm-FFT/telecomm-fft.test | 417|0.024415065| 180732945|0.024456293| -0.17| 180732955| 0|0.024469807| -0.22| 180732955| 0|
|MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm.test | 244|0.091691314| 929866173|0.091410543| 0.31| 929851703| 0|0.091484827| 0.23| 929851702| 0|
|MultiSource/Benchmarks/NPB-serial/is/is.test | 40|4.489582746|11646133712| 4.49184759| -0.05|11612579256| 0.29|4.491813371| -0.05|11612579254| 0.29|
|MultiSource/Benchmarks/Olden/bh/bh.test | 39|0.929609207| 3751128971|0.927774549| 0.2| 3751128988| 0|Segmentation fault |
|MultiSource/Benchmarks/Olden/bisort/bisort.test | 102|0.317283496| 1340409887|0.310934109| 2.04| 1293898275| 3.59| 0.31068572| 2.12| 1293898274| 3.59|
|MultiSource/Benchmarks/Olden/em3d/em3d.test | 40| 1.48982862|18745041476|1.490320995| -0.03|18745041482| 0|1.489432579| 0.03|18745041481| 0|
|MultiSource/Benchmarks/Olden/health/health.test | 178|0.151827671| 729449597|0.152394401| -0.37| 729287331| 0.02|0.152143091| -0.21| 729287332| 0.02|
|MultiSource/Benchmarks/Olden/mst/mst.test | 310|0.055132658| 145175386|0.055015749| 0.21| 144176435| 0.69|0.054914935| 0.4| 144176435| 0.69|
|MultiSource/Benchmarks/Olden/perimeter/perimeter.test | 207|0.118713262| 794950669|0.118572285| 0.12| 794950669| 0|0.118844975| -0.11| 794950667| 0|
|MultiSource/Benchmarks/Olden/power/power.test | 58|0.608757815| 1783836856|0.608647972| 0.02| 1783836856| 0|0.608785459| 0| 1783836856| 0|
|MultiSource/Benchmarks/Olden/treeadd/treeadd.test | 205|0.123738062| 1107496252|0.123807326| -0.06| 1107496265| 0|0.123853276| -0.09| 1107496266| 0|
|MultiSource/Benchmarks/Olden/tsp/tsp.test | 66|0.530158273| 2170421855|0.530183799| 0| 2170421904| 0|0.530303485| -0.03| 2170421904| 0|
|MultiSource/Benchmarks/Olden/voronoi/voronoi.test | 157|0.181976294| 1116536604|0.182142473| -0.09| 1113350268| 0.29|0.182038436| -0.03| 1113350268| 0.29|
|MultiSource/Benchmarks/Prolangs-C++/city/city.test | 530|0.004188359| 31504099|0.004130416| 1.4| 31504084| 0|0.004126741| 1.49| 31504084| 0|
|MultiSource/Benchmarks/Prolangs-C++/employ/employ.test | 531|0.003630414| 26005612|0.003591581| 1.08| 26005611| 0|0.003606218| 0.67| 26005611| 0|
|MultiSource/Benchmarks/Prolangs-C++/life/life.test | 60|0.593411511| 5337258994|0.593464121| -0.01| 5337258990| 0|0.593413885| 0| 5337258990| 0|
|MultiSource/Benchmarks/Prolangs-C++/ocean/ocean.test | 364|0.038291262| 233764588|0.038295532| -0.01| 233263599| 0.21|0.038340759| -0.13| 233263600| 0.21|
|MultiSource/Benchmarks/Prolangs-C++/primes/primes.test | 153|0.188829837| 702321582|0.188790752| 0.02| 702321582| 0|0.188650677| 0.09| 702321582| 0|
|MultiSource/Benchmarks/Prolangs-C++/simul/simul.test | 547|0.002886062| 35432236|0.002880781| 0.18| 35432240| 0| 0.00289412| -0.28| 35432240| 0|
|MultiSource/Benchmarks/Prolangs-C/agrep/agrep.test | 551|0.001939458| 10466411|0.001957115| -0.9| 10466414| 0|0.001939551| 0| 10466414| 0|
|MultiSource/Benchmarks/Prolangs-C/gnugo/gnugo.test | 409|0.021872206| 180503585|0.021904266| -0.15| 180503589| 0| 0.02196555| -0.42| 180503589| 0|
|MultiSource/Benchmarks/Ptrdist/anagram/anagram.test | 200|0.547015945| 3731701775|0.464701347| 17.71| 4461177826|-16.35| 0.46399382| 17.89| 4461177826|-16.35|
|MultiSource/Benchmarks/Ptrdist/bc/bc.test | 103|0.316302602| 2780797147|0.315401853| 0.29| 2770795255| 0.36|0.315646342| 0.21| 2770795254| 0.36|
|MultiSource/Benchmarks/Ptrdist/ft/ft.test | 69|0.494429301| 716815201| 0.49609405| -0.34| 716722143| 0.01|0.495421047| -0.2| 716722143| 0.01|
|MultiSource/Benchmarks/Ptrdist/ks/ks.test | 55|0.647041451| 4430984141|0.647269346| -0.04| 4430743057| 0.01|0.647045686| 0| 4430743057| 0.01|
|MultiSource/Benchmarks/Ptrdist/yacr2/yacr2.test | 81| 0.41891967| 4341406074|0.415939335| 0.72| 4336616952| 0.11|0.415140426| 0.91| 4336616952| 0.11|
|MultiSource/Benchmarks/TSVC/ControlFlow-dbl/ControlFlow-dbl.test | 40|2.252897075|21925124584|2.241870402| 0.49|21925124591| 0|2.245260632| 0.34|21925124594| 0|
|MultiSource/Benchmarks/TSVC/ControlFlow-flt/ControlFlow-flt.test | 40|1.831478707|21917035866|1.839019529| -0.41|21917035878| 0|1.843303704| -0.64|21917035878| 0|
|MultiSource/Benchmarks/TSVC/ControlLoops-dbl/ControlLoops-dbl.test | 40|2.163047813|10131154875|2.167388513| -0.2|10131154868| 0|2.162387173| 0.03|10131154865| 0|
|MultiSource/Benchmarks/TSVC/ControlLoops-flt/ControlLoops-flt.test | 40|1.790150659| 6749980181|1.783526288| 0.37| 6749980171| 0|1.786574704| 0.2| 6749980173| 0|
|MultiSource/Benchmarks/TSVC/CrossingThresholds-dbl/CrossingThresholds-dbl.test| 40| 2.08235624|11637250479|2.082839371| -0.02|11637250478| 0| 2.0841341| -0.09|11637250478| 0|
|MultiSource/Benchmarks/TSVC/CrossingThresholds-flt/CrossingThresholds-flt.test| 40|1.513224597| 9567133532|1.509145173| 0.27| 9567133532| 0|1.505826289| 0.49| 9567133531| 0|
|MultiSource/Benchmarks/TSVC/Equivalencing-dbl/Equivalencing-dbl.test | 40|1.274317975|11224315317|1.110862868| 14.71| 8075528061| 38.99|1.107221342| 15.09| 8075528058| 38.99|
|MultiSource/Benchmarks/TSVC/Equivalencing-flt/Equivalencing-flt.test | 40|1.109170285|10863217124|0.755861233| 46.74| 7339479593| 48.01|0.754342876| 47.04| 7339479593| 48.01|
|MultiSource/Benchmarks/TSVC/Expansion-dbl/Expansion-dbl.test | 40|2.108596265|11394673308| 2.10772576| 0.04|11394673312| 0|2.106914656| 0.08|11394673312| 0|
|MultiSource/Benchmarks/TSVC/Expansion-flt/Expansion-flt.test | 40| 1.19302084| 8429740469|1.201718248| -0.72| 8429740475| 0|1.200748706| -0.64| 8429740477| 0|
|MultiSource/Benchmarks/TSVC/GlobalDataFlow-dbl/GlobalDataFlow-dbl.test | 40|2.003006427|12945473678|2.030239169| -1.34|12945473685| 0|2.005207587| -0.11|12945473680| 0|
|MultiSource/Benchmarks/TSVC/GlobalDataFlow-flt/GlobalDataFlow-flt.test | 43|0.824523437| 5574340604|0.819459866| 0.62| 5574340602| 0|0.808334658| 2| 5574340599| 0|
|MultiSource/Benchmarks/TSVC/IndirectAddressing-dbl/IndirectAddressing-dbl.test| 40| 2.13915252|16924618858|2.139128559| 0|16924618893| 0|2.139481883| -0.02|16924618893| 0|
|MultiSource/Benchmarks/TSVC/IndirectAddressing-flt/IndirectAddressing-flt.test| 40|1.797690481|16924004255|1.798647228| -0.05|16924004256| 0|1.797408641| 0.02|16924004255| 0|
|MultiSource/Benchmarks/TSVC/InductionVariable-dbl/InductionVariable-dbl.test | 40|2.393466475|16204217793|2.382771124| 0.45|16204217795| 0|2.390437006| 0.13|16204217795| 0|
|MultiSource/Benchmarks/TSVC/InductionVariable-flt/InductionVariable-flt.test | 40|1.765705552|15096881613|1.758229966| 0.43|15096881614| 0|1.763492228| 0.13|15096881616| 0|
|MultiSource/Benchmarks/TSVC/LinearDependence-dbl/LinearDependence-dbl.test | 40| 2.0902936|17747770111|2.093725046| -0.16|17747770107| 0|2.089922083| 0.02|17747770106| 0|
|MultiSource/Benchmarks/TSVC/LinearDependence-flt/LinearDependence-flt.test | 40|1.426366572|15128429282|1.421173178| 0.37|15128429276| 0|1.421455642| 0.35|15128429276| 0|
|MultiSource/Benchmarks/TSVC/LoopRerolling-dbl/LoopRerolling-dbl.test | 40|2.479358186|12728810214|2.476810043| 0.1|12728810217| 0|2.479956135| -0.02|12728810217| 0|
|MultiSource/Benchmarks/TSVC/LoopRerolling-flt/LoopRerolling-flt.test | 40|1.749389689| 9799621839|1.751205696| -0.1| 9799621843| 0|1.746036585| 0.19| 9799621844| 0|
|MultiSource/Benchmarks/TSVC/LoopRestructuring-dbl/LoopRestructuring-dbl.test | 40|2.994641973| 4745295255|2.998408679| -0.13| 4745295258| 0|2.986642419| 0.27| 4745295256| 0|
|MultiSource/Benchmarks/TSVC/LoopRestructuring-flt/LoopRestructuring-flt.test | 40|2.402549805| 4419860245| 2.3960923| 0.27| 4419860251| 0|2.403938844| -0.06| 4419860252| 0|
|MultiSource/Benchmarks/TSVC/NodeSplitting-dbl/NodeSplitting-dbl.test | 40|2.308084263|18642442940| 2.30841202| -0.01|18642442945| 0|2.308712261| -0.03|18642442944| 0|
|MultiSource/Benchmarks/TSVC/NodeSplitting-flt/NodeSplitting-flt.test | 40|1.767745677|16921992007|1.767578024| 0.01|16921992009| 0|1.765723718| 0.11|16921992009| 0|
|MultiSource/Benchmarks/TSVC/Packing-dbl/Packing-dbl.test | 40|1.997099913|25028654892|1.990854076| 0.31|25028654895| 0| 1.98528254| 0.6|25028654894| 0|
|MultiSource/Benchmarks/TSVC/Packing-flt/Packing-flt.test | 40|1.790757599|25028055720|1.793887178| -0.17|25028055724| 0|1.792900939| -0.12|25028055723| 0|
|MultiSource/Benchmarks/TSVC/Recurrences-dbl/Recurrences-dbl.test | 40|2.782578903| 8161788064|2.785879224| -0.12| 8161788076| 0|2.785581735| -0.11| 8161788075| 0|
|MultiSource/Benchmarks/TSVC/Recurrences-flt/Recurrences-flt.test | 40|2.721759589| 8161188258|2.721249965| 0.02| 8161188269| 0|2.721682374| 0| 8161188268| 0|
|MultiSource/Benchmarks/TSVC/Reductions-dbl/Reductions-dbl.test | 40|2.533641924| 8342583212|2.535340253| -0.07| 8342583211| 0|2.535789475| -0.08| 8342583212| 0|
|MultiSource/Benchmarks/TSVC/Reductions-flt/Reductions-flt.test | 40|4.475013409| 8356936613|4.475155146| 0| 8356936613| 0|4.475228458| 0| 8356936614| 0|
|MultiSource/Benchmarks/TSVC/Searching-dbl/Searching-dbl.test | 40|2.062359672|17924201662|2.063396566| -0.05|17924201666| 0|2.063218715| -0.04|17924201666| 0|
|MultiSource/Benchmarks/TSVC/Searching-flt/Searching-flt.test | 40|2.057379471|17923778464|2.055515952| 0.09|17923778468| 0|2.055343632| 0.1|17923778467| 0|
|MultiSource/Benchmarks/TSVC/StatementReordering-dbl/StatementReordering-dbl.te| 40|2.267773304|15682563910|2.260157636| 0.34|15682563913| 0|2.268108021| -0.01|15682563915| 0|
|MultiSource/Benchmarks/TSVC/StatementReordering-flt/StatementReordering-flt.te| 40|1.580584254|15681328340|1.580687746| -0.01|15681328345| 0|1.579427267| 0.07|15681328344| 0|
|MultiSource/Benchmarks/TSVC/Symbolics-dbl/Symbolics-dbl.test | 40|1.762957745| 8152774997|1.742350002| 1.18| 8152774995| 0|1.763084983| -0.01| 8152775000| 0|
|MultiSource/Benchmarks/TSVC/Symbolics-flt/Symbolics-flt.test | 51|0.693108134| 4952182033|0.689952536| 0.46| 4952182036| 0|0.691439179| 0.24| 4952182035| 0|
|MultiSource/Benchmarks/Trimaran/enc-3des/enc-3des.test | 40|1.127513838| 8842073434|1.122112713| 0.48| 8842073428| 0|1.124921411| 0.23| 8842073429| 0|
|MultiSource/Benchmarks/Trimaran/enc-md5/enc-md5.test | 40|1.045967332| 8328857620|1.045981934| 0| 8328857624| 0|1.045553313| 0.04| 8328857624| 0|
|MultiSource/Benchmarks/Trimaran/enc-pc1/enc-pc1.test | 95|0.348125956| 2627315589| 0.34799906| 0.04| 2627315558| 0|0.347967402| 0.05| 2627315558| 0|
|MultiSource/Benchmarks/Trimaran/enc-rc4/enc-rc4.test | 60|0.591672553| 4749013932|0.590861035| 0.14| 4749013935| 0|0.592929197| -0.21| 4749013936| 0|
|MultiSource/Benchmarks/Trimaran/netbench-crc/netbench-crc.test | 65|0.542445555| 1724115045|0.542356199| 0.02| 1724115042| 0|0.542384405| 0.01| 1724115038| 0|
|MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.test | 40| 2.51225625| 7492347322|2.511805997| 0.02| 7492347316| 0|2.511224899| 0.04| 7492347316| 0|
|MultiSource/Benchmarks/VersaBench/8b10b/8b10b.test | 40| 3.02230322|16882610042|3.026750117| -0.15|16882610046| 0|3.024096776| -0.06|16882610047| 0|
|MultiSource/Benchmarks/VersaBench/beamformer/beamformer.test | 61|0.580845547| 6859902047|0.672603918|-13.64| 8647956851|-20.68|0.672356913|-13.61| 8647956851|-20.68|
|MultiSource/Benchmarks/VersaBench/bmm/bmm.test | 40|1.292170426| 9932045338|1.305024772| -0.98| 9932045354| 0|1.292741542| -0.04| 9932045350| 0|
|MultiSource/Benchmarks/VersaBench/dbms/dbms.test | 45|0.810661753| 3391009910|0.802206556| 1.05| 3324663069| 2|0.800454835| 1.28| 3324663069| 2|
|MultiSource/Benchmarks/VersaBench/ecbdes/ecbdes.test | 40|1.457606778|11993065051|1.457628021| 0|11993065048| 0|1.458776986| -0.08|11993065048| 0|
|MultiSource/Benchmarks/llubenchmark/llu.test | 40|5.202703599| 2493038287| 5.21286191| -0.19| 2493038293| 0|5.201113006| 0.03| 2493038289| 0|
|MultiSource/Benchmarks/mafft/pairlocalalign.test | 40|11.83656454|96495561521|11.63414059| 1.74|93867683605| 2.8|11.63303648| 1.75|93867683605| 2.8|
|MultiSource/Benchmarks/mediabench/adpcm/rawcaudio/rawcaudio.test | 547|0.002165289| 10021825|0.002171213| -0.27| 10021794| 0|0.002164967| 0.01| 10021794| 0|
|MultiSource/Benchmarks/mediabench/adpcm/rawdaudio/rawdaudio.test | 559|0.000949047| 7984584|0.000947677| 0.14| 7984553| 0|0.000946015| 0.32| 7984553| 0|
|MultiSource/Benchmarks/mediabench/g721/g721encode/encode.test | 395|0.025766454| 196921819|0.025713182| 0.21| 196921823| 0|0.025783182| -0.06| 196921824| 0|
|MultiSource/Benchmarks/mediabench/gsm/toast/toast.test | 488|0.011509398| 112819926|0.011472924| 0.32| 112818068| 0|0.011492431| 0.15| 112818068| 0|
|MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg.test | 10000| 0.0019677| 12557713| 0.00201222| -2.21| 12621648| -0.51|0.002017135| -2.45| 12621708| -0.51|
|MultiSource/Benchmarks/mediabench/mpeg2/mpeg2dec/mpeg2decode.test | 512|0.005408392| 44415847|0.005413902| -0.1| 44415888| 0|0.005397437| 0.2| 44415888| 0|
|MultiSource/Benchmarks/nbench/nbench.test | 40|1.022024835| 7285718014|1.012235917| 0.97| 7285109493| 0.01|1.012869111| 0.9| 7285109494| 0.01|
|MultiSource/Benchmarks/sim/sim.test | 40|2.330088306|22428909837|2.286765812| 1.89|21976372357| 2.06|2.287846691| 1.85|21976372356| 2.06|
|MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4.test | 122|0.254109782| 2175404640|0.179816396| 41.32| 1403783168| 54.97|0.179490547| 41.57| 1403737509| 54.97|
|SingleSource/Benchmarks/Adobe-C++/functionobjects.test | 40|2.173834838| 7456450808|2.153285393| 0.95| 7456450841| 0|2.152186046| 1.01| 7456450841| 0|
|SingleSource/Benchmarks/Adobe-C++/loop_unroll.test | 55|0.631261876| 4991342551|0.631185655| 0.01| 4991342299| 0|0.631218693| 0.01| 4991342299| 0|
|SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding.test | 70|0.493520528| 1690319795|0.493526469| 0| 1690319517| 0|0.493366313| 0.03| 1690319517| 0|
|SingleSource/Benchmarks/Adobe-C++/simple_types_loop_invariant.test | 40|1.523591362| 3396936368|1.519790444| 0.25| 3396936170| 0|1.518031974| 0.37| 3396936169| 0|
|SingleSource/Benchmarks/Adobe-C++/stepanov_abstraction.test | 40|2.746768677|10325009324|2.756353979| -0.35|10325009323| 0|2.757201683| -0.38|10325009322| 0|
|SingleSource/Benchmarks/Adobe-C++/stepanov_vector.test | 40|1.537741532| 7537457571|1.529771183| 0.52| 7512794288| 0.33|1.537713776| 0| 7513984379| 0.31|
|SingleSource/Benchmarks/BenchmarkGame/Large/fasta.test | 67|0.506800383| 1260201726|0.507268641| -0.09| 1260201771| 0|0.507130533| -0.07| 1260201771| 0|
|SingleSource/Benchmarks/BenchmarkGame/fannkuch.test | 40|1.641754504| 9957589773|1.641594175| 0.01| 9957589786| 0|1.641584928| 0.01| 9957589786| 0|
|SingleSource/Benchmarks/BenchmarkGame/n-body.test | 74|0.460147754| 3530012913|0.460207529| -0.01| 3530012927| 0|0.460297178| -0.03| 3530012927| 0|
|SingleSource/Benchmarks/BenchmarkGame/nsieve-bits.test | 73|0.459704364| 2887846613|0.462508445| -0.61| 2887846661| 0|0.460018001| -0.07| 2887846661| 0|
|SingleSource/Benchmarks/BenchmarkGame/partialsums.test | 94|0.306776129| 1918961426| 0.30656601| 0.07| 1918961475| 0|0.306822317| -0.02| 1918961476| 0|
|SingleSource/Benchmarks/BenchmarkGame/puzzle.test | 262|0.073875703| 945775099|0.073682328| 0.26| 945775112| 0|0.073720778| 0.21| 945775112| 0|
|SingleSource/Benchmarks/BenchmarkGame/recursive.test | 68|0.509415882| 4930083245|0.509053286| 0.07| 4930083259| 0|0.509095231| 0.06| 4930083259| 0|
|SingleSource/Benchmarks/BenchmarkGame/spectral-norm.test | 61| 0.57655867| 2560868014|0.576614406| -0.01| 2560868028| 0|0.576462803| 0.02| 2560868028| 0|
|SingleSource/Benchmarks/CoyoteBench/almabench.test | 40|7.778179447|43389260272|7.777261689| 0.01|43389260276| 0| 7.77649329| 0.02|43389260275| 0|
|SingleSource/Benchmarks/CoyoteBench/fftbench.test | 300|0.577644623| 1677763916|0.575129582| 0.44| 1659545051| 1.1|0.574668086| 0.52| 1655350754| 1.35|
|SingleSource/Benchmarks/CoyoteBench/huffbench.test | 40|8.787346305|82150680101|8.787841696| -0.01|82150680104| 0|8.788171583| -0.01|82150680102| 0|
|SingleSource/Benchmarks/CoyoteBench/lpbench.test | 40|2.229018103| 7882704081|2.237111177| -0.36| 7882704087| 0|2.234362009| -0.24| 7882704087| 0|
|SingleSource/Benchmarks/Dhrystone/dry.test | 104|0.309225964| 2900008333|0.257653163| 20.02| 2600008326| 11.54|0.257643637| 20.02| 2600008326| 11.54|
|SingleSource/Benchmarks/Dhrystone/fldry.test | 104| 0.30911735| 3200009655|0.257674916| 19.96| 2700009648| 18.52|0.257713524| 19.95| 2700009648| 18.52|
|SingleSource/Benchmarks/Linpack/linpack-pc.test | 40|1.166713282|12124202635|1.166238827| 0.04|12124202639| 0|1.166710153| 0|12124202639| 0|
|SingleSource/Benchmarks/McGill/chomp.test | 100|1.039244119| 2199695089|1.026021223| 1.29| 2022691279| 8.75|1.005825305| 3.32| 2022691274| 8.75|
|SingleSource/Benchmarks/McGill/misr.test | 236|0.097331789| 993116970|0.097260678| 0.07| 993116970| 0|0.097328554| 0| 993116970| 0|
|SingleSource/Benchmarks/McGill/queens.test | 40|1.277072749| 3750256402|1.274574564| 0.2| 3750256401| 0|1.275354558| 0.13| 3750256401| 0|
|SingleSource/Benchmarks/Misc-C++-EH/spirit.test | 40|5.609729191|26762600467|5.608739395| 0.02|26754795190| 0.03|Segmentation fault |
|SingleSource/Benchmarks/Misc-C++/Large/ray.test | 40|1.906342711|13297641949|1.905762342| 0.03|13297641950| 0|1.903530315| 0.15|13297641949| 0|
|SingleSource/Benchmarks/Misc-C++/Large/sphereflake.test | 40| 3.2061503|17704100965|3.286619781| -2.45|18336685579| -3.45|3.285463897| -2.41|18336685578| -3.45|
|SingleSource/Benchmarks/Misc-C++/bigfib.test | 150|0.191461571| 513566319|0.192312746| -0.44| 513516314| 0.01|0.192279701| -0.43| 513516313| 0.01|
|SingleSource/Benchmarks/Misc-C++/mandel-text.test | 40|1.094141275| 5480623268|1.093537162| 0.06| 5480623264| 0|1.093854114| 0.03| 5480623264| 0|
|SingleSource/Benchmarks/Misc-C++/oopack_v1p8.test | 256|0.082014067| 532366158|0.081900482| 0.14| 532366155| 0|0.081939047| 0.09| 532366154| 0|
|SingleSource/Benchmarks/Misc-C++/stepanov_container.test | 40|2.340649713| 5147979419|2.344395208| -0.16| 5147328718| 0.01|2.342970076| -0.1| 5147328716| 0.01|
|SingleSource/Benchmarks/Misc-C++/stepanov_v1p2.test | 40|4.915020328|10294333477|4.916416816| -0.03|10294333467| 0|4.914229028| 0.02|10294333470| 0|
|SingleSource/Benchmarks/Misc/ReedSolomon.test | 40|2.810030776|34665032211| 2.8095424| 0.02|34665032204| 0|2.809962792| 0|34665032204| 0|
|SingleSource/Benchmarks/Misc/dt.test | 71|0.487815801| 739263908|0.487927234| -0.02| 739263899| 0|0.488082106| -0.05| 739263899| 0|
|SingleSource/Benchmarks/Misc/evalloop.test | 103| 0.31638469| 1434422790|0.316386734| 0| 1434422783| 0|0.316393606| 0| 1434422783| 0|
|SingleSource/Benchmarks/Misc/fbench.test | 39| 0.93296777| 5897040682| 0.93183071| 0.12| 5897040675| 0|0.932808517| 0.02| 5897040675| 0|
|SingleSource/Benchmarks/Misc/ffbench.test | 63|0.550404247| 1799363439|0.537866871| 2.33| 1799363483| 0| 0.5385274| 2.21| 1799363483| 0|
|SingleSource/Benchmarks/Misc/flops-1.test | 40|1.125572165| 5937513995|1.125291834| 0.02| 5937514042| 0|1.125383473| 0.02| 5937514041| 0|
|SingleSource/Benchmarks/Misc/flops-2.test | 52|0.686848557| 1445326407| 0.68678519| 0.01| 1445326453| 0|0.686721149| 0.02| 1445326453| 0|
|SingleSource/Benchmarks/Misc/flops-3.test | 40|1.050192568| 6875013884|1.050371006| -0.02| 6875013930| 0|1.050167961| 0| 6875013930| 0|
|SingleSource/Benchmarks/Misc/flops-4.test | 57| 0.48686156| 3281263860| 0.48705466| -0.04| 3281263907| 0|0.486777353| 0.02| 3281263907| 0|
|SingleSource/Benchmarks/Misc/flops-5.test | 48|0.757804282| 4687513650|0.757921721| -0.02| 4687513697| 0|0.758014067| -0.03| 4687513696| 0|
|SingleSource/Benchmarks/Misc/flops-6.test | 49|0.729726166| 4687513636|0.729725777| 0| 4687513682| 0|0.729815563| -0.01| 4687513682| 0|
|SingleSource/Benchmarks/Misc/flops-7.test | 40|1.131641163| 3281263704|1.131912385| -0.02| 3281263751| 0|1.131789983| -0.01| 3281263751| 0|
|SingleSource/Benchmarks/Misc/flops-8.test | 37|0.980478342| 5937513793|0.980528479| -0.01| 5937513839| 0|0.980442694| 0| 5937513839| 0|
|SingleSource/Benchmarks/Misc/flops.test | 40|5.830239093|31693658565|5.829165538| 0.02|31693658558| 0|5.829623935| 0.01|31693658557| 0|
|SingleSource/Benchmarks/Misc/fp-convert.test | 40|1.041756657|10895011561|1.041857029| -0.01|10895011554| 0|1.041977143| -0.02|10895011554| 0|
|SingleSource/Benchmarks/Misc/himenobmtxpa.test | 100|0.908763609| 9591664042|0.552177504| 64.58| 2999850585|219.74|0.550863158| 64.97| 2999850585|219.74|
|SingleSource/Benchmarks/Misc/mandel-2.test | 62|0.565282569| 3451931181| 0.56538205| -0.02| 3451931174| 0|0.565291149| 0| 3451931174| 0|
|SingleSource/Benchmarks/Misc/mandel.test | 87|0.384127781| 2765575042|0.384125541| 0| 2765575035| 0|0.384110649| 0| 2765575035| 0|
|SingleSource/Benchmarks/Misc/matmul_f64_4x4.test | 71|0.489191861| 6750041890|0.489258207| -0.01| 6750041883| 0|0.489120899| 0.01| 6750041883| 0|
|SingleSource/Benchmarks/Misc/oourafft.test | 40|2.067325053|26308061552|1.997489314| 3.5|25399500162| 3.58|1.998249201| 3.46|25399500162| 3.58|
|SingleSource/Benchmarks/Misc/perlin.test | 40|2.631994495| 9286340904|2.632883781| -0.03| 9286340897| 0|2.631952736| 0| 9286340898| 0|
|SingleSource/Benchmarks/Misc/pi.test | 85|0.392726791| 1320016089|0.392747395| -0.01| 1320016082| 0|0.392178699| 0.14| 1320016082| 0|
|SingleSource/Benchmarks/Misc/revertBits.test | 173|0.158693233| 2155883886|0.158737498| -0.03| 2155883879| 0|0.158739646| -0.03| 2155883879| 0|
|SingleSource/Benchmarks/Misc/richards_benchmark.test | 78|0.434660464| 4033193605|0.434746459| -0.02| 4033193598| 0|0.434897056| -0.05| 4033193598| 0|
|SingleSource/Benchmarks/Misc/salsa20.test | 40|3.685973193|45487828644|3.687791537| -0.05|45487828691| 0|3.685686752| 0.01|45487828690| 0|
|SingleSource/Benchmarks/Misc/whetstone.test | 46|0.784256179| 4409167761|0.784001984| 0.03| 4409167754| 0| 0.78510808| -0.11| 4409167754| 0|
|SingleSource/Benchmarks/Polybench/datamining/correlation/correlation.test | 40|8.877210343| 5619149084|8.889290661| -0.14| 5619149080| 0| 8.88393369| -0.08| 5619149080| 0|
|SingleSource/Benchmarks/Polybench/datamining/covariance/covariance.test | 40|8.881371053| 5597878269|8.874710262| 0.08| 5597878274| 0|8.877396778| 0.04| 5597878273| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/atax/atax.test | 203|0.125524393| 392664271|0.125337528| 0.15| 392664276| 0|0.125380979| 0.11| 392664277| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/bicg/bicg.test | 167|0.167677998| 464571982| 0.16758646| 0.05| 464571983| 0|0.167430814| 0.15| 464571980| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/cholesky/cholesky.tes| 110|0.286466354| 1690933420|0.287513507| -0.36| 1690933424| 0|0.287682162| -0.42| 1690933424| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/doitgen/doitgen.test | 75|0.445250085| 3399897372|0.446000827| -0.17| 3399897368| 0|0.446003224| -0.17| 3399897368| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemver/gemver.test | 72|0.476362624| 714917745|0.479692636| -0.69| 714917750| 0|0.475910147| 0.1| 714917746| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gesummv/gesummv.test | 146|0.201739334| 480454033|0.201744234| 0| 480454024| 0|0.201747124| 0| 480454027| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/mvt/mvt.test | 82|0.414533228| 480687267| 0.41015712| 1.07| 480687258| 0| 0.41239746| 0.52| 480687260| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/symm/symm.test | 40| 20.1390727|11903527707|20.09564897| 0.22|11903527704| 0| 20.0699099| 0.34|11903527697| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/syr2k/syr2k.test | 40|3.707457334|28020923169|3.697747398| 0.26|28020923166| 0|3.703755898| 0.1|28020923169| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/syrk/syrk.test | 40|1.709082542|14059124595|1.711252681| -0.13|14059124600| 0|1.711381977| -0.13|14059124601| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/trisolv/trisolv.test | 225|0.103904759| 216435815|0.104026252| -0.12| 216435811| 0|0.104078778| -0.17| 216435811| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/kernels/trmm/trmm.test | 39|0.918241363| 7071633999|0.921063844| -0.31| 7071634004| 0|0.921306267| -0.33| 7071634004| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/solvers/durbin/durbin.test | 50|0.723007084| 462483129|0.722617922| 0.05| 462483134| 0|0.723354695| -0.05| 462483134| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/solvers/dynprog/dynprog.test | 121|0.257576728| 2736056554|0.258435731| -0.33| 2736056550| 0|0.257280074| 0.12| 2736056550| 0|
|SingleSource/Benchmarks/Polybench/linear-algebra/solvers/gramschmidt/gramschmi| 40|1.620215144| 3060031368|1.615894212| 0.27| 3060031348| 0|1.623027315| -0.17| 3060031345| 0|
|SingleSource/Benchmarks/Polybench/medley/floyd-warshall/floyd-warshall.test | 40|1.453502166|14087698898|1.446967252| 0.45|14087698901| 0|1.452044968| 0.1|14087698904| 0|
|SingleSource/Benchmarks/Polybench/medley/reg_detect/reg_detect.test | 1000|0.011356117| 106220639|0.011360151| -0.04| 106220643| 0|0.011330179| 0.23| 106220643| 0|
|SingleSource/Benchmarks/Polybench/stencils/adi/adi.test | 40|2.044818632| 4254099532|2.045805251| -0.05| 4254099531| 0|2.046308981| -0.07| 4254099532| 0|
|SingleSource/Benchmarks/Polybench/stencils/fdtd-2d/fdtd-2d.test | 74|0.452778289| 1655964959|0.453290602| -0.11| 1655964959| 0| 0.45232197| 0.1| 1655964958| 0|
|SingleSource/Benchmarks/Polybench/stencils/fdtd-apml/fdtd-apml.test | 40|0.641113235| 2407565807|0.643359954| -0.35| 2407565809| 0|0.641981502| -0.14| 2407565810| 0|
|SingleSource/Benchmarks/Polybench/stencils/jacobi-1d-imper/jacobi-1d-imper.tes| 552|0.001836389| 11174707|0.001859998| -1.27| 11174707| 0|0.001857384| -1.13| 11174707| 0|
|SingleSource/Benchmarks/Polybench/stencils/jacobi-2d-imper/jacobi-2d-imper.tes| 203|0.116332295| 378809843|0.116979718| -0.55| 378809842| 0|0.116003351| 0.28| 378809842| 0|
|SingleSource/Benchmarks/Polybench/stencils/seidel-2d/seidel-2d.test | 82|0.408001586| 827552481|0.408103668| -0.03| 827552474| 0|0.407782615| 0.05| 827552474| 0|
|SingleSource/Benchmarks/Shootout-C++/EH/Shootout-C++-except.test | 197|0.127303942| 1215944814|0.127350943| -0.04| 1215944815| 0|0.127290381| 0.01| 1215944818| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-ackermann.test | 64|0.544478722| 6798746650|0.544868418| -0.07| 6798746629| 0|0.544653931| -0.03| 6798746629| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-ary.test | 393|0.030586908| 13569765|0.030578047| 0.03| 13569743| 0|0.030586414| 0| 13569743| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-ary2.test | 382| 0.03056193| 31569694|0.030576303| -0.05| 31569672| 0|0.030656052| -0.31| 31569672| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-ary3.test | 55|0.643855642| 1783106181|0.642993615| 0.13| 1783106160| 0|0.648657152| -0.74| 1783106160| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-fibo.test | 40|1.330984497|12893323142| 1.33100591| 0|12893323120| 0|1.331637378| -0.05|12893323121| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-hash.test | 111|0.287885926| 1912084291|0.287794193| 0.03| 1912084270| 0|0.288558767| -0.23| 1912084270| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-hash2.test | 40|1.270321343| 9689538467|1.269216982| 0.09| 9689538445| 0|1.269293009| 0.08| 9689538445| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-heapsort.test | 40| 1.75794887| 3919890400|1.754011318| 0.22| 3919890376| 0|1.758925061| -0.06| 3919890376| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-lists.test | 40|2.739061691|29793102996|2.740463212| -0.05|29793102974| 0|2.739521217| -0.02|29793102974| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-lists1.test | 201|0.125677348| 1166151573|0.125606508| 0.06| 1166151553| 0|0.125619352| 0.05| 1166151552| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-matrix.test | 40|1.160447756|11919784092|0.988233915| 17.43|10389784094| 14.73|0.987882481| 17.47|10389784093| 14.73|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-methcall.test | 40|3.172758454|20666719699|3.175058568| -0.07|20666719678| 0| 3.17390483| -0.04|20666719677| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-moments.test | 316|0.055939856| 215074597|0.055949457| -0.02| 215074576| 0|0.055892729| 0.08| 215074576| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-random.test | 40|1.470423781| 4400054983|1.470597393| -0.01| 4400054960| 0|1.470670857| -0.02| 4400054960| 0|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-sieve.test | 40|1.185858242| 3461649928|1.185756384| 0.01| 3461135918| 0.01| 1.18394198| 0.16| 3461135918| 0.01|
|SingleSource/Benchmarks/Shootout-C++/Shootout-C++-strcat.test | 354|0.040319038| 329835468|0.040429092| -0.27| 329835446| 0| 0.04028371| 0.09| 329835445| 0|
|SingleSource/Benchmarks/Shootout/Shootout-ackermann.test | 543|0.003060169| 26469562|0.003081942| -0.71| 26469558| 0|0.003073281| -0.43| 26469558| 0|
|SingleSource/Benchmarks/Shootout/Shootout-ary3.test | 54|0.649109744| 1783063149|0.648322161| 0.12| 1783063145| 0|0.645780095| 0.52| 1783063144| 0|
|SingleSource/Benchmarks/Shootout/Shootout-fib2.test | 40|1.333849001|12893280983|1.333856586| 0|12893280979| 0|1.333797266| 0|12893280979| 0|
|SingleSource/Benchmarks/Shootout/Shootout-hash.test | 40|2.835678675|13468880571|3.126493844| -9.3|13454912752| 0.1|3.161558794|-10.31|13395521694| 0.55|
|SingleSource/Benchmarks/Shootout/Shootout-heapsort.test | 40|1.782079491| 3919849059|1.789658296| -0.42| 3919849056| 0|1.786379433| -0.24| 3919849054| 0|
|SingleSource/Benchmarks/Shootout/Shootout-lists.test | 100|3.546476203|25320009952|3.655184772| -2.97|23268009976| 8.82|3.654873206| -2.97|23268009975| 8.82|
|SingleSource/Benchmarks/Shootout/Shootout-matrix.test | 40|0.914124212|11454020507|0.914234925| -0.01|11454020502| 0|0.914320846| -0.02|11454020503| 0|
|SingleSource/Benchmarks/Shootout/Shootout-methcall.test | 40|2.743907354|14000010155|2.743247489| 0.02|14000010151| 0|2.743393361| 0.02|14000010151| 0|
|SingleSource/Benchmarks/Shootout/Shootout-random.test | 40|1.470284439| 4400011701| 1.47032099| 0| 4400011697| 0|1.470363557| -0.01| 4400011696| 0|
|SingleSource/Benchmarks/Shootout/Shootout-sieve.test | 40|2.852198659|21785679872|2.850246735| 0.07|21785679867| 0| 2.85101485| 0.04|21785679868| 0|
|SingleSource/Benchmarks/Shootout/Shootout-strcat.test | 232|0.100245468| 319817791|0.100168107| 0.08| 319817786| 0|0.100184823| 0.06| 319817787| 0|
|SingleSource/Benchmarks/SmallPT/smallpt.test | 40|5.239863596|30531918340|5.240930496| -0.02|30531918346| 0|5.239798764| 0|30531918345| 0|
|SingleSource/Benchmarks/Stanford/Bubblesort.test | 441|0.015584076| 76543755|0.015530674| 0.34| 76543751| 0|0.015531506| 0.34| 76543751| 0|
|SingleSource/Benchmarks/Stanford/FloatMM.test | 275|0.069482096| 549642410|0.069534833| -0.08| 549642406| 0|0.069590491| -0.16| 549642406| 0|
|SingleSource/Benchmarks/Stanford/Oscar.test | 563|0.001015371| 7107593|0.000947407| 7.17| 7107590| 0|0.000927899| 9.43| 7107590| 0|
|SingleSource/Benchmarks/Stanford/Perm.test | 461|0.012860231| 112254639|0.012819633| 0.32| 112254635| 0|0.012869658| -0.07| 112254635| 0|
|SingleSource/Benchmarks/Stanford/Puzzle.test | 303|0.059227622| 454201062|0.059239688| -0.02| 454201058| 0| 0.05926861| -0.07| 454201058| 0|
|SingleSource/Benchmarks/Stanford/Queens.test | 515|0.006686264| 79619077|0.006702058| -0.24| 79619073| 0|0.006740698| -0.81| 79619073| 0|
|SingleSource/Benchmarks/Stanford/RealMM.test | 566|0.000541859| 1637288|0.000553031| -2.02| 1637284| 0|0.000540962| 0.17| 1637284| 0|
|SingleSource/Benchmarks/Stanford/Towers.test | 512|0.006855776| 69265251|0.006841316| 0.21| 69265247| 0|0.006845202| 0.15| 69265247| 0|
|SingleSource/Benchmarks/Stanford/Treesort.test | 330|0.045405366| 164531268|0.045481322| -0.17| 164531263| 0|0.045384358| 0.05| 164531263| 0|
|------------------------------------------------------------------------------|------|-----------|-----------|-----------|------|-----------|------|-----------|------|-----------|------|
More information about the llvm-dev
mailing list