[llvm-testresults] Grawp nightly tester results

[Daniel Dunbar]

On Sat, Sep 5, 2009 at 4:11 PM, Evan Cheng<evan.cheng at apple.com> wrote:
>
> On Sep 5, 2009, at 2:58 PM, Daniel Dunbar wrote:
>
>> On Fri, Sep 4, 2009 at 6:28 PM, Evan Cheng<evan.cheng at apple.com> wrote:
>>>
>>> What JIT regressions? All of the following failures are crashing in the
>>> loop
>>> unswitching pass. Devang is looking into it.
>>
>> The performance regressions, a large number of JIT tests seem to have
>> dropped by maybe an average of 5-10%.
>
> You're right. It looks real too. But none of the patches look suspicious.

That's why I asked if you can remember if the JIT callback patch was
in this round or the previous one. Not sure if just changing the
compilation callback could account for these swings, but I don't
remember any other obvious JIT changes.

 - Daniel

>>
>>> New Test Failures:
>>>
>>> Externals/SPEC/CFP2006/470_lbm/470_lbm [JIT, CBE, JIT codegen, LLC,
>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>
>>> Externals/SPEC/CINT2006/403_gcc/403_gcc [JIT, JIT codegen, LLC, LLC-BETA,
>>> LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>
>>> Externals/SPEC/CINT2006/445_gobmk/445_gobmk [JIT, CBE, JIT codegen, LLC,
>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>
>>> MultiSource/Applications/JM/ldecod/ldecod [JIT, CBE, JIT codegen, LLC,
>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>
>>> MultiSource/Benchmarks/Prolangs-C/unix-tbl/unix-tbl [JIT, CBE, JIT
>>> codegen,
>>> LLC, LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>
>>> Added Tests:
>>>
>>> Removed Tests:
>>>
>>> Significant Changes in Test Results:
>>>
>>> LLC-BETA compile:
>>>
>>>  Externals/SPEC/CFP2000/177_mesa/177_mesa: -32.20% (7.5199 => 5.0983)
>>>
>>>  Externals/SPEC/CFP2000/188_ammp/188_ammp: -9.90% (1.5340 => 1.3821)
>>>
>>>  Externals/SPEC/CFP2006/433_milc/433_milc: -10.86% (0.8879 => 0.7915)
>>>
>>>  Externals/SPEC/CINT2006/464_h264ref/464_h264ref: -6.50% (5.9258 =>
>>> 5.5404)
>>>
>>>  MultiSource/Applications/JM/lencod/lencod: -11.97% (7.7828 => 6.8510)
>>>
>>>  MultiSource/Applications/SPASS/SPASS: -12.42% (6.5489 => 5.7352)
>>>
>>>  MultiSource/Applications/hbd/hbd: -9.01% (0.4473 => 0.4070)
>>>
>>>  MultiSource/Benchmarks/FreeBench/pifft/pifft: -37.28% (0.5788 => 0.3630)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg: -7.48%
>>> (0.9996
>>> => 0.9248)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame: -15.16%
>>> (1.4610
>>> => 1.2395)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset:
>>> -14.09%
>>> (7.1303 => 6.1255)
>>>
>>>  MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm: -5.30% (0.3097
>>> =>
>>> 0.2933)
>>>
>>>  MultiSource/Benchmarks/PAQ8p/paq8p: -71.58% (3.3815 => 0.9611)
>>>
>>>  MultiSource/Benchmarks/Prolangs-C/gnugo/gnugo: -56.70% (0.4649 =>
>>> 0.2013)
>>>
>>>  MultiSource/Benchmarks/VersaBench/dbms/dbms: -12.53% (0.2346 => 0.2052)
>>>
>>>  MultiSource/Benchmarks/mafft/pairlocalalign: -11.51% (1.9353 => 1.7125)
>>>
>>>  MultiSource/Benchmarks/mediabench/gsm/toast/toast: -7.11% (0.3093 =>
>>> 0.2873)
>>>
>>>  MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg: -6.40% (0.8172 =>
>>> 0.7649)
>>>
>>>  MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4: -8.14% (5.2939 => 4.8631)
>>>
>>> JIT:
>>>
>>>  Externals/SPEC/CINT2000/176_gcc/176_gcc: 19.98% (17.1200 => 20.5400)
>>>
>>>  Externals/SPEC/CINT2000/252_eon/252_eon: 8.06% (2.1100 => 2.2800)
>>>
>>>  Externals/SPEC/CINT2000/253_perlbmk/253_perlbmk: 55.43% (12.5200 =>
>>> 19.4600)
>>>
>>>  Externals/SPEC/CINT2000/300_twolf/300_twolf: 5.48% (6.0200 => 6.3500)
>>>
>>>  Externals/SPEC/CINT2006/400_perlbench/400_perlbench: 76.58% (17.1200 =>
>>> 30.2300)
>>>
>>>  Externals/SPEC/CINT2006/401_bzip2/401_bzip2: 5.18% (3.8600 => 4.0600)
>>>
>>>  Externals/SPEC/CINT2006/464_h264ref/464_h264ref: 5.05% (26.1600 =>
>>> 27.4800)
>>>
>>>  MultiSource/Applications/ClamAV/clamscan: 10.80% (4.2600 => 4.7200)
>>>
>>>  MultiSource/Applications/JM/lencod/lencod: 5.57% (14.9100 => 15.7400)
>>>
>>>  MultiSource/Applications/SPASS/SPASS: 58.90% (12.3600 => 19.6400)
>>>
>>>  MultiSource/Applications/d/make_dparser: 21.09% (1.2800 => 1.5500)
>>>
>>>  MultiSource/Applications/hbd/hbd: 7.89% (0.3800 => 0.4100)
>>>
>>>  MultiSource/Applications/lemon/lemon: 27.06% (153.3700 => 194.8700)
>>>
>>>  MultiSource/Applications/oggenc/oggenc: 9.93% (1.4100 => 1.5500)
>>>
>>>  MultiSource/Applications/treecc/treecc: 11.63% (0.4300 => 0.4800)
>>>
>>>  MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000: 7.53% (7.9700 =>
>>> 8.5700)
>>>
>>>  MultiSource/Benchmarks/MallocBench/espresso/espresso: 5.96% (2.3500 =>
>>> 2.4900)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg: 5.56%
>>> (0.5400
>>> => 0.5700)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame: 12.06%
>>> (1.4100
>>> => 1.5800)
>>>
>>>  MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset: 16.46%
>>> (5.6500 => 6.5800)
>>>
>>>  MultiSource/Benchmarks/Prolangs-C/TimberWolfMC/timberwolfmc: 27.66%
>>> (0.4700
>>> => 0.6000)
>>>
>>>  MultiSource/Benchmarks/Prolangs-C/assembler/assembler: 25.00% (0.2800 =>
>>> 0.3500)
>>>
>>>  MultiSource/Benchmarks/Prolangs-C/bison/mybison: 35.06% (0.7700 =>
>>> 1.0400)
>>>
>>>  MultiSource/Benchmarks/Ptrdist/bc/bc: 6.10% (1.6400 => 1.7400)
>>>
>>>  MultiSource/Benchmarks/Ptrdist/yacr2/yacr2: 11.84% (1.5200 => 1.7000)
>>>
>>>  MultiSource/Benchmarks/mediabench/gsm/toast/toast: 12.00% (0.2500 =>
>>> 0.2800)
>>>
>>>  MultiSource/Benchmarks/mediabench/mpeg2/mpeg2dec/mpeg2decode: 8.51%
>>> (0.4700
>>> => 0.5100)
>>>
>>>  SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding: 6.53%
>>> (2.9100 => 3.1000)
>>>
>>> LLC compile:
>>>
>>>  MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm: -5.16% (0.2943
>>> =>
>>> 0.2791)
>>>
>>> LLC-BETA:
>>>
>>>  MultiSource/Benchmarks/FreeBench/mason/mason: 13.64% (0.2200 => 0.2500)
>>>
>>>  MultiSource/Benchmarks/Trimaran/enc-pc1/enc-pc1: -48.19% (0.8300 =>
>>> 0.4300)
>>>
>>>  SingleSource/Benchmarks/Dhrystone/dry: 22.22% (0.4500 => 0.5500)
>>>
>>>  SingleSource/Benchmarks/Misc/fbench: -9.92% (2.4200 => 2.1800)
>>>
>>>  SingleSource/UnitTests/Vector/SSE/sse_stepfft: -5.26% (0.5700 => 0.5400)
>>>
>>> CBE:
>>>
>>>  SingleSource/Benchmarks/Dhrystone/fldry: 6.78% (1.1800 => 1.2600)
>>>
>>> _______________________________________________
>>>
>>> llvm-testresults mailing list
>>>
>>> llvm-testresults at cs.uiuc.edu
>>>
>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults
>>>
>>>
>>>
>
>