[llvm-testresults] Grawp nightly tester results

Evan Cheng evan.cheng at apple.com
Sat Sep 5 17:37:21 PDT 2009


On Sep 5, 2009, at 4:50 PM, Evan Cheng wrote:

>
> On Sep 5, 2009, at 4:45 PM, Daniel Dunbar wrote:
>
>> On Sat, Sep 5, 2009 at 4:11 PM, Evan Cheng<evan.cheng at apple.com>
>> wrote:
>>>
>>> On Sep 5, 2009, at 2:58 PM, Daniel Dunbar wrote:
>>>
>>>> On Fri, Sep 4, 2009 at 6:28 PM, Evan Cheng<evan.cheng at apple.com>
>>>> wrote:
>>>>>
>>>>> What JIT regressions? All of the following failures are crashing
>>>>> in the
>>>>> loop
>>>>> unswitching pass. Devang is looking into it.
>>>>
>>>> The performance regressions, a large number of JIT tests seem to
>>>> have
>>>> dropped by maybe an average of 5-10%.
>>>
>>> You're right. It looks real too. But none of the patches look
>>> suspicious.
>>
>> That's why I asked if you can remember if the JIT callback patch was
>> in this round or the previous one. Not sure if just changing the
>> compilation callback could account for these swings, but I don't
>> remember any other obvious JIT changes.
>
> No. It's not due to that change. I just tried r80904 on 253.perlbmk.
> It took 11.271s vs. TOT 18.325s.

Ok, this doesn't really make sense to me, but numbers don't lie:

80923: 12.982s
80926: 19.403s

Dan, please take a look.

Evan

>
>
> Evan
>
>>
>> - Daniel
>>
>>>>
>>>>> New Test Failures:
>>>>>
>>>>> Externals/SPEC/CFP2006/470_lbm/470_lbm [JIT, CBE, JIT codegen,  
>>>>> LLC,
>>>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>>>
>>>>> Externals/SPEC/CINT2006/403_gcc/403_gcc [JIT, JIT codegen, LLC,
>>>>> LLC-BETA,
>>>>> LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>>>
>>>>> Externals/SPEC/CINT2006/445_gobmk/445_gobmk [JIT, CBE, JIT
>>>>> codegen, LLC,
>>>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>>>
>>>>> MultiSource/Applications/JM/ldecod/ldecod [JIT, CBE, JIT codegen,
>>>>> LLC,
>>>>> LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>>>
>>>>> MultiSource/Benchmarks/Prolangs-C/unix-tbl/unix-tbl [JIT, CBE, JIT
>>>>> codegen,
>>>>> LLC, LLC-BETA, LLC compile, Bitcode, GCCAS, LLC_BETA compile]
>>>>>
>>>>> Added Tests:
>>>>>
>>>>> Removed Tests:
>>>>>
>>>>> Significant Changes in Test Results:
>>>>>
>>>>> LLC-BETA compile:
>>>>>
>>>>> Externals/SPEC/CFP2000/177_mesa/177_mesa: -32.20% (7.5199 =>
>>>>> 5.0983)
>>>>>
>>>>> Externals/SPEC/CFP2000/188_ammp/188_ammp: -9.90% (1.5340 =>
>>>>> 1.3821)
>>>>>
>>>>> Externals/SPEC/CFP2006/433_milc/433_milc: -10.86% (0.8879 =>
>>>>> 0.7915)
>>>>>
>>>>> Externals/SPEC/CINT2006/464_h264ref/464_h264ref: -6.50% (5.9258 =>
>>>>> 5.5404)
>>>>>
>>>>> MultiSource/Applications/JM/lencod/lencod: -11.97% (7.7828 =>
>>>>> 6.8510)
>>>>>
>>>>> MultiSource/Applications/SPASS/SPASS: -12.42% (6.5489 => 5.7352)
>>>>>
>>>>> MultiSource/Applications/hbd/hbd: -9.01% (0.4473 => 0.4070)
>>>>>
>>>>> MultiSource/Benchmarks/FreeBench/pifft/pifft: -37.28% (0.5788 =>
>>>>> 0.3630)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg: -7.48%
>>>>> (0.9996
>>>>> => 0.9248)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame:
>>>>> -15.16%
>>>>> (1.4610
>>>>> => 1.2395)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-typeset:
>>>>> -14.09%
>>>>> (7.1303 => 6.1255)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm: -5.30%
>>>>> (0.3097
>>>>> =>
>>>>> 0.2933)
>>>>>
>>>>> MultiSource/Benchmarks/PAQ8p/paq8p: -71.58% (3.3815 => 0.9611)
>>>>>
>>>>> MultiSource/Benchmarks/Prolangs-C/gnugo/gnugo: -56.70% (0.4649 =>
>>>>> 0.2013)
>>>>>
>>>>> MultiSource/Benchmarks/VersaBench/dbms/dbms: -12.53% (0.2346 =>
>>>>> 0.2052)
>>>>>
>>>>> MultiSource/Benchmarks/mafft/pairlocalalign: -11.51% (1.9353 =>
>>>>> 1.7125)
>>>>>
>>>>> MultiSource/Benchmarks/mediabench/gsm/toast/toast: -7.11%
>>>>> (0.3093 =>
>>>>> 0.2873)
>>>>>
>>>>> MultiSource/Benchmarks/mediabench/jpeg/jpeg-6a/cjpeg: -6.40%
>>>>> (0.8172 =>
>>>>> 0.7649)
>>>>>
>>>>> MultiSource/Benchmarks/tramp3d-v4/tramp3d-v4: -8.14% (5.2939 =>
>>>>> 4.8631)
>>>>>
>>>>> JIT:
>>>>>
>>>>> Externals/SPEC/CINT2000/176_gcc/176_gcc: 19.98% (17.1200 =>
>>>>> 20.5400)
>>>>>
>>>>> Externals/SPEC/CINT2000/252_eon/252_eon: 8.06% (2.1100 => 2.2800)
>>>>>
>>>>> Externals/SPEC/CINT2000/253_perlbmk/253_perlbmk: 55.43% (12.5200
>>>>> =>
>>>>> 19.4600)
>>>>>
>>>>> Externals/SPEC/CINT2000/300_twolf/300_twolf: 5.48% (6.0200 =>
>>>>> 6.3500)
>>>>>
>>>>> Externals/SPEC/CINT2006/400_perlbench/400_perlbench: 76.58%
>>>>> (17.1200 =>
>>>>> 30.2300)
>>>>>
>>>>> Externals/SPEC/CINT2006/401_bzip2/401_bzip2: 5.18% (3.8600 =>
>>>>> 4.0600)
>>>>>
>>>>> Externals/SPEC/CINT2006/464_h264ref/464_h264ref: 5.05% (26.1600 =>
>>>>> 27.4800)
>>>>>
>>>>> MultiSource/Applications/ClamAV/clamscan: 10.80% (4.2600 =>
>>>>> 4.7200)
>>>>>
>>>>> MultiSource/Applications/JM/lencod/lencod: 5.57% (14.9100 =>
>>>>> 15.7400)
>>>>>
>>>>> MultiSource/Applications/SPASS/SPASS: 58.90% (12.3600 => 19.6400)
>>>>>
>>>>> MultiSource/Applications/d/make_dparser: 21.09% (1.2800 => 1.5500)
>>>>>
>>>>> MultiSource/Applications/hbd/hbd: 7.89% (0.3800 => 0.4100)
>>>>>
>>>>> MultiSource/Applications/lemon/lemon: 27.06% (153.3700 =>
>>>>> 194.8700)
>>>>>
>>>>> MultiSource/Applications/oggenc/oggenc: 9.93% (1.4100 => 1.5500)
>>>>>
>>>>> MultiSource/Applications/treecc/treecc: 11.63% (0.4300 => 0.4800)
>>>>>
>>>>> MultiSource/Benchmarks/ASCI_Purple/SMG2000/smg2000: 7.53%
>>>>> (7.9700 =>
>>>>> 8.5700)
>>>>>
>>>>> MultiSource/Benchmarks/MallocBench/espresso/espresso: 5.96%
>>>>> (2.3500 =>
>>>>> 2.4900)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-jpeg/consumer-jpeg: 5.56%
>>>>> (0.5400
>>>>> => 0.5700)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-lame/consumer-lame: 12.06%
>>>>> (1.4100
>>>>> => 1.5800)
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/consumer-typeset/consumer-
>>>>> typeset: 16.46%
>>>>> (5.6500 => 6.5800)
>>>>>
>>>>> MultiSource/Benchmarks/Prolangs-C/TimberWolfMC/timberwolfmc:
>>>>> 27.66%
>>>>> (0.4700
>>>>> => 0.6000)
>>>>>
>>>>> MultiSource/Benchmarks/Prolangs-C/assembler/assembler: 25.00%
>>>>> (0.2800 =>
>>>>> 0.3500)
>>>>>
>>>>> MultiSource/Benchmarks/Prolangs-C/bison/mybison: 35.06% (0.7700 =>
>>>>> 1.0400)
>>>>>
>>>>> MultiSource/Benchmarks/Ptrdist/bc/bc: 6.10% (1.6400 => 1.7400)
>>>>>
>>>>> MultiSource/Benchmarks/Ptrdist/yacr2/yacr2: 11.84% (1.5200 =>
>>>>> 1.7000)
>>>>>
>>>>> MultiSource/Benchmarks/mediabench/gsm/toast/toast: 12.00%
>>>>> (0.2500 =>
>>>>> 0.2800)
>>>>>
>>>>> MultiSource/Benchmarks/mediabench/mpeg2/mpeg2dec/mpeg2decode:
>>>>> 8.51%
>>>>> (0.4700
>>>>> => 0.5100)
>>>>>
>>>>> SingleSource/Benchmarks/Adobe-C++/simple_types_constant_folding:
>>>>> 6.53%
>>>>> (2.9100 => 3.1000)
>>>>>
>>>>> LLC compile:
>>>>>
>>>>> MultiSource/Benchmarks/MiBench/telecomm-gsm/telecomm-gsm: -5.16%
>>>>> (0.2943
>>>>> =>
>>>>> 0.2791)
>>>>>
>>>>> LLC-BETA:
>>>>>
>>>>> MultiSource/Benchmarks/FreeBench/mason/mason: 13.64% (0.2200 =>
>>>>> 0.2500)
>>>>>
>>>>> MultiSource/Benchmarks/Trimaran/enc-pc1/enc-pc1: -48.19% (0.8300
>>>>> =>
>>>>> 0.4300)
>>>>>
>>>>> SingleSource/Benchmarks/Dhrystone/dry: 22.22% (0.4500 => 0.5500)
>>>>>
>>>>> SingleSource/Benchmarks/Misc/fbench: -9.92% (2.4200 => 2.1800)
>>>>>
>>>>> SingleSource/UnitTests/Vector/SSE/sse_stepfft: -5.26% (0.5700 =>
>>>>> 0.5400)
>>>>>
>>>>> CBE:
>>>>>
>>>>> SingleSource/Benchmarks/Dhrystone/fldry: 6.78% (1.1800 => 1.2600)
>>>>>
>>>>> _______________________________________________
>>>>>
>>>>> llvm-testresults mailing list
>>>>>
>>>>> llvm-testresults at cs.uiuc.edu
>>>>>
>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults
>>>>>
>>>>>
>>>>>
>>>
>>>
>
> _______________________________________________
> llvm-testresults mailing list
> llvm-testresults at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-testresults




More information about the llvm-testresults mailing list