[LLVMdev] ARM LNT test-suite Buildbot

[Arnold Schwaighofer]

Hi Renato,

I noticed the bot yesterday. Thanks for working on this!

On Feb 19, 2013, at 7:45 AM, Renato Golin <renato.golin at linaro.org> wrote:

> Hi Folks,
> 
> Looks like our LNT ARM buildbot with the vectorizer is running and producing good results.

Do you have a base run with vectorization turned off? So we could see where we are degrading things?

When you say good results, I take it you mean successfully completing the test, not execution time of the resulting binary? Or did you do an analysis of performance, too?

> There are only 11 failures:
> 
> FAIL: MultiSource/Applications/Burg/burg.execution_time (1 of 1104)
> FAIL: MultiSource/Applications/ClamAV/clamscan.execution_time (2 of 1104)
> FAIL: MultiSource/Applications/lemon/lemon.execution_time (3 of 1104)
> FAIL: MultiSource/Applications/sqlite3/sqlite3.execution_time (4 of 1104)
> FAIL: MultiSource/Benchmarks/McCat/12-IOtest/iotest.execution_time (5 of 1104)
> FAIL: MultiSource/Benchmarks/MiBench/automotive-bitcount/automotive-bitcount.execution_time (6 of 1104)
> FAIL: MultiSource/Benchmarks/MiBench/telecomm-FFT/telecomm-fft.execution_time (7 of 1104)
> FAIL: MultiSource/Benchmarks/Ptrdist/anagram/anagram.execution_time (8 of 1104)
> FAIL: MultiSource/Benchmarks/TSVC/Reductions-flt/Reductions-flt.execution_time (9 of 1104)
> FAIL: SingleSource/Benchmarks/BenchmarkGame/puzzle.execution_time (10 of 1104)
> FAIL: SingleSource/Benchmarks/Shootout-C++/except.execution_time (11 of 1104)
> 
> Plus 8 Exception Handling "expected" failures, since EHABI is not working yet:
> 
> FAIL: SingleSource/Regression/C++/EH/ctor_dtor_count.execution_time (12 of 1104)
> FAIL: SingleSource/Regression/C++/EH/ctor_dtor_count-2.execution_time (13 of 1104)
> FAIL: SingleSource/Regression/C++/EH/exception_spec_test.execution_time (14 of 1104)
> FAIL: SingleSource/Regression/C++/EH/function_try_block.execution_time (15 of 1104)
> FAIL: SingleSource/Regression/C++/EH/inlined_cleanup.execution_time (16 of 1104)
> FAIL: SingleSource/Regression/C++/EH/simple_rethrow.execution_time (17 of 1104)
> FAIL: SingleSource/Regression/C++/EH/simple_throw.execution_time (18 of 1104)
> FAIL: SingleSource/Regression/C++/EH/throw_rethrow_test.execution_time (19 of 1104)
> 
> I'll be investigating each one of the 11 failures during the next months alongside other projects, so I'm not sure I can get them green by the next release, 3.3. But with the check-all buildbots green and fast, at least we can make sure we won't regress from now on.
> 
> About exception handling, we should have at least a roadmap. I believe we should tackle them after the 11 generic bugs, and do it in one big sprint. Anton, I'll need your help here.
> 
> I also want to increase the coverage (more bots, different types of bots, etc) in the near future. I hope to make ARM a certifiable first-class target, especially during releases.
> 
> I'd like to thank Galina and David for the their support in setting the bots up and testing all possibilities.
> 
> Now, on with the hard work... ;)

To me setting up build infrastructure *is* the hard work ;)

Thanks,
Arnold