[LLVMdev] buildbot with -vectorize

[Tobias Grosser]

On 06/24/2012 02:42 PM, Hal Finkel wrote:
> On Sun, 24 Jun 2012 08:17:32 +0200
> Tobias Grosser<tobias at grosser.es>  wrote:
>
>> On 06/24/2012 05:42 AM, Hal Finkel wrote:
>>> On Thu, 21 Jun 2012 16:25:13 +0200
>>> Tobias Grosser<tobias at grosser.es>   wrote:
>>>
>>>> On 06/21/2012 04:23 PM, Hal Finkel wrote:
>>>>> On Thu, 21 Jun 2012 01:03:46 +0200
>>>>> Tobias Grosser<tobias at grosser.es>    wrote:
>>>>>
>>>>>> On 06/20/2012 11:07 PM, Hal Finkel wrote:
>>>>>>> On Tue, 12 Jun 2012 11:15:12 +0200
>>>>>>> Tobias Grosser<tobias at grosser.es>     wrote:
>>>>>>>
>>>>>>>> On 02/05/2012 02:42 PM, Hal Finkel wrote:
>>>>>>>>> I think that it would be a good idea to have a buildbot set up
>>>>>>>>> to run the test suite with -vectorize. Could one of the
>>>>>>>>> existing machines do this also?
>>>>>>>>
>>>>>>>> It took some time, but we have this now:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize
>>>>>>>>
>>>>>>>> The first run shows that there are a couple of test-suite
>>>>>>>> failures with the vectorizer:
>>>>>>>>
>>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/108/steps/lnt.nightly-test/logs/tests.FAIL
>>>>>>>>
>>>>>>>> Hal, interested to take a look?
>>>>>>>
>>>>>>> If I'm looking at this correctly, there are now no failures.
>>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test
>>>>>>
>>>>>> I believe there are:
>>>>>>
>>>>>> http://lab.llvm.org:8011/builders/clang-x86_64-darwin10-nt-O3-vectorize/builds/220/steps/lnt.nightly-test/logs/tests.XFAIL
>>>>>>
>>>>>> They are just automatically XFAILED after they occur a second
>>>>>> time. ;-)
>>>>>
>>>>> OIC... then that's not good...
>>>>>
>>>>> So it seems that prior the build 108 the testing process always
>>>>> timed out (it looks like it had a timeout of 20 minutes which was
>>>>> not long enough). At build 108, it lists those applications as
>>>>> failing.
>>>>
>>>> Yes.
>>>>
>>>>> I am pretty sure that there were no x86 test-suite failures when I
>>>>> committed the vectorizer. We'll need to investigate...
>>>>
>>>> It may be due to the different platform (darwin). The list is sort
>>>> so maybe you can try some of them on linux.
>>>
>>> Build 270 lists some of the nightly tests under improved and
>>> regressed (because I enabled vectorizing comparisons). Is there a
>>> way to see the relative timings? -- I thought that there was but
>>> I'm not seeing anything to click to see a timing history.
>>
>> Yes, have a look here: http://llvm.org/perf/db_default/v4/nts/1437
>
> Thanks! How does it choose which set of runs to call the 'baseline'?

That's the very first run. I normally ignore it as this probably was 
only run with one iteration and the timings are consequently not really 
reliable.

Tobi