[LLVMdev] ARM LNT test-suite Buildbot

[David Blaikie]

On Tue, Feb 19, 2013 at 7:36 AM, Renato Golin <renato.golin at linaro.org>wrote:

> On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote:
>
>> Do you have a base run with vectorization turned off? So we could see
>> where we are degrading things?
>>
>
> I wanted to, but after a few failed attempts, I couldn't pass the option
> to clang to disable vectorization. I don't want to make Galina reconfig the
> master every time, so I set up a master on my own laptop and will fiddle.
> But the fastest way I can test, for now, is to run the LNT tests manually
> with and without vectorization and compare.
>
> I'm not expecting many issues with vectorization, to be honest, but you
> never know... ;)
>
>
>
> When you say good results, I take it you mean successfully completing the
>> test, not execution time of the resulting binary? Or did you do an analysis
>> of performance, too?
>>
>
> Good results because this is the first public test-suite for ARM and we
> only had 19 errors out of 1104!! And 8 of them are "expected", so it's
> about 1% or failures.
>
> The non-EH problems should be either mechanical changes on tests, or
> simple fixes in LLVM, so I'm not expecting a lot of work to get the LNT on
> the same state on ARM than x86.
>
> I'm not checking performance yet, but the data is being collected here
> http://llvm.org/perf/db_default/v4/nts/machine/10 and should give us some
> idea on how to proceed from now on on performance measurements.
>
> For now, I'm interested in correctness, so I won't worry too much with
> those numbers (I've heard I should disable some Turbo mode to make more
> predictable results, though I only saw one test running at a time, so maybe
> it was off).
>

Turbo is a CPU option, not a test suite execution option. Turning it off
stops the system from varying the CPU clock based on load (when this
feature is enabled it can be a power saving, but it results in varying
performance - bad for perf analysis).

Anotehr thing to consider disabling is Address Space Layout Randomization
so that you get consistent hashing & other behavior run-over-run.


> Once we have an acceptable state (mostly green, except EH), I'll start
> worrying about performance.
>

Sounds reasonable.

- David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/d0bf5b28/attachment.html>