[LLVMdev] ARM LNT test-suite Buildbot

[JF Bastien]

Hi Renato,

I'm playing with A15 bots too (running Ubuntu). This is probably what you
want to have predictable performance:

Disable auto-resetting the CPU scaling governor to ondemand:
  sudo update-rc.d -f ondemand remove

Then add this to /etc/rc.local:
  # Disable power management.
  for cpu in `find /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor`;
do
    echo performance > $cpu
  done



On Tue, Feb 19, 2013 at 9:24 AM, David Blaikie <dblaikie at gmail.com> wrote:

>
>
>
> On Tue, Feb 19, 2013 at 7:36 AM, Renato Golin <renato.golin at linaro.org>wrote:
>
>> On 19 February 2013 15:16, Arnold Schwaighofer <aschwaighofer at apple.com>wrote:
>>
>>> Do you have a base run with vectorization turned off? So we could see
>>> where we are degrading things?
>>>
>>
>> I wanted to, but after a few failed attempts, I couldn't pass the option
>> to clang to disable vectorization. I don't want to make Galina reconfig the
>> master every time, so I set up a master on my own laptop and will fiddle.
>> But the fastest way I can test, for now, is to run the LNT tests manually
>> with and without vectorization and compare.
>>
>> I'm not expecting many issues with vectorization, to be honest, but you
>> never know... ;)
>>
>>
>>
>> When you say good results, I take it you mean successfully completing the
>>> test, not execution time of the resulting binary? Or did you do an analysis
>>> of performance, too?
>>>
>>
>> Good results because this is the first public test-suite for ARM and we
>> only had 19 errors out of 1104!! And 8 of them are "expected", so it's
>> about 1% or failures.
>>
>> The non-EH problems should be either mechanical changes on tests, or
>> simple fixes in LLVM, so I'm not expecting a lot of work to get the LNT on
>> the same state on ARM than x86.
>>
>> I'm not checking performance yet, but the data is being collected here
>> http://llvm.org/perf/db_default/v4/nts/machine/10 and should give us
>> some idea on how to proceed from now on on performance measurements.
>>
>> For now, I'm interested in correctness, so I won't worry too much with
>> those numbers (I've heard I should disable some Turbo mode to make more
>> predictable results, though I only saw one test running at a time, so maybe
>> it was off).
>>
>
> Turbo is a CPU option, not a test suite execution option. Turning it off
> stops the system from varying the CPU clock based on load (when this
> feature is enabled it can be a power saving, but it results in varying
> performance - bad for perf analysis).
>
> Anotehr thing to consider disabling is Address Space Layout Randomization
> so that you get consistent hashing & other behavior run-over-run.
>
>
>>  Once we have an acceptable state (mostly green, except EH), I'll start
>> worrying about performance.
>>
>
> Sounds reasonable.
>
> - David
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130219/3945078e/attachment.html>