[LLVMdev] LNT ClamAV - Sorting output

[Renato Golin]

On 19 April 2013 17:48, Török Edwin <edwin at etorok.net> wrote:

> Otherwise what might seem like a 20% improvement
> could very well be just a 0.2% improvement in practice.
>

This is (maybe to a lesser extent) what happens with most of our
benchmarks, and running them 3 times doesn't add that much confidence but
makes it run much slower. I end up treating the test-suite as functionality
and correctness test, rather than useful benchmark data.

I agree it would be great to have a decent benchmark infrastructure for
LLVM, but I'm not sure the test-suite is the appropriate place. Maybe a
different type of run that crank the inputs up to 11 and let the
applications run for longer, to be run once a week or so wouldn't be a bad
idea, though.

cheers,
--renato
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130419/2bbe41ef/attachment.html>