[llvm-dev] Noisy benchmark results?

Wed Mar 1 16:02:59 PST 2017

It would also be good to implement re-running directly at the test-suite/litsupport level. In principle that is trivial to do, however last time I looked at it lit would only support outputting 1 value per benchmark/metric ...

> On Mar 1, 2017, at 3:36 PM, Mikhail Zolotukhin via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> 
>> On Feb 28, 2017, at 11:59 PM, Kristof Beyls <kristof.beyls at arm.com <mailto:kristof.beyls at arm.com>> wrote:
>> 
>> 
>>> On 28 Feb 2017, at 22:50, Michael Zolotukhin <mzolotukhin at apple.com <mailto:mzolotukhin at apple.com>> wrote:
>>> 
>>> I also usually rerun suspiciously improved or regressed tests to verify the performance change. Most of the time, if it was just a noise, the test doesn’t appear on another run. I wish LNT (or any other script) could do that for me :)
>>> 
>>> Michael
>> 
>> Doesn't the lnt runtest nt --rerun command line option allow you to do this?
> Hmm, I think I tried to use it in the past, and it didn't work for me for some reason - but I don't remember for sure. Maybe that's exactly what I asked for.
> 
>> If not, what is missing?I don't use this option at the moment, but it would be nice to know if it does scratch your itch or not. Also, we still need to implement that functionality for lnt runtest test-suite.
> Yeah, I'm using 'lnt runtest test-suite' now most of the time.
> 
> Thanks,
> Michael
>> 
>> Thanks,
>> 
>> Kristof
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170301/1ebb5629/attachment.html>