[LLVMdev] [RFC] Benchmarking subset of the test suite

[Andrew Trick]

On May 21, 2014, at 1:57 PM, Evan Cheng <evan.cheng at apple.com> wrote:

> 
> On May 21, 2014, at 4:29 AM, Hal Finkel <hfinkel at anl.gov> wrote:
> 
>> ----- Original Message -----
>>> From: "Evan Cheng" <evan.cheng at apple.com>
>>> To: "Hal Finkel" <hfinkel at anl.gov>
>>> Cc: "LLVM Developers Mailing List" <llvmdev at cs.uiuc.edu>
>>> Sent: Monday, May 19, 2014 3:35:45 PM
>>> Subject: Re: [LLVMdev] [RFC] Benchmarking subset of the test suite
>>> 
>>> 
>>> 
>>> 
>>> On May 4, 2014, at 5:39 AM, Hal Finkel < hfinkel at anl.gov > wrote:
>>> 
>>> 
>>> security-blowfish
>>> 
>>> Sorry that I'm late to this. I scanned the list you provided and it
>>> makes sense to me. But this one caught me by surprise. I remember we
>>> put quite a bit of effort tuning the scheduler to improve codegen
>>> quality of blowfish. Are you sure this particular benchmark isn't
>>> serving its purpose?
>> 
>> Yes. As currently configured, the running time is much too short. On PPC64, the running time is <= 0.001s, and -O3 and -O0 cannot be statistically distinguished. On x86-64, the running time is even smaller (timeit reports 0.0000 for -O3 builds).
> 
> Can we fix the short running time? Unless there is another blowfish benchmark in the suite, I would like to keep it.

FYI, the version that we tuned is available in the public domain here: http://www.openwall.com/crypt/

It’s quite different than the one in MiBench. The feature that was interesting to us at the time was the encryption of two blocks in parallel.

The v1.2 code looks like it has a testing harness that could be adapted as a benchmark. 

http://llvm.org/bugs/show_bug.cgi?id=19860

-Andy

> 
> Evan
> 
>> 
>> -Hal
>> 
>>> 
>>> 
>>> Evan
>> 
>> -- 
>> Hal Finkel
>> Assistant Computational Scientist
>> Leadership Computing Facility
>> Argonne National Laboratory
> 
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140526/1559740c/attachment.html>