[cfe-dev] GCC 4.6, LLVM/Clang 2.9, DragonEgg Five-System Benchmarks

[Michael Larabel]

On 03/30/2011 06:14 PM, Chris Lattner wrote:
> On Mar 30, 2011, at 12:16 PM, Michael Larabel wrote:
>>>>>> Himeno is built -O0, which makes zero sense as a performance benchmark. And you are right GraphicsMagick uses OpenMP, which means the comparison between Clang and the rest of the compilers are totally meaningless. 7-Zip builds just fine for me with Clang, I have no idea why they couldn't get it to build.
>> Yes, I've already discussed some of this with Eric in a private thread
>> the other day.
>>
>> While the latest upstream Himeno release will be pulled, which evidently
>> is now multi-threaded and uses -O3, I would argue though that in the
>> context of benchmarking defaults, even '-O0' does matter if looking at
>> the compiler performance. Such as for software projects that may not
>> specify any optimization flag, I really doubt __most users__ when
>> pulling down new code are checking what a given program is using as
>> their compiler flags.
> Hi Michael,
>
> I personally don't agree that -O0 matters in this context, but it doesn't really matter what I think :).
>
> On request though: the web site just presents numbers, and leaves it up to the reader to draw their own conclusion, as though all of the numbers were measuring the same thing and reflected on the compilers in an arbitrary way.  I think it would be a great service to your users to say things like "and we see here that clang doesn't support OpenMP, which is why it has terrible multicore performance on that benchmark" for example.
>
> Does it sound reasonable to mention things like OpenMP and -O0 performance?
>
> -Chris
>

Hi Chris,

Yes, it's certainly reasonable to include such information when it's 
brought up. I had added the OpenMP GraphicsMagick note a short while 
ago, but I've now added the Himeno note as well to avoid confusion.

Thanks for the feedback, as always.

-- Michael