[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

[Renato Golin]

2009/9/16 Olivier Meurant <meurant.olivier at gmail.com>:
>         Average 13836499.46     12447973.17     1388526.29      10.03
>         Standard deviation      53189.13        515638.56
> 522400.98       3.77


That was pretty much what I was expecting from the article... numbers.
It doesn't matter who's best, you can't be best in all areas, but
profiling must be done right. The standard deviation is, at least, one
order of magnitude lower than the difference between the two averages,
which means the measurements have a meaning and express some reality.

There is little value in the results on the article. People had to
re-do it properly again, anyway...

Shouldn't it be part of the standard release? I mean, add profiling as
an automated task before every big release, comparing to previous
versions of LLVM and other important compilers. Not to be waving the
results about, but to know the weakness and work on them on the next
release. It might take a while to build such infrastructure, but it's
a good thing to do, I guess.

cheers,
--renato

Reclaim your digital rights, eliminate DRM, learn more at
http://www.defectivebydesign.org/what_is_drm