[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks

[Olivier Meurant]

Hi everyone,
I have replayed the "unix benchmark v3.6 dhrystone 2" test.

You can find the archive of the test here :
http://www.phoronix-test-suite.com/benchmark-files/byte-benchmark-1.tar.gz
To play the test on gcc :
make
./Run dhry2
To play the test on llvm-gcc :
Replace in Makefile : CC=gcc by CC=llvm-gcc
in Run : CC=gcc by CC=llvm-gcc
make
./Run dhry2

Some information on the test platform :

cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.04
DISTRIB_CODENAME=jaunty
DISTRIB_DESCRIPTION="Ubuntu 9.04"

uname -a
Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC 2009
i686 GNU/Linux

/proc/cpuinfo
dual core Intel(R) Core(TM)2 CPU X6800  @ 2.93GHz (cache size 4096)

llvm-gcc -v
Using built-in specs.
Target: i686-pc-linux-gnu
Configured with: ../llvm-gcc4.2-2.5.source/
configure
--prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install
--program-prefix=llvm-
--enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/
--disable-bootstrap --enable-languages=c,c++,fortran
Thread model: posix
gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
(Binary from official x86 2.5 release)

gcc -v
Using built-in specs.
Target: i486-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.3.3-5ubuntu4' --with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared
--with-system-zlib --libexecdir=/usr/lib --without-included-gettext
--enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr
--enable-targets=all --with-tune=generic --enable-checking=release
--build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu
Thread model: posix
gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)

The test displays its results in form of a number with unit seems to be
"loops per seconds". So it's logical to say that the bigger is better.
I run the test 10 times for each compiler (for an approximate duration of 20
minutes each compiler).

Results:
LLVM average : 10696687.14
GCC average : 9563046.99
LLVM is on average 10.58% faster than GCC on this test.

If you need more informations or if you think my test was not correctly
made... ping me. :)

Here a quick report on the test. (shows individual result of each test) :


LLVM GCC Difference Difference %




 Run 1 10720897.7 9875431 845466.7 7.89  Run 2 10411474.6 9666138.5 745336.1
7.16  Run 3 10580705.2 9265904.6 1314800.6 12.43  Run 4 10746408 9539499.9
1206908.1 11.23  Run 5 10818845.5 9766210.7 1052634.8 9.73  Run 6 10904404.3
10034620.8 869783.5 7.98  Run 7 10861482.8 9028718.6 1832764.2 16.87  Run 8
10480568.5 9568849.4 911719.1 8.7  Run 9 10857968.2 9306331.6 1551636.6
14.29  Run 10 10584116.6 9578764.8 1005351.8 9.5




 Average 10696687.14 9563046.99 1133640.15 10.58  Standard deviation
172714.92 300445.48 345992.26 3.13
Cheers,
Olivier.


On Mon, Sep 14, 2009 at 10:02 PM, Keir Mierle <mierle at gmail.com> wrote:
>
> On Mon, Sep 14, 2009 at 12:23 PM, Renato Golin <rengolin at systemcall.org>
wrote:
>>
>> 2009/9/14 Edward O'Callaghan <eocallaghan at auroraux.org>:
>> > screw that site, its useless info run by a linux gnu zealot.
>>
>> Well, being a Linux GNU zealot doesn't invalidate numbers. What does
>> invalidate is that he has no numbers!
>>
>> He just fired a few applications and counted the seconds, this is
>> utterly pointless. No reference to how many times he ran, standard
>> deviations, possible effects of OS cache, compilation parameters, etc,
>> etc. There simply isn't anything in there. It even lack text in
>> between the charts, a conclusion, and... well... everything.
>>
>> "Ignore that site", would also be more polite... ;)
>
> Nevertheless, the site has a large audience. From a publicity perspective,
it may be worthwhile to work with the author to get more representative
numbers.
> Keir
>
>>
>> cheers,
>> --renato
>>
>> Reclaim your digital rights, eliminate DRM, learn more at
>> http://www.defectivebydesign.org/what_is_drm
>> _______________________________________________
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090915/edc5ef63/attachment.html>