[LLVMdev] FYI: Phoronix GCC vs. LLVM-GCC benchmarks
Olivier Meurant
meurant.olivier at gmail.com
Wed Sep 16 04:37:06 PDT 2009
Some additional tests :
With : -O2 -march=core2 for both gcc and llvm-gcc : LLVM is better with 10%.
LLVM GCC Difference Difference %
Run 1 13771597.2 13105010.6 666586.6 4.84
Run 2 13813420.8 12536327.1 1277093.7 9.25
Run 3 13769573.8 12124207.3 1645366.5 11.95
Run 4 13883222.6 12794159.1 1089063.5 7.84
Run 5 13890414.6 12105946 1784468.6 12.85
Run 6 13784627.3 11648245 2136382.3 15.5
Run 7 13920748.9 11781692.4 2139056.5 15.37
Run 8 13816843.9 12460451.3 1356392.6 9.82
Run 9 13864682.1 12959671.4 905010.7 6.53
Run 10 13849863.4 12964021.5 885841.9 6.4
Average 13836499.46 12447973.17 1388526.29 10.03
Standard deviation 53189.13 515638.56
522400.98 3.77
With : -O3 -march=core2 for both compiler : GCC is better with 5.75%
LLVM GCC Difference Difference %
Run 1 13790728.3 14902550.9 -1111822.6 -8.06
Run 2 13840499.3 14759093.2 -918593.9 -6.64
Run 3 13975018.5 14545983.5 -570965 -4.09
Run 4 13853799.5 15126446.4 -1272646.9 -9.19
Run 5 13993633.5 14824100.4 -830466.9 -5.93
Run 6 13903645.3 12768793.8 1134851.5 8.16
Run 7 13816401.7 15003783.8 -1187382.1 -8.59
Run 8 13735232.8 14976854.3 -1241621.5 -9.04
Run 9 13902380.7 14778159.5 -875778.8 -6.3
Run 10 13936761.8 15022553.5 -1085791.7 -7.79
Average 13874810.14 14670831.93 -796021.79 -5.75
Standard deviation 82443.34 688446.69
712123.9 5.14
Olivier.
2009/9/16 Török Edwin <edwintorok at gmail.com>
> On 2009-09-16 11:42, Olivier Meurant wrote:
> > On Tue, Sep 15, 2009 at 5:35 PM, David Greene <dag at cray.com
> > <mailto:dag at cray.com>> wrote:
> >
> >
> > Remember, the goal here isn't to show how great LLVM is. It's to
> > get an
> > honest assessment of where we are at. Phoronix did us a big
> > favor. Getting
> > more details about his tests would help us even more.
> >
> >
> > It's exactly what I'm trying to do. I'm not even a LLVM contributor,
> > so I don't feel responsible for a good or bad performances of LLVM. :)
> > Anyway, thanks for pointing this target differences. After some hours
> > at recompiling gcc, here we are :
> >
> > Some information on the test platform :
> >
> > cat /etc/lsb-release
> > DISTRIB_ID=Ubuntu
> > DISTRIB_RELEASE=9.04
> > DISTRIB_CODENAME=jaunty
> > DISTRIB_DESCRIPTION="Ubuntu 9.04"
> >
> > uname -a
> > Linux zaraki 2.6.28-15-generic #49-Ubuntu SMP Tue Aug 18 18:40:08 UTC
> > 2009 i686 GNU/Linux
> >
> > /proc/cpuinfo
> > dual core Intel(R) Core(TM)2 CPU X6800 @ 2.93GHz (cache size 4096)
> >
> > llvm-gcc -v
> > Using built-in specs.
> > Target: i686-pc-linux-gnu
> > Configured with: ../llvm-gcc4.2-2.5.source/configure
> >
> --prefix=/mounts/zion/disks/0/localhome/tbrethou/2.5/prerelease2/llvm-gcc/obj/../install
> > --program-prefix=llvm-
> > --enable-llvm=/localhome/tbrethou/2.5/prerelease2/llvm-2.5/
> > --disable-bootstrap --enable-languages=c,c++,fortran
> > Thread model: posix
> > gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build 2.5)
> > (Binary from official x86 2.5 release)
> >
> > gcc -v
> > Using built-in specs.
> > Target: i686-pc-linux-gnu
> > Configured with: ./configure --prefix=/usr/local
> > Thread model: posix
> > gcc version 4.4.1 (GCC)
>
> Phoronix tested with gcc 4.2.1 (Apple's).
>
> >
> > The compilation line are :
> > gcc -o bin src.c -O3
> > llvm-gcc -o bin src.c -O3
>
> The Makefile uses -O by default, perhaps that is what Phoronix used too.
>
> >
> >
> > Results :
> > LLVM average : 13553161.95
> > gcc average : 14624441.45
> > Average : gcc is better with 8%
> >
> > The full report :
> > "LLVM" "GCC" "Difference" "Difference %"
> >
> > "Run 1" 13842616 14693115.8 -850499.8 -6.14
> > "Run 2" 13773202.9 14785734.9 -1012532 -7.35
> > "Run 3" 13671309.3 14985114 -1313804.7 -9.61
> > "Run 4" 12916939 14640392.9 -1723453.9 -13.34
> > "Run 5" 13572133.3 14425575 -853441.7 -6.29
> > "Run 6" 13810096.5 14694240.6 -884144.1 -6.4
> > "Run 7" 13427435.8 14695094.6 -1267658.8 -9.44
> > "Run 8" 14056659.9 14468968.4 -412308.5 -2.93
> > "Run 9" 13699125.3 14373360.9 -674235.6 -4.92
> > "Run 10 12762101.5 14482817.4 -1720715.9 -13.48
> >
> > "Average" 13553161.95 14624441.45 -1071279.5 -7.99
> > "Standard deviation" 412810.77 187898.68
> > 430895.31 3.46
> >
> > Is there again something wrong on this test ?
>
> No, it seems fine.
>
> I get different results on x86-64 though (with -O), the difference is
> very small (4% with 3% stddev, or 3% with 1.5% stddev).
>
> Maybe you should try to reproduce one of the other benchmarks that show
> a higher difference, according to phoronix JOhn the Ripper shown 40%
> performance drop, there was also a huge drop for blowfish.
>
>
>
> GCC 4.4.1 GCC 4.2.4 LLVM-GCC 2.5 LLVM-GCC 2.6
> GCC-4.4/LLVM2.5
> GCC-4.4/LLVM2.6 GCC-4.4/GCC-4.2
>
> 144663277 139072037 154747143 151467440
> 0.935 0.955 1.040
>
> 144413249 139042455 150949846 150322055
> 0.957 0.961 1.039
>
> 144915166 138131283 150630261 150650540
> 0.962 0.962 1.049
>
> 145602711 138587982 154993781 147281539
> 0.939 0.989 1.051
>
> 145547868 139224734 150650813 147998859
> 0.966 0.983 1.045
>
> 145719970 139224734 144055081 152474829
> 1.012 0.956 1.047
> average 145143706.83 138880537.5 151004487.5
> 150032543.67 0.962
> 0.968 1.045
> stdev 551855.54 435155.79 3961926.31 2009642.82
> 0.027 0.015 0.005
>
>
> Using built-in specs.
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v --with-pkgversion='Debian 4.4.1-4'
> --with-bugurl=file:///usr/share/doc/gcc-4.4/README.Bugs
> --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
> --enable-shared --enable-multiarch --enable-linker-build-id
> --with-system-zlib --libexecdir=/usr/lib --without-included-gettext
> --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.4
> --program-suffix=-4.4 --enable-nls --enable-clocale=gnu
> --enable-libstdcxx-debug --enable-mpfr --enable-objc-gc
> --with-arch-32=i486 --with-tune=generic --enable-checking=release
> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 4.4.1 (Debian 4.4.1-4)
>
> llvm2.5:
> Using built-in specs.
> Target: x86_64-unknown-linux-gnu
> Configured with: ../llvm-gcc-4.2/configure --program-prefix=llvm-
> --enable-llvm=/home/edwin/llvm2.5/obj --enable-languages=c,c++,fortran
> Thread model: posix
> gcc version 4.2.1 (Based on Apple Inc. build 5636) (LLVM build)
>
> llvm2.6pre1:
> Using built-in specs.
> Target: x86_64-unknown-linux-gnu
> Configured with: ../llvm-gcc4.2-2.6.source/configure
> --program-prefix=llvm- --enable-languages=c,c++,fortran
> --enable-llvm=/home/edwin/llvm2.6/obj/
> Thread model: posix
> gcc version 4.2.1 (Based on Apple Inc. build 5649) (LLVM build)
>
> Using built-in specs.
> Target: x86_64-linux-gnu
> Configured with: ../src/configure -v
> --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
> --enable-shared --with-system-zlib --libexecdir=/usr/lib
> --without-included-gettext --enable-threads=posix --enable-nls
> --with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2
> --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
> --enable-mpfr --with-tune=generic --enable-checking=release
> --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
> Thread model: posix
> gcc version 4.2.4 (Debian 4.2.4-6)
>
>
> Best regards,
> --Edwin
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20090916/b0ee0726/attachment.html>
More information about the llvm-dev
mailing list