[LLVMdev] LLVM benchmarks against GCC

[Chris Lattner]

On Fri, 30 Apr 2004, [koi8-r] "Valery A.Khamenya[koi8-r] "  wrote:
> > This one is just noise, if you look today it's 1.0's straight across the
> > board.  Also note that the test runs for 0.003 seconds, which is the
> > resolution of the time command on the system the program is being run on:
> > this is not a good test for checking performance.  :)
>
> yes, i saw, that the numbers are quite discrete, it is just another
> feature telling the same. Well, but who really needs insecure numbers?..
> maybe it would be a good idea to change the test a bit or make the
> same test but in a loop? otherwize it makes no sense to include them
> into benchmarking...

The nightly tester is used for two purposes: making sure that nothing
breaks (the unit tests) and keeping tabs on how well performance is doing
(the spec and most multisource tests).  It's not a reliable way to do
serious benchmarking, but can give good insights into where things can be
improved.

> > With GCC it
> > takes 9.639s to run the test, with LLVM-CBE it takes 4.3s, and with
> > LLVM-LLC-LS it takes 4.963s.  I think these are pretty good numbers.  :)
>
> oh, you are right, i like those numbers :)

:)

> > Consider this a small plea for help.  :)  Once we know what to fix, it's
> > usually pretty easy to do so, but identifying the problems takes time, and
> > we have plenty of other things we need to be doing as well.
>
> well, 90+ % of my spare time is dedicated to SuSE 9 and win32/cygwin.
> Neither of two is good for LLVM today.

What is wrong with SuSE 9?  Is it the wierd GCC ICE bug?  If so, I believe
that it only effects one .cpp file in the LLVM sources and can probably be
worked around.  I don't have a copy of the buggy compiler handy, but if
you were to look into it it would probably be a pretty easy problem to
solve.

> But me and others always have a web client! Would it be a big deal to
> add small functionality to the http://llvm.cs.uiuc.edu/demo/ page? I
> mean, why not to add output in assembler there? Then everyone could try
> to help you from anywhere. If i remember right, Misha was the one who
> prepared this nice page, could we ask Misha for that great add-on?

It would certainly be possible, but the point of the demo page is to show
how the C front-end and optimizer transform programs to LLVM code.  I
might be able to con Brian or Misha into adding support for native code
generation, but we'll have to see (they're all really busy with
end-of-semester stuff).

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/