[LLVMdev] LLVM benchmarks against GCC

[Chris Lattner]

On Fri, 30 Apr 2004, [koi8-r] "Valery A.Khamenya[koi8-r] "  wrote:

> > The nightly tester is used for two purposes: making sure that nothing
> > breaks (the unit tests) and keeping tabs on how well performance is doing
> > (the spec and most multisource tests).  It's not a reliable way to do
> > serious benchmarking, but can give good insights into where things can be
> > improved.
>
> hm, one day, the great benchmarking will be a reason to use LLVM,
> so, i think a bit more benchmarking info could be fruitful.
> If test might be enveloped in a loop to get run about few
> seconds then it makes hardly nightly testings much slower,
> but will bring a lot of interesting info.

Definitely, it just takes time to do all of these things.  Besides, most
of the unit tests are entirely artificial and not-interesting for
benchmarking purposes.  Here are some examples:

http://llvm.cs.uiuc.edu/cvsweb/cvsweb.cgi/llvm/test/Programs/SingleSource/UnitTests/2003-05-31-CastToBool.c?rev=1.3&content-type=text/x-cvsweb-markup
http://llvm.cs.uiuc.edu/cvsweb/cvsweb.cgi/llvm/test/Programs/SingleSource/UnitTests/2004-02-02-NegativeZero.c?rev=1.1&content-type=text/x-cvsweb-markup

Doing more performance tuning is certainly important though, it's just
that other projects have taken priority.  There are *so* many good things
that can be done in a compiler, especially in one as multifaceted as LLVM.
:)

> > What is wrong with SuSE 9?  Is it the wierd GCC ICE bug?  If so, I believe
> > that it only effects one .cpp file in the LLVM sources and can probably be
> > worked around.  I don't have a copy of the buggy compiler handy, but if
> > you were to look into it it would probably be a pretty easy problem to
> > solve.
>
> yesterday I got new SuSE 9.1 DVD, so i am going to enter this
> river again. Perhaps, this time all will be fine.

Sounds great, please let me know how it goes.

> > It would certainly be possible, but the point of the demo page is to show
> > how the C front-end and optimizer transform programs to LLVM code.  I
> > might be able to con Brian or Misha into adding support for native code
> > generation, but we'll have to see (they're all really busy with
> > end-of-semester stuff).
>
> Such an add-on would be really nice.

Yes, it would.  On the other hand, we also want to get people looking at
the LLVM code as well: it's really easy to understand and follow once you
get the hang of it, much moreso than machine code.

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/