[LLVMdev] benchmarking LLVM

[Chris Lattner]

On Mon, 21 Jun 2004, [koi8-r] "Valery A.Khamenya[koi8-r] "  wrote:

> > This test seems PARTICULARLY noisy for some reason.  Take a look at this:
> > [...]
> > ... the native version swings from 3.13-3.34s and the llc-ls version
> > swings from 3.83-4.0s.  The C backend is also noisy, swinging from
> > 3.98-4.58s.  FWIW, these tests are on a Intel P4 Xeon 3.06Ghz machine with
> > HT enabled. In any case, it appears that we're slower than GCC on this one.
>
> what is very funny, that the situation on my AMD is not similar
> at all to what you state here. On my side tests show GREAT spead up
> with ackerman, i.e. 2 to 3 times. Sorry, I can't access my
> linux box now to get concrete values.

It's quite possible that it is the difference between the two chips.  The
microarchitectures are very different between the two.  If there is
something else going on here we do want to know about it though...

> > If you
> > wanted to dig into the test to see what is going wrong (is the LLVM code
> > missing an optimization, or is it bad code generation?), that would be
> > quite helpful.
>
> i will access my linux box next Friday only :(

LLVM works under cygwin and Mac OS/X.  If you have access to either of
those, it should work for you  :)

> > Thanks for looking into this.  We have (obviously) spent very limited time
> > looking into performance issues, it's great that you're able to do so!
>
> I am sorry for helping too few. I'd wish more helpers join this nice
> project.

Everything helps.  :)

> Still, i am wondering how could it happens that result are
> so different by my side and your side. I'd not expect a very
> big difference becasue of AMD <-> Xeon difference...

I'm not sure.  I would expect such a big difference either, but if you are
using the same llvmgcc and llc as we are (CVS HEAD) you should be running
exactly the same machine code.

> I'd propose to check error codes with benchmarking suite.
> Maybe to quick result ist just aborted execution?..
> (anyway i still do not see smth very cool for ary*
> tests at my side)
>
> Are there anyone how could test Shootout test as well?

I don't understand what you're saying here, can you please rephrase it?

-Chris

-- 
http://llvm.cs.uiuc.edu/
http://www.nondot.org/~sabre/Projects/