[LLVMdev] execution time of bytecode and native

[Tanu Sharma]

Hello All,
 
Thanks for the reply.I can generate the reports by compiling Spec through llvm, but that couldn't resolve my problem.
 
I m trying to determine execution time for the bytecode and native files , which are obtained as a result of running my pass over the original bytecode .I am running these experiments on spec benchmark.
 
In SPEC we have command line tools such as runspec where we can specify the parameters such as ref or test etc, but if I am taking the llvm-test approach, I have to make all the changes in Makefile.spec.Is that correct? Even for this purpose I am including my pass in Makefile.spec.Is there any simpler or command line option possible?
 
Because of this particular problem, i am not able to use simple commands like "time " to determine time as I can't work on individual benchmarks.
 
If I sound confusing please let me know.Please help me with this.Thanks for taking out time !!
 
Regards
TS

Chris Lattner <sabre at nondot.org> wrote:
On Thu, 30 Jun 2005, Tanu Sharma wrote:
> I am compiling SPEC 2000 benchmarks with llvm .Got stuck with 
> calculating "execution time" of all the .bc and native files.
>
> The log for nightly test itself gives execution times but I am passing 
> the bytecode files to my pass which gives another bytecode file.I have 
> to calculate execution time of such bytecode and native files as well.If 
> i simply do this:

You should look into the llvm-test framework. All of the tests in it (as 
well as 'external tests' which we don't provide source for, like SPEC) 
allow you to do things like this. If you look at llvm-test/TEST.*Makefile 
and llvm-test/TEST.*report files, you'll see ways to automatically 
generate custom reports.

Unfortunately there is no documentation for how to build a custom report, 
but there are many examples. The 'jit' report (TEST.jit.Makefile and 
TEST.jit.report) are a simple example to demonstrate this. For example, 
go into llvm-test/MultiSource/Benchmarks/Olden and run:

$ make TEST=jit report

This should spit out something like this (on X86):

Name: | Total CodeGen InstSel LiveVar RA FPStack Peep | MCSize #MCInsts #Glob | #store #load #fp #fxch |
treeadd/treeadd | 6.38 0.0300 0.0133 0.0023 0.0054 0.0001 0.0002 | 1000 245 115 | * 10 * * |
power/power | 4.87 0.0899 0.0368 0.0044 0.0366 0.0022 0.0003 | 5720 1336 702 | 105 216 790 58 |
tsp/tsp | 4.72 0.1400 0.0439 0.0073 0.0591 0.0023 0.0005 | 4463 1128 106 | 62 149 469 28 |
bh/bh | 3.09 0.2700 0.1231 0.0107 0.1184 0.0030 0.0006 | 11983 2483 137 | 223 408 968 27 |
bisort/bisort | 2.86 0.0800 0.0332 0.0046 0.0163 0.0002 0.0002 | 3825 914 148 | 17 64 * * |
em3d/em3d | 2.84 0.0600 0.0569 0.0034 0.0195 0.0003 0.0002 | 3005 727 406 | 32 79 58 1 |
voronoi/voronoi | 1.11 0.1900 0.0601 0.0081 0.0755 0.0024 0.0009 | 8676 2205 322 | 141 275 761 38 |
health/health | 0.97 0.0800 0.0298 0.0036 0.0171 0.0004 0.0002 | 3438 823 268 | 35 49 101 6 |
perimeter/perimeter | 0.64 0.0800 0.0307 0.0040 0.0149 0.0002 0.0002 | 4266 987 79 | 21 57 * * |
mst/mst | 0.21 0.0500 0.0167 0.0025 0.0117 0.0001 0.0001 | 2094 491 223 | 15 33 * * |

This recursively traverses llvm-test building all of the programs in the 
subdirectory from where you start. The report stuff can emit a text file 
like this, HTML files, CSV, and latex output.

There is a lot of stuff there to play with, I would suggest giving it a 
try. To use SPEC 95/2K, set up the External/SPEC/* stuff and it should 
just work (including automatic management of input/output files etc).

BTW, the nightly tester is just one client of this framework.

-Chris

> time lli spec_benchmark.bc OR time ./spec_benchmark.native
>
> I need to know the command line options and also how to run for "test" or "ref" etc and like.Is
>
> I run experiments (run my pass) some 30 times so is there any way to achieve it globally ?
>
> I appreciate your time,
>
> Thanks
>
>
>
>
>
> ---------------------------------
> Yahoo! Sports
> Rekindle the Rivalries. Sign up for Fantasy Football

-Chris

-- 
http://nondot.org/sabre/
http://llvm.cs.uiuc.edu/

_______________________________________________
LLVM Developers mailing list
LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
http://mail.cs.uiuc.edu/mailman/listinfo/llvmdev

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20050720/4e6b958f/attachment.html>