[LLVMdev] Running the programs in the LLVM test suite with a pass and without it.

[Andrew Trick]

On Feb 7, 2011, at 4:20 AM, Douglas do Couto Teixeira wrote:

> Hi guys,
> 
> I have a pass called "MyPass". And I can run this pass in the LLVM test suite programs and produce a report with the statistics of the execution of this pass. Now, I would like to run the programs in LLVM test suite and generate a report comparing the results obtained without using the pass with the results obtained using the pass. Is there any way to do this?

If by "stats" you mean execution time, then you may use the Makefile support for "llc beta" as Jim suggested. I use the following without editing the Makefiles:

TEST=nightly ENABLE_LLCBETA=1 LLCBETAOPTION="-my-pass" report.csv

To compare across separate runs, I just load the report.csv into a spreadsheet and define some formulas.

If you actually want to compare output from the -stats option, then you can compare the *.llc.s.info files (one for each benchmark) from two separate runs. I attached the script that I use to do this.

-Andy

-------------- next part --------------
A non-text attachment was scrubbed...
Name: comparestats.py
Type: text/x-python-script
Size: 2416 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20110209/0b90ce5b/attachment.bin>