[LLVMdev] How to test my pass

[Nick Lewycky]

Chayan Sarkar wrote:
> Hi there,
>
> I have written a PRE pass using LLVM. How can I test my  pass? Is
> there any standard test-cases or bench-marks to test a pass. Please
> suggest accordingly.

Use the LLVM nightly test suite: 
http://llvm.org/docs/TestingGuide.html#testsuiterun

Make a copy of an existing test like TEST.example.Makefile and modify it 
to run 'opt -yourpass -time-passes -stats', then 'make TEST=yourexample 
report' and read the report.raw.out it produces. Look for crashes first 
and foremost, then look at how many times it actually modified the user 
code, then finally make sure it didn't take show absurd compile-time 
problems. Fix all the crashers and O(n^2) behaviour.

Now you're ready to try your pass on real code. Modify 
include/llvm/Support/StandardPasses.html to add your pass to the list of 
defaults. Go back to the nightly test suite and 'make TEST=nightly 
report.html' which will not only use your pass when building the tests, 
but actually run the resulting executables and verify their results. 
Most of the * cells in the resulting HTML will be as a result of your 
pass, but not all. Your first step in debugging them is to cd to the 
correct source directory and run 'make bugpoint-opt'[1] to get a reduced 
testcase. Fix all the crashers and miscompiles.

If you have access to SPEC, that's also a great source of benchmarks. 
You should certainly try it out over spec if you at all possibly can, 
since spec turns up all sorts of miscompiles that the llvm test suite 
doesn't.

Finally, do a run of the nightly test with your pass enabled, and a 
reference run without it. How much did you pass really speed the 
programs up? How much did it slow the compiler down? Is your pass worth 
it to our users? If so, it's time to commit!

Nick

[1] In MultiSource/... that does the right thing, in SingleSource/... it 
will print out a message telling you what you should really run.
>
> Regards,
> Chayan
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>