[LLVMdev] Using the test suite to benchmark patches

[Matthijs Kooijman]

Hi Devang,

I've tried the OPTBETA approach, and it now runs with and without my patch
succesfully. I've found two problems, however:
 1. The output of the nightly report does not include the figures from
    opt-beta by default. I've modified the TEST.nightly.report script to add
		two columns (OPT-BETA and LLC/OPT-BETA), but committing this change would
		mean that most users will be having a lot of useless columns (though they
		already have that for the LLC-BETA stuff).

		Is there a way to access the ENABLE_* variables from the makefile in the
		report script? If so, you could only show the columns that will contain
		actual output, which will make the report a lot easier to read as well.
 2. The opt-beta output is always processed by llc, so you can only compare
    llc results with and without a change. It seems that for my change, the
		largest degradations are in the cbe test, not llc. I don't think there is
		an easy way to solve this, though.

Gr.

Matthijs
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20080522/2a47ece7/attachment.sig>