llvm test-suite '---large' problem size

[Robert Lytton]

Hi Daniel,

Should '--large' be supported for the LNT?

The 'Makefile.programs' uses the $(REFERENCE_OUTPUT_KEY) to pick up alternative forms of reference_output
when SMALL_PROBLEM_SIZE or LARGE_PROBLEM_SIZE is defined e.g.
    $*.reference_output.$(ENDIAN)-endian.$(REFERENCE_OUTPUT_KEY)

Thus I can include files such as:
    minisat.reference_output.little-endian.large
    minisat.reference_output.big-endian.large
However, there seem to be none.

Not surprisingly running the 'lnt runtest nt' using '--large' the following 'LARGE_PROBLEM_SIZE' tests fail due to output diffs:
    FAIL: MultiSource/Applications/minisat/minisat.execution_time (496 of 990)
    FAIL: MultiSource/Applications/sgefa/sgefa.execution_time (497 of 990)
    FAIL: MultiSource/Benchmarks/Fhourstones-3_1/fhourstones3_1.execution_time (498 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/analyzer/analyzer.execution_time (499 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/distray/distray.execution_time (500 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow.execution_time (501 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/mason/mason.execution_time (502 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/neural/neural.execution_time (503 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/pcompress2/pcompress2.execution_time (504 of 990)
    FAIL: MultiSource/Benchmarks/FreeBench/pifft/pifft.execution_time (505 of 990)
    FAIL: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.execution_time (506 of 990)
    FAIL: MultiSource/Benchmarks/Olden/bh/bh.execution_time (507 of 990)
    FAIL: MultiSource/Benchmarks/Olden/bisort/bisort.execution_time (508 of 990)
    FAIL: MultiSource/Benchmarks/Olden/health/health.execution_time (509 of 990)
    FAIL: MultiSource/Benchmarks/Olden/mst/mst.execution_time (510 of 990)
    FAIL: MultiSource/Benchmarks/Olden/perimeter/perimeter.execution_time (511 of 990)
    FAIL: MultiSource/Benchmarks/Olden/tsp/tsp.execution_time (512 of 990)
    FAIL: MultiSource/Benchmarks/Olden/voronoi/voronoi.execution_time (513 of 990)
    FAIL: MultiSource/Benchmarks/Ptrdist/ft/ft.execution_time (514 of 990)
    FAIL: MultiSource/Benchmarks/SciMark2-C/scimark2.execution_time (515 of 990)
    FAIL: MultiSource/Benchmarks/Trimaran/enc-md5/enc-md5.execution_time (516 of 990)
    FAIL: MultiSource/Benchmarks/Trimaran/enc-pc1/enc-pc1.execution_time (517 of 990)
    FAIL: MultiSource/Benchmarks/Trimaran/netbench-crc/netbench-crc.execution_time (518 of 990)
    FAIL: MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.execution_time (519 of 990)
    FAIL: MultiSource/Benchmarks/VersaBench/beamformer/beamformer.execution_time (520 of 990)
    FAIL: MultiSource/Benchmarks/VersaBench/bmm/bmm.execution_time (521 of 990)
    FAIL: MultiSource/Benchmarks/llubenchmark/llu.execution_time (522 of 990)

Robert

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140305/6488d452/attachment.html>