llvm test-suite '---large' problem size

[Daniel Dunbar]

I think its supported in that it works, but the reference outputs have
never been generated. The use of reference outputs is optional from the
standpoint of the test suite, although I don't think that option is used
much anymore.

Feel free to start adding large reference outputs if you feel like it,
although it pays to be mindful of the file size and use the hashed
reference output option where appropriate.

 - Daniel


On Wed, Mar 5, 2014 at 3:35 AM, Robert Lytton <robert at xmos.com> wrote:

>  Hi Daniel,
>
> Should '--large' be supported for the LNT?
>
> The 'Makefile.programs' uses the $(REFERENCE_OUTPUT_KEY) to pick up
> alternative forms of reference_output
> when SMALL_PROBLEM_SIZE or LARGE_PROBLEM_SIZE is defined e.g.
>     $*.reference_output.$(ENDIAN)-endian.$(REFERENCE_OUTPUT_KEY)
>
> Thus I can include files such as:
>     minisat.reference_output.little-endian.large
>     minisat.reference_output.big-endian.large
> However, there seem to be none.
>
> Not surprisingly running the 'lnt runtest nt' using '--large' the
> following 'LARGE_PROBLEM_SIZE' tests fail due to output diffs:
>     FAIL: MultiSource/Applications/minisat/minisat.execution_time (496 of
> 990)
>     FAIL: MultiSource/Applications/sgefa/sgefa.execution_time (497 of 990)
>     FAIL:
> MultiSource/Benchmarks/Fhourstones-3_1/fhourstones3_1.execution_time (498
> of 990)
>     FAIL:
> MultiSource/Benchmarks/FreeBench/analyzer/analyzer.execution_time (499 of
> 990)
>     FAIL: MultiSource/Benchmarks/FreeBench/distray/distray.execution_time
> (500 of 990)
>     FAIL:
> MultiSource/Benchmarks/FreeBench/fourinarow/fourinarow.execution_time (501
> of 990)
>     FAIL: MultiSource/Benchmarks/FreeBench/mason/mason.execution_time (502
> of 990)
>     FAIL: MultiSource/Benchmarks/FreeBench/neural/neural.execution_time
> (503 of 990)
>     FAIL:
> MultiSource/Benchmarks/FreeBench/pcompress2/pcompress2.execution_time (504
> of 990)
>     FAIL: MultiSource/Benchmarks/FreeBench/pifft/pifft.execution_time (505
> of 990)
>     FAIL: MultiSource/Benchmarks/MallocBench/cfrac/cfrac.execution_time
> (506 of 990)
>     FAIL: MultiSource/Benchmarks/Olden/bh/bh.execution_time (507 of 990)
>     FAIL: MultiSource/Benchmarks/Olden/bisort/bisort.execution_time (508
> of 990)
>     FAIL: MultiSource/Benchmarks/Olden/health/health.execution_time (509
> of 990)
>     FAIL: MultiSource/Benchmarks/Olden/mst/mst.execution_time (510 of 990)
>     FAIL: MultiSource/Benchmarks/Olden/perimeter/perimeter.execution_time
> (511 of 990)
>     FAIL: MultiSource/Benchmarks/Olden/tsp/tsp.execution_time (512 of 990)
>     FAIL: MultiSource/Benchmarks/Olden/voronoi/voronoi.execution_time (513
> of 990)
>     FAIL: MultiSource/Benchmarks/Ptrdist/ft/ft.execution_time (514 of 990)
>     FAIL: MultiSource/Benchmarks/SciMark2-C/scimark2.execution_time (515
> of 990)
>     FAIL: MultiSource/Benchmarks/Trimaran/enc-md5/enc-md5.execution_time
> (516 of 990)
>     FAIL: MultiSource/Benchmarks/Trimaran/enc-pc1/enc-pc1.execution_time
> (517 of 990)
>     FAIL:
> MultiSource/Benchmarks/Trimaran/netbench-crc/netbench-crc.execution_time
> (518 of 990)
>     FAIL:
> MultiSource/Benchmarks/Trimaran/netbench-url/netbench-url.execution_time
> (519 of 990)
>     FAIL:
> MultiSource/Benchmarks/VersaBench/beamformer/beamformer.execution_time (520
> of 990)
>     FAIL: MultiSource/Benchmarks/VersaBench/bmm/bmm.execution_time (521 of
> 990)
>     FAIL: MultiSource/Benchmarks/llubenchmark/llu.execution_time (522 of
> 990)
>
> Robert
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20140305/cfdf52a3/attachment.html>