[PATCH] D14734: [test-suite] PGO and reference input support for SPEC

[Adam Nemet via llvm-commits]

anemet created this revision.
anemet added a reviewer: cmatthews.
anemet added subscribers: MatzeB, llvm-commits.

LNT has this underused feature that lets you defined tests in python.  I used this feature to implement PGO and reference input support for SPEC2000_int, SPEC2006_{int,fp}.

The module currently uses configure/make to build the benchmarks (this can obviously be changed to cmake in the long run).  The tests are then directly run from python and then the output is verified.

This new method is off by default.  You need to invoke lnt runtest nt with —spec-ref to enable it.  Also PGO is supported with —pgo.

I still need to add RunSafely and remote support.  Also some tests depend on endianness and I only added little endian support at this point.

http://reviews.llvm.org/D14734

Files:
  External/SPEC/CFP2006/Makefile
  External/SPEC/Makefile
  LNTBased/lib/spec.py
  LNTBased/speccpu2000/int/164.gzip/TestModule
  LNTBased/speccpu2000/int/175.vpr/TestModule
  LNTBased/speccpu2000/int/176.gcc/TestModule
  LNTBased/speccpu2000/int/181.mcf/TestModule
  LNTBased/speccpu2000/int/186.crafty/TestModule
  LNTBased/speccpu2000/int/197.parser/TestModule
  LNTBased/speccpu2000/int/252.eon/TestModule
  LNTBased/speccpu2000/int/253.perlbmk/TestModule
  LNTBased/speccpu2000/int/254.gap/TestModule
  LNTBased/speccpu2000/int/255.vortex/TestModule
  LNTBased/speccpu2000/int/256.bzip2/TestModule
  LNTBased/speccpu2000/int/300.twolf/TestModule
  LNTBased/speccpu2006/fp/433.milc/TestModule
  LNTBased/speccpu2006/fp/444.namd/TestModule
  LNTBased/speccpu2006/fp/447.dealII/TestModule
  LNTBased/speccpu2006/fp/450.soplex/TestModule
  LNTBased/speccpu2006/fp/453.povray/TestModule
  LNTBased/speccpu2006/fp/470.lbm/TestModule
  LNTBased/speccpu2006/fp/482.sphinx3/TestModule
  LNTBased/speccpu2006/fp/482.sphinx3/ref/ctlfile.little-endian
  LNTBased/speccpu2006/fp/482.sphinx3/train/ctlfile.little-endian
  LNTBased/speccpu2006/int/400.perlbench/TestModule
  LNTBased/speccpu2006/int/401.bzip2/TestModule
  LNTBased/speccpu2006/int/403.gcc/TestModule
  LNTBased/speccpu2006/int/429.mcf/TestModule
  LNTBased/speccpu2006/int/445.gobmk/TestModule
  LNTBased/speccpu2006/int/456.hmmer/TestModule
  LNTBased/speccpu2006/int/458.sjeng/TestModule
  LNTBased/speccpu2006/int/462.libquantum/TestModule
  LNTBased/speccpu2006/int/464.h264ref/TestModule
  LNTBased/speccpu2006/int/471.omnetpp/TestModule
  LNTBased/speccpu2006/int/473.astar/TestModule
  LNTBased/speccpu2006/int/483.xalancbmk/TestModule

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D14734.40350.patch
Type: text/x-patch
Size: 77067 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151116/916cc381/attachment.bin>