[llvm-dev] Using lnt to run SPEC tests: possible or not?

[Kristof Beyls via llvm-dev]

Hi Bill,

Yes, it is possible to run SPEC tests through LNT/test-suite.
You won’t get official reportable SPEC numbers as you don’t use the official build-and-run framework from SPEC, but OTOH you get the extra features in LNT/test-suite working out of the box, e.g. the ones documented at http://blog.llvm.org/2016/06/using-lnt-to-track-performance.html.

To make use of this, I think you need to use the cmake driver in the test-suite.
Which, if you use the LNT wrapper to drive runs, is invoked using “lnt runtest test-suite”.

A few example of the important options on the command line that I hope will get you started:

  *   Running SPEC2K6int reference inputs: lnt runtest test-suite --only-test=External/SPEC/CINT2006 --cmake-define TEST_SUITE_SPEC2006_ROOT=/work/Benchmark/spec2006 --cmake-define TEST_SUITE_RUN_TYPE=ref

  *   Running SPEC2K6int train inputs: lnt runtest test-suite --only-test=External/SPEC/CINT2006 --cmake-define TEST_SUITE_SPEC2006_ROOT=/work/Benchmark/spec2006 --cmake-define TEST_SUITE_RUN_TYPE=train
  *   Running SPEC2Kfp reference inputs:  lnt runtest test-suite --only-test=External/SPEC/CFP2000 --cmake-define TEST_SUITE_SPEC2000_ROOT=/work/Benchmark/spec2000 --cmake-define TEST_SUITE_RUN_TYPE=ref


Thanks,

Kristof


On 19 Jun 2018, at 23:53, Bill Seurer via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:

It appears that lnt has the capability to run the SPEC2000/2006/2017(maybe) tests from looking at the options and from poking around in the mailing list archives.  Or does it?  I experimented a bit and cannot get it to work right.  However, it isn't really documented anywhere so maybe I didn't set it up properly.  Plus there's all the mention of llvmgcc...

So is anyone doing this?  Is there some setup information that I missed?  Thanks for any tips on how to do this!
--

-Bill Seurer

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180620/980d44bc/attachment.html>