[llvm-dev] Using lnt to run SPEC tests: possible or not?
Kristof Beyls via llvm-dev
llvm-dev at lists.llvm.org
Wed Jun 20 00:02:42 PDT 2018
Yes, it is possible to run SPEC tests through LNT/test-suite.
You won’t get official reportable SPEC numbers as you don’t use the official build-and-run framework from SPEC, but OTOH you get the extra features in LNT/test-suite working out of the box, e.g. the ones documented at http://blog.llvm.org/2016/06/using-lnt-to-track-performance.html.
To make use of this, I think you need to use the cmake driver in the test-suite.
Which, if you use the LNT wrapper to drive runs, is invoked using “lnt runtest test-suite”.
A few example of the important options on the command line that I hope will get you started:
* Running SPEC2K6int reference inputs: lnt runtest test-suite --only-test=External/SPEC/CINT2006 --cmake-define TEST_SUITE_SPEC2006_ROOT=/work/Benchmark/spec2006 --cmake-define TEST_SUITE_RUN_TYPE=ref
* Running SPEC2K6int train inputs: lnt runtest test-suite --only-test=External/SPEC/CINT2006 --cmake-define TEST_SUITE_SPEC2006_ROOT=/work/Benchmark/spec2006 --cmake-define TEST_SUITE_RUN_TYPE=train
* Running SPEC2Kfp reference inputs: lnt runtest test-suite --only-test=External/SPEC/CFP2000 --cmake-define TEST_SUITE_SPEC2000_ROOT=/work/Benchmark/spec2000 --cmake-define TEST_SUITE_RUN_TYPE=ref
On 19 Jun 2018, at 23:53, Bill Seurer via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
It appears that lnt has the capability to run the SPEC2000/2006/2017(maybe) tests from looking at the options and from poking around in the mailing list archives. Or does it? I experimented a bit and cannot get it to work right. However, it isn't really documented anywhere so maybe I didn't set it up properly. Plus there's all the mention of llvmgcc...
So is anyone doing this? Is there some setup information that I missed? Thanks for any tips on how to do this!
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev