[PATCH] D36717: [test-suite] Add SPEC CPU 2017

Mon Aug 14 15:06:55 PDT 2017

Meinersbur created this revision.
Herald added subscribers: fedor.sergeev, mgorny.

Add CMakeLists.txt files for SPEC CPU 2017.

Add C/C++ benchmarks are included, grouped by benchmark suite (SPECrate 2017 Integer, SPECspeed 2017
Integer, SPECrate 2017 Floating Point, SPECspeed 2017 Floating Point).

SPEC's changes to some of the benchmarks (at least perlbench) require then to be invoked with a relative path (because for finding its working directory, it prepends a path to basename(argv[0])). When the executable is referred to by a relative path, litsupport needs to take cd commands before into account, otherwise it won't find the executable.

Some reference files in SPEC are stored with CRLF line endings (511.povray_r) while the benchmark generates LF endings, at least on UNIX. fpcmp got stock in an endless loop in this case.

Useful additions for the future:

Some mixed-language projects of SPEC CPU 2017 have most of sources written in C/C++ (e.g. cactuBSSN). A useful addition might try compiling the Fortran parts with gfortran.

The only platform I tried is x86_64. I can try make it work with -m32 as well. Support for other platforms has to be added by people who can test on those.

Most 5xx_r/6xx_s benchmark pairs are (nearly) identical and/or only have different test inputs. One might save compile/runtime by running only one of them.

Many _s flavors have support for OpenMP. This is unconditionally disabled at the moment, but one might add an option to compile them with OpenMP.

The new CMakeLists.txt support multiple run types (test,train,ref) in one build but test-suite's cmake system doesn't allow this yet.

https://reviews.llvm.org/D36717

Files:
  External/SPEC/CFP2017rate/508.namd_r/CMakeLists.txt
  External/SPEC/CFP2017rate/510.parest_r/CMakeLists.txt
  External/SPEC/CFP2017rate/511.povray_r/CMakeLists.txt
  External/SPEC/CFP2017rate/519.lbm_r/CMakeLists.txt
  External/SPEC/CFP2017rate/526.blender_r/CMakeLists.txt
  External/SPEC/CFP2017rate/538.imagick_r/CMakeLists.txt
  External/SPEC/CFP2017rate/544.nab_r/CMakeLists.txt
  External/SPEC/CFP2017rate/997.specrand_fr/CMakeLists.txt
  External/SPEC/CFP2017rate/CMakeLists.txt
  External/SPEC/CFP2017speed/619.lbm_s/CMakeLists.txt
  External/SPEC/CFP2017speed/638.imagick_s/CMakeLists.txt
  External/SPEC/CFP2017speed/644.nab_s/CMakeLists.txt
  External/SPEC/CFP2017speed/996.specrand_fs/CMakeLists.txt
  External/SPEC/CFP2017speed/CMakeLists.txt
  External/SPEC/CINT2017rate/500.perlbench_r/CMakeLists.txt
  External/SPEC/CINT2017rate/502.gcc_r/CMakeLists.txt
  External/SPEC/CINT2017rate/505.mcf_r/CMakeLists.txt
  External/SPEC/CINT2017rate/520.omnetpp_r/CMakeLists.txt
  External/SPEC/CINT2017rate/523.xalancbmk_r/CMakeLists.txt
  External/SPEC/CINT2017rate/525.x264_r/CMakeLists.txt
  External/SPEC/CINT2017rate/531.deepsjeng_r/CMakeLists.txt
  External/SPEC/CINT2017rate/541.leela_r/CMakeLists.txt
  External/SPEC/CINT2017rate/557.xz_r/CMakeLists.txt
  External/SPEC/CINT2017rate/999.specrand_ir/CMakeLists.txt
  External/SPEC/CINT2017rate/CMakeLists.txt
  External/SPEC/CINT2017speed/600.perlbench_s/CMakeLists.txt
  External/SPEC/CINT2017speed/602.gcc_s/CMakeLists.txt
  External/SPEC/CINT2017speed/605.mcf_s/CMakeLists.txt
  External/SPEC/CINT2017speed/620.omnetpp_s/CMakeLists.txt
  External/SPEC/CINT2017speed/623.xalancbmk_s/CMakeLists.txt
  External/SPEC/CINT2017speed/625.x264_s/CMakeLists.txt
  External/SPEC/CINT2017speed/631.deepsjeng_s/CMakeLists.txt
  External/SPEC/CINT2017speed/641.leela_s/CMakeLists.txt
  External/SPEC/CINT2017speed/657.xz_s/CMakeLists.txt
  External/SPEC/CINT2017speed/998.specrand_is/CMakeLists.txt
  External/SPEC/CINT2017speed/CMakeLists.txt
  External/SPEC/CMakeLists.txt
  External/SPEC/SpecCPU2017.cmake
  cmake/modules/TestFile.cmake
  litsupport/shellcommand.py
  tools/fpcmp.c

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Malware Alert Text.txt
Type: application/octet-stream
Size: 167 bytes
Desc: Malware Alert Text.txt
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170814/0719e7a0/attachment.obj>