[PATCH] D25277: [test-suite] Run FP tests twice with -ffp-contract=off/on

[Sebastian Pop via llvm-commits]

sebpop created this revision.
sebpop added reviewers: rengolin, cmatthews, MatzeB, Abe.
sebpop added a subscriber: llvm-commits.
Herald added subscribers: mgorny, beanz.

With this patch all the tests specifying an FP_TOLERANCE or FP_ABSTOLERANCE will be run in two stages:

- first step is with with extra flags "-ffp-contract=off -fno-fast-math", and recording the full output that will be checked against in the second step
- second step is with the default flags specified by the user, and checks against the output recorded in the first step with the FP_TOLERANCE specified by the test.

There are at least two main problems with this approach:

- some of the current golden files in the test-suite require testing with FP_TOLERANCE even for stage1
- the build directory goes from 300M to 1.2G due to the extra reference outputs recorded by stage1, and for at least 2 tests (polybench/bicg, polybench/mvt disabled in the current patch) fpcmp takes more than 10 minutes to completely parse the outputs of stage1 and 2. When running test-suite over small devices will cost 1G more transfer over the network.

The current implementation has another smaller technical issue:
lit does not allow to specify dependences between tests.
When running "lit -j8" one can end up running stage 2 before stage 1.
The solution I was considering is to run step 1 as a PREPARE: step,
though that would fail to add the exit code in the output,
and that will end up mismatching the hash.


https://reviews.llvm.org/D25277

Files:
  HashProgramOutput.sh
  MultiSource/Applications/oggenc/CMakeLists.txt
  MultiSource/Benchmarks/VersaBench/beamformer/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/datamining/correlation/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/datamining/covariance/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/2mm/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/3mm/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/atax/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/bicg/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/cholesky/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/doitgen/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemm/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gemver/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/gesummv/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/mvt/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/symm/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/syr2k/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/syrk/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/trisolv/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/kernels/trmm/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/solvers/durbin/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/solvers/dynprog/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/solvers/gramschmidt/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/solvers/lu/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/linear-algebra/solvers/ludcmp/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/medley/floyd-warshall/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/adi/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/fdtd-2d/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/fdtd-apml/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/jacobi-1d-imper/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/jacobi-2d-imper/CMakeLists.txt
  SingleSource/Benchmarks/Polybench/stencils/seidel-2d/CMakeLists.txt
  cmake/modules/SingleMultiSource.cmake

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D25277.73639.patch
Type: text/x-patch
Size: 21448 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20161005/d315c40d/attachment-0001.bin>