[PATCH] D25346: [test-suite] [Polybench] run tests twice with -ffp-contract=on/off

[Sebastian Pop via llvm-commits]

sebpop added a comment.

In https://reviews.llvm.org/D25346#564218, @rengolin wrote:

> > What I mean by bisimulation is that we would copy the kernel()
> >  function of each polybench/test.c and name it kernelNoFP() and add
> >  flag attributes "-fno-fast-math -ffp-contract=off" (can be split
> >  compilation if flag attributes do not work.)
> >  main() will call kernel() and kernelNoFP() and compare their output
> >  with FP_TOLERANCE.
> >  Only the execution of kernel() will be timed for benchmark performance result.
> >  main() will only print the output from kernelNoFP() that will be
> >  hashed and compared against the reference hash (as we currently expect
> >  exact match of the output hash.)
>
> So, the end result is that we'll *test* what we don't ship and we'll *benchmark* what we don't test.
>
> Meaning we'll have green bots and fast benchmarks, but we'll have no idea if the "fast" is correct of if the correct is fast.


This statement is false.

kernel() is compiled with whatever Cflags are specified by the end user of the test-suite.
The output of kernel() is matched with FP_TOLERANCE against the output of kernel_Strict_FP().
The output of kernel_Strict_FP() is hashed and compared against reference output.

kernel() is timed, as this will be what the end-user of the test-suite asked for "-O0", or "-Ofast -polly", or any other combination.


https://reviews.llvm.org/D25346