[PATCH] D25346: [test-suite] [Polybench] run tests twice with -ffp-contract=on/off

[Sebastian Pop via llvm-commits]

sebpop added a comment.

In https://reviews.llvm.org/D25346#564282, @rengolin wrote:

> In https://reviews.llvm.org/D25346#564230, @sebpop wrote:
>
> > kernel() is compiled with whatever Cflags are specified by the end user of the test-suite.
> >  The output of kernel() is matched with FP_TOLERANCE against the output of kernel_Strict_FP().
> >  The output of kernel_Strict_FP() is hashed and compared against reference output.
>
>
> Right, now I get it. So, you basically just duplicated the kernel to force FP=OFF on the second. Then you compare against the *real* timed run for FP_TOLERANCE.


Correct.

> This should work as intended, I imagine, but would that scale to *all* problematic benchmarks? If there are 50 of them, duplicating all the cores may bring more problems than solving...

As I was mentioning, this is intended only for the Polybench, that are following a pretty regular pattern of testing loop kernels.
The patch makes 2mm run correctly for CFLAGS="", "-Ofast", and "-O3 -ffp-contract=on".
I have not looked at the other failing benchmarks to say whether this can apply to those.
I know that oggenc would be impossible to modify like this.

There are also these resource problems I was mentioning:

- compilation time will double: e.g., Polly will optimize both kernels,
- memory requirements on the device will almost double: added one extra output array, input arrays are not modified, so no need to duplicate them,
- compute time on the device will more than double: running the kernel twice, plus an extra loop over both outputs to compare with FP_TOLERANCE.

Not perfect, though it will give us some extra FP tests that we can run at -Ofast and other flags that need FP_TOLERANCE testing.


https://reviews.llvm.org/D25346