[llvm-dev] [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"

[Sebastian Pop via llvm-dev]

On Wed, Oct 12, 2016 at 8:49 AM, Renato Golin <renato.golin at linaro.org> wrote:
> On 12 October 2016 at 13:04, Sebastian Pop <sebpop.llvm at gmail.com> wrote:
>> The other problem is the reference output does not match
>> at "-O0 -ffp-contract=off". It might be that the reference output was recorded
>> at "-O3 -ffp-contract=off". I think that this hides either a compiler
>> bug or a test bug.
>
> Ah, yes! You mentioned before and I forgot to reply, you're absolutely right.
>
> If the tolerance is zero, then it's "ok" to "fail" at O0, because
> whatever O3 produces is "some" version of the expected value +- some
> delta. The error is expecting the tolerance to be zero (or smaller
> than delta).
>
> My point, since the beginning, has been to understand what the
> expected value (with its inherent error bars), and make that the
> reference output. Only then the test will be meaningful *and*
> accurate.

Correct me if I misunderstood: you would be ok changing the
reference output to exactly match the output of "-O0 -ffp-contract=off".

I am asking this for practical reasons:
clang currently only supports __attribute__((optnone)) to compile
a function at -O0.  All other optimization levels are not yet supported.
In the updated patch for Proposal 2: https://reviews.llvm.org/D25346
we do use that attribute together with
#pragma STDC FP_CONTRACT OFF
to compile the kernel_StrictFP() function at "-O0 -ffp-contract=off".
The output of kernel_StrictFP is then used in exact matching against
the reference output.

In polybench there are 5 benchmarks that need adjustment of the
reference output to match the output of optnone.

polybench/linear-algebra/kernels/symm
polybench/linear-algebra/solvers/gramschmidt
polybench/medley/reg_detect
polybench/stencils/adi
polybench/stencils/seidel-2d

Thanks,
Sebastian