[PATCH] D24925: [testsuite] turn fp-contract off for ARM because output checking is not flexible enough

[Stephen Canon via llvm-commits]

> On Sep 26, 2016, at 2:59 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> 
> I understand you're being sarcastic, but this is unfair. This is not a random reason, we understand exactly why this change produces changes in program output, and I also believe that testing using -ffp-contract=off is the most-useful test configuration to have (although having others in addition would be great too). Why is it the most useful? Because it is the testing configuration for which we can have the most-precise known-good answer for comparison. As you point out, having strict thresholds for these tests has caught important miscompilation bugs in the past. When you start using FMAs, one issue is that there are lots of places where you now might or might not actually use them, and so the space of potential correct answers is large. We can come up with loser tolerances to capture this freedom, but even doing this precisely is potentially degenerate with bugs for some inputs, and so only having the loser version is actually worse than just testing with FMAs disabled. I would love to have both, but in the mean time, this is the right thing to do.
> 
> FWIW, the reason we do this on PPC/LInux, and have for a long time, is exactly because GCC on PPC/Linux defaults to using FMAs, and so GCC needs -ffp-contract=off to pass our test suite (independent of anything else). This is just a general statement about our reference outputs: they're all generated without FMAs. Our flags should encode that dependency so that we don't see spurious failures (with Clang or any other compiler).


I agree with Hal, but would go a little bit further: we should really be running these tests twice.  First, for platforms with FLOAT_EVAL_METHOD=0 in -ffp-contract=off with a bit-exact result requirement, and second for all platforms with -ffp-contract=on and only checking that the appropriate error bound is satisfied (i.e. *no* comparison to previous results, only the established bound).

– Steve
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160926/456136bf/attachment.html>