[llvm-dev] [test-suite] making the test-suite succeed with "-Ofast" and "-ffp-contract=on"

[Renato Golin via llvm-dev]

Proposal 4:

Investigate each problematic benchmark and apply the best solution for
each one of them, independently. For oggenc we may need something
different.

While investigating povray on a similar case (very small FP
differences over very few points of the output), I noticed we were
emitting NEON instructions as if they were IEEE compliant (they're
not). That lead me to fix a bad compiler bug.

Are we sure all the FP=on differences are *just* due to fusions? If
so, then lets look at the benchmarks and make them output less garbage
without resorting to hashes. I've done that to a number of tests and
benchmarks already. It's quite boring, yes, but it's necessary if we
want them to be meaningful.

The proposal 2 is actually good for Polybench, at least for the one
case where Sebastian has implemented. Yes, it doubles run time, but
it's validation run time, which is part of the test, and it doesn't
bloat disk/memory. Benchmark bots nowadays only run for a few
iterations anyway, and even the ARM bot (the slowest) is now only
taking 2hs per build.

My proposal is to go through all 50 cases and propose the lowest
number of solutions possible for all of them. I'm guessing this will
be between 2 and 4 different cases.

cheers,
--renato