[PATCH] D25277: [test-suite] Run FP tests twice with -ffp-contract=off/on

[Matthias Braun via llvm-commits]

> On Oct 5, 2016, at 11:48 AM, Abe Skolnik via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> 
> [Matthias Braun wrote:]
> 
>> Ugh, I don't like the added complexity and disk space bloat.
> 
> Unfortunately, I think we are up against _essential_ complexity at least a little: at least for tests for which the contraction/fast-math build differs from the conservatively-FP-optimized build, we need to do _two_ tests each.  What we compare the output of the _first_ against is another question [next quote-and-reply section].
> 
> In principle, the test harness could compare the two builds [aggressively-FP-optimized vs. conservatively-FP-optimized] and if the executables are binary-equal, then not bother running the second test.  However, TTBOMK that is still room for future improvement.
> 
> 
>> Is there no way to summarize/hash/compress a list of floating point numbers in a way that we can compare
>> them against a reference output and at the same time allow some fluctuations in floatingpoint 
> accuracy?
> 
> Yes, there is a way -- compressing the ref. outputs in the repo., decompressing at the test site -- but it has been viewed negatively because:

What I am hoping for/asking about is a way to perform some sort of arithmetic inside the benchmark so that we only need to print very few numbers.

- Matthias