[llvm-dev] [cfe-dev] a proposed script to help with test-suite programs that output _lots_ of FP numbers

[Abe Skolnik via llvm-dev]

[On Thu, Sep 29, 2016 at 4:32 PM, Hal Finkel <hfinkel at anl.gov> wrote:]
> We don't want to lose the more-stringent test coverage just because that no longer might be the default mode.
>  We'll also want, where practical, some looser test mode that will work regardless of the FP 
contraction setting.
>  Both are important.

[On 09/29/2016 04:41 PM, Sebastian Pop wrote:]
> What about recording two golden files with fp-contract=on and off,
> then diff the output against both, and if none is similar enough error.
> I see two problems with recording the output of fp-contract=on:
> - based on the target there may be more or less contractions done,
> - and also the results may vary over time as the compiler gets better
> at folding.

Yes, separate "golden files" with FP contraction on [or "fast"] is problematic.

However, if I/we can get compression working, then in some sense we can "have our cake and eat 
it too": have small reference files in the repository and in downloads, and have any 
precision/tolerance we like [including requiring exact matching] at test time.

If all the big outputs compress very nicely, as does the one from 
"MultiSource/Benchmarks/VersaBench/beamformer", then we can probably do this across all tests 
the output of which depends on FP computations.

-- Abe