[cfe-dev] [llvm-dev] a proposed script to help with test-suite programs that output _lots_ of FP numbers

[Hal Finkel via cfe-dev]

----- Original Message -----
> From: "Sebastian Pop" <sebpop.llvm at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "llvm-dev" <llvm-dev at lists.llvm.org>, "Sebastian Paul Pop" <s.pop at samsung.com>, "Abe Skolnik"
> <a.skolnik at samsung.com>, "cfe-dev" <cfe-dev at lists.llvm.org>, "Renato Golin" <renato.golin at linaro.org>
> Sent: Thursday, September 29, 2016 4:41:23 PM
> Subject: Re: [cfe-dev] [llvm-dev] a proposed script to help with test-suite programs that output _lots_ of FP numbers
> 
> On Thu, Sep 29, 2016 at 4:32 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> > We don't want to lose the more-stringent test coverage just because
> > that no longer might be the default mode. We'll also want, where
> > practical, some looser test mode that will work regardless of the
> > FP contraction setting. Both are important.
> 
> What about recording two golden files with fp-cotract=on and off,
> then
> diff the output against both, and if none is similar enough error.
> I see two problems with recording the output of fp-contract=on:
> - based on the target there may be more or less contractions done,
> - and also the results may vary over time as the compiler gets better
> at folding.

Recording the output with -ffp-contract=on to use as a reference output does not seem useful, exactly for the reasons you specify. We should generate the reference outputs with -ffp-contract=off as we do now. That's the only good target-independent configuration. Maybe we could do this:

 - Have a reference output with -ffp-contract=off
 - Run program compiled with -ffp-contract=off and the default
 - Compare the former against the reference output for validation
 - Compare the latter (the default build) against the former with fpcmp with some tolerance

This also has the advantage that we still only need the hashed reference outputs in the repository.

 -Hal

> 

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory