[llvm-dev] [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"

[Sebastian Pop via llvm-dev]

On Fri, Oct 14, 2016 at 10:50 AM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
>> From: "Renato Golin" <renato.golin at linaro.org>
>> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
>> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Sebastian Paul Pop" <s.pop at samsung.com>, "llvm-dev" <llvm-dev at lists.llvm.org>,
>> "Matthias Braun" <matze at braunis.de>, "Clang Dev" <cfe-dev at lists.llvm.org>, "nd" <nd at arm.com>, "Abe Skolnik"
>> <a.skolnik at samsung.com>
>> Sent: Friday, October 14, 2016 10:36:12 AM
>> Subject: Re: [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"
>>
>> On 14 October 2016 at 15:50, Sebastian Pop <sebpop.llvm at gmail.com>
>> wrote:
>> > These 3 tests are passing with the following configurations:
>> > -O3 -ffp-contract=off
>> > -O3 -ffp-contract=on
>> > -O0 -ffp-contract=off
>> > -O0 -ffp-contract=on
>> >
>> > They are not passing at:
>> > -Ofast -ffp-contract=on
>> > -Ofast -ffp-contract=off
>>
>> Let's separate completely FP-contract and fast-math. They're
>> different
>> things and need different solutions.
>>
>>
>> > if(TEST_SUITE_USES_FAST_MATH)
>> >   add_definitions(-DFP_ABSTOLERANCE=1e0)
>> > else()
>> >   add_definitions(-DFP_ABSTOLERANCE=1e-5)
>> > endif()
>>
>> This doesn't make sense. If my program decreased precision by 5
>> orders
>> of magnitude with -ffast-math, I'd be *very* worried.
>
> Is this an absolute or a relative tolerance? If this is absolute, as the name implies, then I'm not sure how to feel. We should probably use a relative tolerance where possible.
>

This is absolute tolerance: here is the code I'm using to check.

        double V1 = A[i][j];
        double V2 = B[i][j];
        double Diff = fabs(V1 - V2);
        if (Diff > AbsTolerance) {
          fprintf(stderr, "A[%d][%d] = %lf and B[%d][%d] = %lf differ more than"
                  " FP_ABSTOLERANCE = %lf\n", i, j, V1, i, j, V2, AbsTolerance);
          return 0;
        }

Do you want me to add the relative tolerance for the 2 tests?
polybench/linear-algebra/kernels/symm, FP_ABSTOLERANCE=1e1
polybench/linear-algebra/solvers/gramschmidt, FP_ABSTOLERANCE=1e0
What should be a good relative tolerance to set for these two tests?

Here is the code to compute the relative tolerance from fpcmp:

  // Check to see if these are inside the absolute tolerance
  if (AbsTolerance < fabs(V1-V2)) {
    // Nope, check the relative tolerance...
    double Diff;
    if (V2)
      Diff = fabs(V1/V2 - 1.0);
    else if (V1)
      Diff = fabs(V2/V1 - 1.0);
    else
      Diff = 0;  // Both zero.
    if (Diff > RelTolerance) {
      fprintf(stderr, ("%s: Compared: %e and %e\n"
                       "abs. diff = %e rel.diff = %e\n"
                       "Out of tolerance: rel/abs: %e/%e\n"),
              g_program, V1, V2, fabs(V1-V2), Diff, RelTolerance, AbsTolerance);
      return true;
    }
  }

>  -Hal
>
>>
>> I hope that fast-math in Clang isn't that broken, so that's probably
>> to do with the assumptions in the output reduction phase.
>>
>> But, as I said, let's do Ofast *later*. One thing at a time.

Agreed. I will update the patch, make sure all Polybench pass with
"-O3 -ffp-contract=on/off" and it will be ready to commit.

Thanks,
Sebastian