[llvm-dev] [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"

[Sebastian Pop via llvm-dev]

On Tue, Oct 11, 2016 at 10:39 PM, Sebastian Pop <sebpop.llvm at gmail.com> wrote:
> On Tue, Oct 11, 2016 at 10:20 PM, Hal Finkel <hfinkel at anl.gov> wrote:
>> ----- Original Message -----
>>> From: "Renato Golin" <renato.golin at linaro.org>
>>> To: "Sebastian Pop" <sebpop.llvm at gmail.com>
>>> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Sebastian Paul Pop" <s.pop at samsung.com>, "llvm-dev" <llvm-dev at lists.llvm.org>,
>>> "Matthias Braun" <matze at braunis.de>, "Clang Dev" <cfe-dev at lists.llvm.org>, "nd" <nd at arm.com>, "Abe Skolnik"
>>> <a.skolnik at samsung.com>
>>> Sent: Tuesday, October 11, 2016 6:33:43 AM
>>> Subject: Re: [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"
>>>
>>> On 11 October 2016 at 12:15, Sebastian Pop <sebpop.llvm at gmail.com>
>>> wrote:
>>> >>  1. Only test the non-FP-contracted output
>>> >
>>> > Yes, this is what I'm doing.
>>>
>>> If the whole test is about testing multiplications, what's the point
>>> of this?
>>>
>>>
>>> >>  2. Run the FP-contracted test only for a very small size (so that
>>> >>  we'll stay within some reasonable tolerance of the reference
>>> >>  output)
>>> >>  3. Change the matrix to something that will make the test
>>> >>  numerically stable (it does not look like the matrix itself
>>> >>  matters to the performance; where do the values come from?).
>>>
>>> 3 is more sound, 2 may be more practical.
>>>
>>>
>>> > -      C_StrictFP[i][j] = C[i][j] = ((DATA_TYPE) i*j) / ni;
>>> > -      B[i][j] = ((DATA_TYPE) i*j) / ni;
>>> > +      C_StrictFP[i][j] = C[i][j] = ((DATA_TYPE) i-j) / ni;
>>> > +      B[i][j] = ((DATA_TYPE) i-j) / ni;
>>> >      }
>>> >    for (i = 0; i < nj; i++)
>>> >      for (j = 0; j < nj; j++)
>>> > -      A[i][j] = ((DATA_TYPE) i*j) / ni;
>>> > +      A[i][j] = ((DATA_TYPE) i-j) / ni;
>>>
>>> Changing from multiplication to subtraction changes completely the
>>> nature of the test and goes towards "return 0;", ie, fiddling with
>>> the
>>> code so that the compiler "behaves" better. This is *not* a solution.

It is not uncommon to see in several polybench tests adjustments to
the initial values:

  /*
  LLVM: This change ensures we do not calculate nan values, which are
        formatted differently on different platforms and which may also
        be optimized unexpectedly.
  Original code:
  for (i = 0; i < ni; i++)
    for (j = 0; j < nj; j++) {
      A[i][j] = ((DATA_TYPE) i*j) / ni;
      Q[i][j] = ((DATA_TYPE) i*(j+1)) / nj;
    }
  for (i = 0; i < nj; i++)
    for (j = 0; j < nj; j++)
      R[i][j] = ((DATA_TYPE) i*(j+2)) / nj;
  */
  for (i = 0; i < ni; i++)
    for (j = 0; j < nj; j++) {
      A[i][j] = ((DATA_TYPE) i*j+ni) / ni;
      Q[i][j] = ((DATA_TYPE) i*(j+1)+nj) / nj;
    }
  for (i = 0; i < nj; i++)
    for (j = 0; j < nj; j++)
      R[i][j] = ((DATA_TYPE) i*(j+2)+nj) / nj;

git grepping gives us:

linear-algebra/kernels/cholesky/cholesky.c:  LLVM: This change ensures
we do not calculate nan values, which are
linear-algebra/kernels/cholesky/cholesky.c:      LLVM: This change
ensures we do not calculate nan values, which are
linear-algebra/kernels/cholesky/cholesky.c:      LLVM: This change
ensures we do not calculate nan values, which are
linear-algebra/kernels/trisolv/trisolv.c:  LLVM: This change ensures
we do not calculate nan values, which are
linear-algebra/solvers/gramschmidt/gramschmidt.c:  LLVM: This change
ensures we do not calculate nan values, which are
linear-algebra/solvers/lu/lu.c:  LLVM: This change ensures we do not
calculate nan values, which are