[PATCH] D128261: [Fortran] Relax relative tolerance for FCVS tests

[Michael Kruse via Phabricator via llvm-commits]

Meinersbur added a comment.

Some general remarks about floating point:

- Floating point formatters have different approached they output. One approach is to emit the decimal representation that is closest to the number represented by the IEEE bit pattern. Because this can lead to a lot of trailing digits in scientific notations for what should be "0,1" exactly, a more recent approach is to find the decimal representations with the fewest number of digits that would still be parsed back to the same IEEE bit pattern <https://en.wikipedia.org/wiki/Floating-point_arithmetic#Binary-to-decimal_conversion_with_minimal_number_of_digits> (as @rovka mentions, but the implementation <https://github.com/llvm/llvm-project/blob/main/flang/lib/Decimal/binary-to-decimal.cpp> doesn't seem to do that). So it is possible that 2.50000003e-11 is the result of the first approach, where 2.5e-11 follows the second approach (assuming single precision).
- fpcmp was not written with whether two string representations parse to the same IEEE bit pattern in mind. Only to account for the general fuzziness that comes with floating point processing, including effects through `-ffast-math`, transcendental functions that only compute approximations, during string formatting, etc.. Considering that single only has ~7.2 significant mantissa bits, a relative tolerance of 1.0e-11 effectively only allows exact matches.
- I think the tests should indeed consider the differences in floating point formatting as valid, and if single precision is a valid implementation choice for a Fortran compiler, increasing the tolerance seems right to me. I'd even go as far to increase it for all tests since that kind of approximation could occur with the other tests as well.



In D128261#3622187 <https://reviews.llvm.org/D128261#3622187>, @awarzynski wrote:

>> so this patch does 2 things:
>
> 🤔 I see only one change :)

A previous version also updated the reference output to "2.5e-11" as well, which you get you run `driver_run` using flang to update all reference outputs. With the fpcmp tolerance it does not matter which one is in the reference file, but I assume "2.5e-11" would be closer to what the result would be with infinite (/double?) precision and hence IMHO the one that we should put into the reference output.


Repository:
  rT test-suite

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D128261/new/

https://reviews.llvm.org/D128261