[PATCH] D128261: [Fortran] Relax relative tolerance for FCVS tests

[Peter Klausler via Phabricator via llvm-commits]

klausler added a comment.

In D128261#3622933 <https://reviews.llvm.org/D128261#3622933>, @Meinersbur wrote:

> Some general remarks about floating point:
>
> - Floating point formatters have different approached they output. One approach is to emit the decimal representation that is closest to the number represented by the IEEE bit pattern. Because this can lead to a lot of trailing digits in scientific notations for what should be "0,1" exactly, a more recent approach is to find the decimal representations with the fewest number of digits that would still be parsed back to the same IEEE bit pattern <https://en.wikipedia.org/wiki/Floating-point_arithmetic#Binary-to-decimal_conversion_with_minimal_number_of_digits> (as @rovka mentions, but the implementation <https://github.com/llvm/llvm-project/blob/main/flang/lib/Decimal/binary-to-decimal.cpp> doesn't seem to do that). So it is possible that 2.50000003e-11 is the result of the first approach, where 2.5e-11 follows the second approach (assuming single precision).

See above.  f18's runtime will emit a minimal representation that reads back to the same bit representation (of the same real kind) when Fortran allows it to do so (viz., list-directed and NAMELIST output).

The binary-to-decimal conversion algorithm in flang/lib/Decimal is not the whole story.  See the code in flang/runtime/edit-output.cpp to see how the Decimal library is driven for all of Fortran's various output editing modes.

> - fpcmp was not written with whether two string representations parse to the same IEEE bit pattern in mind. Only to account for the general fuzziness that comes with floating point processing, including effects through `-ffast-math`, transcendental functions that only compute approximations, during string formatting, etc.. Considering that single only has ~7.2 significant mantissa bits, a relative tolerance of 1.0e-11 effectively only allows exact matches.

IEC599/IEEE754 32-bit single precision has 24 effective fractional bits, enough to accurately distinguish 7 decimal digits.

> - I think the tests should indeed consider the differences in floating point formatting as valid, and if single precision is a valid implementation choice for a Fortran compiler, increasing the tolerance seems right to me. I'd even go as far to increase it for all tests since that kind of approximation could occur with the other tests as well.

The hard part is that you can't tell from the output whether the runtime was editing single-precision data (or half, double, extended, or quad precision), and if you relax tolerances to allow single precision to differ, you can miss errors in double.  FPCMP probably can't detect output formatting errors of quad-precision data as it is.


Repository:
  rT test-suite

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D128261/new/

https://reviews.llvm.org/D128261