[cfe-dev] [llvm-dev] Should isnan be optimized out in fast-math mode?

[Arthur O'Dwyer via cfe-dev]

On Thu, Sep 16, 2021 at 1:31 PM Serge Pavlov <sepavloff at gmail.com> wrote:

> On Tue, Sep 14, 2021 at 12:50 AM Serge Pavlov <sepavloff at gmail.com> wrote:
>
>> On Mon, Sep 13, 2021 at 11:46 PM Chris Tetreault <ctetreau at quicinc.com>
>> wrote:
>>
>>> … is guaranteed to work, and I read that fast-math enables the compiler
>>> to reason about constructs like `x + 0` being equal to `x`, then I’m going
>>> to be very confused when:
>>>
>> You are right, this was a bad idea. Compiler may optimize out `isnan` but
>> only when it deduces that the value cannot be NaN, but not due to the
>> user's promise. It is especially important for `isinf`. Addition of two
>> finite values may produce infinity and there is no universal way to predict
>> it. It is probably not an issue for types like float or double, but ML
>> cores use halfs or even minifloats, where overflow is much more probable.
>> If in the code:
>> ```
>> float r = a + b;
>> if (isinf(r)) {...
>> ```
>> `isinf` were optimized out just because -ffinite-math-only is in effect,
>> the user cannot check if overflow did not occur.
>>
>
> Rules proposed by Richard are also formulated using arguments, not
> results. Now there is no intention to optimize such a case.
>

Infinity (HUGE_VAL) is already not NaN, so this example doesn't have
anything to do with the NaN cases being discussed.
However, let's rephrase as a NaN situation:

    bool f1(float a, float b) {
        float r = a + b;
        return isnan(r);
    }
    bool result = f1(-HUGE_VAL, HUGE_VAL);  // expect "true"

Here, `a + b` can produce quiet-NaN (if `a` is -HUGE_VAL and `b`
is +HUGE_VAL).
By Richard Smith's -ffast-math proposal as I understand it, this quiet-NaN
result would be treated "as if" it were a signaling NaN.
Under IEEE 754, no operation ever produces a signaling NaN, so
unfortunately IEEE 754 can't guide us here; but intuitively, I think we'd
all say that merely *producing* a signaling NaN would not itself *cause* a
signal. So we store the quiet-NaN result in `r`.
Then we ask whether `isnan(r)`. The quiet-NaN result in `r` is used. By
Richard Smith's -ffast-math proposal as I understand it, any operation
*would* produce an unspecified result if it would raise a signal; but in
fact `isnan(r)` is a non-signaling operation, so even though we're treating
quiet-NaN as signaling-NaN, isnan(r) never raises any signal. So this code
has *well-defined behavior* in -ffast-math mode.
(And because the code's behavior is well-defined, therefore `isnan(r)` has
its usual meaning. When `r` holds a quiet-NaN, as in this case, `isnan(r)`
will correctly return `true`.)

I've googled, but failed to discover, whether comparison against a
signaling NaN is expected to signal. That is,

    bool f2(float a, float b) {
        float r = a + b;
        return (r != r);
    }
    bool result = f2(-HUGE_VAL, HUGE_VAL);  // expect "true" in IEEE754
mode, but perhaps "false" in -ffast-math mode

I'm really *hoping* that comparison is a signaling operation. If it is,
then according to Richard Smith's proposal as I understand it, the compiler
would be free to optimize `(r != r)` into `(false)` in -ffast-math mode.
(And, as a corollary, the compiler would *not* generally be free to
transform `isnan(r)` into `(r != r)`, because the latter expression has
more preconditions than the former.)

–Arthur

>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210916/1dd5e134/attachment.html>