[LLVMdev] Constant Propagation of Floating Point NaN

[John Leidel (jleidel)]

This may be somewhat of a bizarre question that may or may not spark an interesting discussion.  I recently had an interesting test case land on my desk that involves testing the integrity of the NaN signals of various floating point arithmetic operations.  The test deliberately multiplies two IEEE double precision floating point values together where one value is a quiet NaN, the other a signaling NaN.  The user is attempting to test the sensitivity of operations that produce quiet/signaling NaN's on GCC and LLVM 3.4.  We realize that the IEEE standard states the floating point arithmetic isn't necessarily commutative.  However, we're actually seeing the resulting signaling/quiet NaN change depending upon the order of operations.  The 2008 IEEE Spec (IEEE 754-2008) Section 7.2 states: 

"For operations producing results in floating-point format, the default result of an operation that signals the invalid operation exception shall be a quiet NaN”  

The test passes fine at -O0, but we see the issue at -O1.  The way the test is written, it triggers the constant propagation to occur and force the compiler to perform the arithmetic.  Depending upon the order of the input values, the results differ (which is fine per the spec), but the NaN's also differ.  GCC produces the quiet NaN result at -O1, but only when we pass the '-fsignaling-nans' flag (which is ignored in CLANG 3.4).  

We're using CLANG/LLVM 3.4 release compiled from source on x86_64.  


Any thoughts??

The algorithm can be distilled to: 
uint64_t quiet_NaN_i = 0x7FF8000000000000;
uint64_t signaling_NaN_i = 0x7FF0000001000000;
uint64_t quiet_location_bit = 0x0008000000000000;
double x;
double quiet_NaN = *(double *) &quiet_NaN_i;
double signaling_NaN = *(double *) &signaling_NaN_i;       


/* correct?: produces a quiet NaN */
x = quiet_NaN * signaling_NaN;*/

/* incorrect?: produces a signaling NaN */
x = signaling_NaN * quiet_NaN;

IR: 
/* correct version: quiet_NaN * signaling_NaN */
%call6 = tail call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([15 x i8]* @.str6, i64 0, i64 0), double 0x7FF8000000000000, i64 9221120237041090560) #2

/* incorrect version: signaling_NaN * quiet_NaN */
%call6 = tail call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([15 x i8]* @.str6, i64 0, i64 0), double 0x7FF0000001000000, i64 9218868437244182528) #2

Full Source: 
#include <stdio.h>
#include <stdlib.h>
#include <stdint.h>

int
main()
{
    uint64_t x_i;
    uint64_t quiet_NaN_i = 0x7FF8000000000000;
    uint64_t signaling_NaN_i = 0x7FF0000001000000;
    uint64_t quiet_location_bit = 0x0008000000000000;
    double x;
    double quiet_NaN = *(double *) &quiet_NaN_i;
    double signaling_NaN = *(double *) &signaling_NaN_i;
    printf("Multiplication should never return a signaling NaN\n");
    printf("A quiet NaN has an initial fraction bit of 1, that is \n");
    printf(" & with %16.16lx returns a non-zero\n", quiet_location_bit);

    printf("\nThis is a quiet_NaN %16.16lx\n", quiet_NaN_i);
    printf("This is a signaling_NaN %16.16lx\n", signaling_NaN_i);
    printf("perform x = %16.16lx * %16.16lx\n", signaling_NaN_i, quiet_NaN_i);

    /* uncomment for correct results: x = quiet_NaN * signaling_NaN;*/
    x = signaling_NaN * quiet_NaN;
    x_i = *(uint64_t *)&x;

    printf("x %e %16.16lx\n", x, x_i);
    if (x_i == quiet_NaN_i)
        printf(" output equals quiet_NaN input \n");
    else if (x_i == signaling_NaN_i)
        printf(" output equals signaling_NaN input \n");
    else
        printf(" output equals neither input \n");

    x_i = x_i & quiet_location_bit;
    if (x_i != 0)
        printf(" output is a quiet NaN \n");
    else
        printf(" output is a signaling NaN \n");
    return (0);
}
	

John D. Leidel