[LLVMdev] Optimization of sqrt() with invalid argument

[Bill Schmidt]

My colleague Will Schmidt recently discovered
(http://llvm.org/bugs/show_bug.cgi?id=21048) that the LLVM optimizers
are converting sqrt(n), where n is negative, into 0.0.  Now, as Sanjay
pointed out, the value of sqrt(n) for negative n other than -0.0 is
undefined according to the C99 standard, so this is allowable behavior.
But I think that it's not necessarily good behavior, nonetheless.

The problem is that an unoptimized call to sqrt(n) in libm will produce
a NaN, at least in the implementations I have access to.  So this
particular choice changes the behavior of optimized versus unoptimized
code, which of course is allowable, but probably to be avoided when
possible.

However, there's a de facto standard of producing NaN for these cases.
The gcc and xlc compilers will optimize the call into a constant NaN,
given the right options (-O2 -ffast-math for gcc, -O2 -qignerrno for
xlc).  All of gcc, icc, and xlc produce NaN via the library call with
unoptimized code.  Choosing 0.0 for Clang/LLVM introduces an unnecessary
incompatibility.

This is important in practice because of the xalanc benchmark in SPEC
CPU2006.  This code initializes a variable to a NaN value by assigning
it the value sqrt(-2.01) -- don't ask me why.  This variable is used to
terminate a loop, and when Clang/LLVM produces 0.0 instead of NaN, the
loop spins forever.

Are there any objections to changing the LLVM optimization behavior to
produce a constant NaN instead of 0.0 in this case?

Thanks,
Bill