[llvm] specify NaN behavior more precisely (PR #66579)

[via llvm-commits]

llvmbot wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-ir

<details>
<summary>Changes</summary>

The current docs  don't say anything about the possible set of values returned by a NaN-producing operation. However, there are some coding patterns where such guarantees are strongly desired, such as NaN boxing (as used e.g. in the Spidermonkey JS engine). And in fact the set of possible payloads that can be observed today when compiling code via LLVM is fairly limited (based on what optimizations and backends and hardware currently do) -- but without a documented guarantee, languages compiling to LLVM cannot really rely on this.

There was a long discussion about the exact shape that the guarantees should take [on Discourse](https://discourse.llvm.org/t/stronger-floating-point-nan-guarantees/72165). This PR transcribes what I perceived as the consensus, plus some further clarifications.

The point I am least sure about is the status of the min/max operations on x86: the operations provided by hardware do not match the IEEE spec, so the backend has to emit a sequence of instructions that implement the IEEE operations; the bitwise operations used by the backend to achieve this could run afoul of the NaN guarantees. [Here](https://discourse.llvm.org/t/stronger-floating-point-nan-guarantees/72165/60) @<!-- -->jyknight stated that what the backend does might be compatible with the NaN guarantee -- but they didn't say that it affirmatively *is* compatible, so it would be great if someone with knowledge of the x86 backend could confirm or deny this.
---
Full diff: https://github.com/llvm/llvm-project/pull/66579.diff


1 Files Affected:

- (modified) llvm/docs/LangRef.rst (+30-6) 


``````````diff
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index f542e70bcfee810..021e7ba2fb41722 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3394,17 +3394,41 @@ Floating-Point Environment
 The default LLVM floating-point environment assumes that traps are disabled and
 status flags are not observable. Therefore, floating-point math operations do
 not have side effects and may be speculated freely. Results assume the
-round-to-nearest rounding mode.
+round-to-nearest rounding mode, and subnormals are assumed to be preserved.
+Running default LLVM code in an environment where these assumptions are not met
+can lead to undefined behavior.
+
+The representation bits of a floating-point value do not mutate arbitrarily; if
+there is no floating-point operation being performed, the NaN payload (if any)
+is preserved.
+
+When a floating-point math operation produces a NaN value, the result has a
+non-deterministic sign. The payload is non-deterministically chosen from the
+following set:
+
+- The payload that is all-zero except that the ``quiet`` bit is set.
+  ("Preferred NaN" case)
+- The payload of any input operand that is a NaN, bit-wise ORed with a payload that has
+  the ``quiet`` bit set. ("Quieting NaN propagation" case)
+- The payload of any input operand that is a NaN. ("Unchanged NaN propagation" case)
+- A target-specific set of further NaN payloads, that definitely all have their
+  ``quiet`` bit set. The set can depend on the payloads of the input NaNs.
+  This set is empty on x86 and ARM, but can be non-empty on other architectures.
+  (For instance, on wasm, if any input NaN is not the preferred NaN, then
+  this set contains all quiet NaNs; otherwise, it is empty.
+  On SPARC, this set consists of the all-one payload.)
+
+In particular, if all input NaNs are quiet, then the output NaN is definitely
+quiet. Signaling NaN outputs can only occur if they are provided as an input
+value. For example, "fmul SNaN, 1.0" may be simplified to SNaN rather than QNaN.
 
 Floating-point math operations are allowed to treat all NaNs as if they were
-quiet NaNs. For example, "pow(1.0, SNaN)" may be simplified to 1.0. This also
-means that SNaN may be passed through a math operation without quieting. For
-example, "fmul SNaN, 1.0" may be simplified to SNaN rather than QNaN. However,
-SNaN values are never created by math operations. They may only occur when
-provided as a program input value.
+quiet NaNs. For example, "pow(1.0, SNaN)" may be simplified to 1.0.
 
 Code that requires different behavior than this should use the
 :ref:`Constrained Floating-Point Intrinsics <constrainedfp>`.
+In particular, constrained intrinsics rule out the "Unchanged NaN propagation" case;
+they are guaranteed to return a QNaN.
 
 .. _fastmath:
 

``````````

</details>


https://github.com/llvm/llvm-project/pull/66579