[llvm] specify NaN behavior more precisely (PR #66579)

[Joshua Cranmer via llvm-commits]

================
@@ -3394,17 +3394,41 @@ Floating-Point Environment
 The default LLVM floating-point environment assumes that traps are disabled and
 status flags are not observable. Therefore, floating-point math operations do
 not have side effects and may be speculated freely. Results assume the
-round-to-nearest rounding mode.
+round-to-nearest rounding mode, and subnormals are assumed to be preserved.
+Running default LLVM code in an environment where these assumptions are not met
+can lead to undefined behavior.
+
+The representation bits of a floating-point value do not mutate arbitrarily; if
+there is no floating-point operation being performed, the NaN payload (if any)
+is preserved.
+
+When a floating-point math operation produces a NaN value, the result has a
+non-deterministic sign. The payload is non-deterministically chosen from the
+following set:
+
+- The payload that is all-zero except that the ``quiet`` bit is set.
+  ("Preferred NaN" case)
+- The payload of any input operand that is a NaN, bit-wise ORed with a payload that has
+  the ``quiet`` bit set. ("Quieting NaN propagation" case)
----------------
jcranmer-intel wrote:

> According to which definition?

Per IEEE 754-2019, section 6.2.1:
> In the preferred encoding just described, a signaling NaN shall be quieted by setting d1 to 1, leaving the remaining bits of T unchanged. Bits d2 d3 … dp−1 of the trailing significand field contain the encoding of the payload

https://github.com/llvm/llvm-project/pull/66579