[llvm] specify NaN behavior more precisely (PR #66579)

[James Y Knight via llvm-commits]

jyknight wrote:

The section "What happens to the payload when the precision is converted?" in [This 2018 paper](https://grouper.ieee.org/groups/msc/ANSI_IEEE-Std-754-2019/background/nan-propagation.pdf) claimed that all known implementations preserve the upper NaN bits when truncating, and append zeros when extending.

Of course, that's only the case for the ISAs which propagate NaN payloads at all (so, not RISC-V and not WebAssembly). On RISCV, the sign is always positive and the NaN canonical. On Wasm, the sign is non-deterministic, and the payload follows Wasm's usual "nondeterministic result if given non-canonical input NaNs, canonical NaN with nondeterministic sign otherwise" rule.

Given that these are the exact same issues that arise for all other FP instructions, I think these instructions should use our normal rules, just clarified to say how the bits are copied *if* they happen to be copied.

Thus, I suggest the spec should say something like this for `fptrunc`,
> NaN values follow the usual :ref:\`NaN behaviors \<floatnan\>\`, except that \_if\_ a NaN payload is copied from the input, then the low order bits of the NaN payload which cannot fit in the resulting type are discarded.

And for `fpext`:
> NaN values follow the usual :ref:\`NaN behaviors \<floatnan\>\`, except that \_if\_ a NaN payload is copied from the input, then it is copied to the high order bits of the resulting payload, and the remaining low order bits are zero.

https://github.com/llvm/llvm-project/pull/66579