[llvm] specify NaN behavior more precisely (PR #66579)

[Nuno Lopes via llvm-commits]

nunoplopes wrote:

> > I've a question regarding fpext & fptrunc: what's the payload we want there? We can't just copy the inputs.
> 
> `fpext` and `fptrunc` correspond to the IEEE 754 operation **convertFormat**, so the rules related to SNaN handling need to apply (i.e., it may or may not be quieting input SNaN). IEEE 754 furthermore recommends NaN propagation as follows:
> 
> > Conversion of a quiet NaN from a narrower format to a wider format in the same radix, and then back to
> > the same narrower format, should not change the quiet NaN payload in any way except to make it
> > canonical.
> 
> The natural way to implement this is to designate certain bits to chop, but I don't know what the full suite of implementations here is. In x86, the afflicted bits are the LSB, so `fpext` on `0xffffffff` results in `0xffffffffe0000000`, and I think `APFloat` is following the same rules here, but I'm not certain all NaN-propagating implementations would follow the same rules.

I've implemented what you suggested and the 4 cases above now verify.
I have zero regressions on the LLVM test suite with the proposed semantics! 🚀

https://github.com/llvm/llvm-project/pull/66579