[llvm] [SelectionDAG] Use `KnownBits` to determine if an operand may be NaN. (PR #188606)

[Robert Imschweiler via llvm-commits]

================
@@ -6251,6 +6193,38 @@ bool SelectionDAG::isKnownNeverNaN(SDValue Op, const APInt &DemandedElts,
                                                Depth);
     }
 
+    // Try to infer NaN from known bits, but only for detecting signaling or
+    // nonsignaling NaNs
+    EVT VT = Op.getValueType();
+    if (!SNaN && VT.isFloatingPoint()) {
+      const fltSemantics &FltSem = VT.getFltSemantics();
+      if (FltSem.precision > 0) {
+        KnownBits Known = computeKnownBits(Op, DemandedElts);
+        unsigned Mantissa = FltSem.precision - 1;
+        unsigned Exponent = FltSem.sizeInBits - FltSem.precision;
+        KnownBits KnownMan = Known.extractBits(Mantissa, 0);
+        KnownBits KnownExp = Known.extractBits(Exponent, Mantissa);
+
+        switch (FltSem.nanEncoding) {
+        case fltNanEncoding::IEEE: {
+          if (!KnownExp.getMaxValue().isAllOnes() || KnownMan.isZero())
+            return true;
+          break;
+        }
+        case fltNanEncoding::AllOnes: {
+          if (!KnownExp.getMaxValue().isAllOnes() ||
+              !KnownMan.getMaxValue().isAllOnes())
+            return true;
+          break;
+        }
+        case fltNanEncoding::NegativeZero:
+          if (Known.Zero.isSignBitSet() || !KnownExp.isZero() ||
----------------
ro-i wrote:

`KnownExp.isZero()` - should that test if it's not all-zeroes? Because I don't think it does that in the current state. It only says that it's not *known* to be zero, but it *could* be. You would probably rather need to use `getMinValue()` or check the `One` component. Similar issue with `KnownMan.isZero()`, I guess

https://github.com/llvm/llvm-project/pull/188606