[llvm] InstCombine: Handle canonicalize in SimplifyDemandedFPClass (PR #173189)

[Matt Arsenault via llvm-commits]

================
@@ -2106,6 +2106,77 @@ Value *InstCombinerImpl::SimplifyDemandedUseFPClass(Value *V,
       Known.copysign(KnownSign);
       break;
     }
+    case Intrinsic::canonicalize: {
+      Type *EltTy = VTy->getScalarType();
+
+      // TODO: This could have more refined support for PositiveZero denormal
+      // mode.
+      if (EltTy->isIEEELikeFPTy()) {
+        DenormalMode Mode = F.getDenormalMode(EltTy->getFltSemantics());
+
+        FPClassTest SrcDemandedMask = DemandedMask;
+
+        // A demanded quiet nan result may have come from a signaling nan, so we
+        // need to expand the demanded mask.
+        if ((DemandedMask & fcQNan) != fcNone)
+          SrcDemandedMask |= fcSNan;
+
+        // This cannot have produced a signaling nan, so we can trim any snan
+        // inputs.
+        if ((DemandedMask & fcSNan) != fcNone)
----------------
arsenm wrote:

This needs to interpret the possible results, backwards. The assertion that the result cannot be snan does not imply the source is not snan as the quieting was guaranteed. We need to infer that the source could still be an snan, so we need to remove that assertion from the demanded source values 

https://github.com/llvm/llvm-project/pull/173189