[llvm] [WIP][DAG] Add legalization handling for AVGCEIL/AVGFLOOR nodes (PR #92096)

[Jay Foad via llvm-commits]

================
@@ -4752,6 +4752,19 @@ unsigned SelectionDAG::ComputeNumSignBits(SDValue Op, const APInt &DemandedElts,
         (VTBits - SignBitsOp0 + 1) + (VTBits - SignBitsOp1 + 1);
     return OutValidBits > VTBits ? 1 : VTBits - OutValidBits + 1;
   }
+  case ISD::AVGCEILS:
+  case ISD::AVGFLOORS:
+    // Treat the AVG nodes like ADD/SUB nodes as that's the only stage that
+    // can lose sign bits:
+    // avgceils(lhs, rhs) -> sub(or(lhs,rhs),ashr(xor(lhs,rhs),1))
+    // avgfloors(lhs, rhs) -> add(and(lhs,rhs),ashr(xor(lhs,rhs),1))
+    Tmp = ComputeNumSignBits(Op.getOperand(0), DemandedElts, Depth + 1);
+    if (Tmp == 1)
+      break; // Early out.
+    Tmp2 = ComputeNumSignBits(Op.getOperand(1), DemandedElts, Depth + 1);
+    if (Tmp2 == 1)
+      break; // Early out.
+    return std::min(Tmp, Tmp2) - 1;
----------------
jayfoad wrote:

> No, I'm starting to wonder if we should have some form of SignBits equivalent to KnownBits so we can share more valuetracking code - what do you think?

We could create something similar to KnownBits that calculates numsignbits for an operation based on the numsignbits of its inputs, but I'm not sure how useful that would be. My hunch is that most of the useful stuff that happens in numsignbits calculations relies on more information than just the numsignbits of the inputs.

https://github.com/llvm/llvm-project/pull/92096