[llvm] [X86] Lower vector integer division and remainder through float division (PR #205263)

[Adam Scott via llvm-commits]

================
@@ -50564,6 +50572,123 @@ static SDValue combineMulToPMADD52(SDNode *N, const SDLoc &DL,
   return SDValue();
 }
 
+// x86 has no vector integer divide instructions. Lower vector
+// UDIV/SDIV/UREM/SREM through float division instead of scalarizing into N
+// scalar hardware divides.
+static SDValue combineIntDivRem(SDNode *N, SelectionDAG &DAG,
+                                TargetLowering::DAGCombinerInfo &DCI,
+                                const X86Subtarget &Subtarget) {
+  EVT VT = N->getValueType(0);
+  SDLoc DL(N);
+
+  // Vector division never survives op legalization to reach later rounds.
+  if (!VT.isVector() || !Subtarget.hasSSE2())
+    return SDValue();
+
+  SDValue Dividend = N->getOperand(0);
+  SDValue Divisor = N->getOperand(1);
+  unsigned Opc = N->getOpcode();
+
+  // Disabled lanes are poison and fdiv never traps, so ignore the mask.
+  if (Opc == ISD::MASKED_UDIV || Opc == ISD::MASKED_SDIV ||
+      Opc == ISD::MASKED_UREM || Opc == ISD::MASKED_SREM)
+    Opc = ISD::getUnmaskedBinOpOpcode(Opc);
+  bool IsRem = Opc == ISD::UREM || Opc == ISD::SREM;
+  bool IsSigned = Opc == ISD::SDIV || Opc == ISD::SREM;
+
+  // If the result is only read back as scalar extracts, scalarization computes
+  // just the demanded lanes.
+  if (all_of(N->users(), [](const SDNode *U) {
+        return U->getOpcode() == ISD::EXTRACT_VECTOR_ELT;
+      }))
+    return SDValue();
+
+  // Magic multiply lowers constant divisors cheaper than a divide.
+  if (DAG.isConstantIntBuildVectorOrConstantInt(Divisor))
+    return SDValue();
+
+  unsigned EltBits = VT.getScalarSizeInBits();
+  auto FitsFP = [&](SDValue V, const fltSemantics &Sem) {
+    unsigned Precision = APFloat::semanticsPrecision(Sem);
+    return IsSigned ? DAG.ComputeNumSignBits(V) + Precision > EltBits
+                    : DAG.computeKnownBits(V).countMaxActiveBits() <= Precision;
+  };
+  auto BothFitFP = [&](const fltSemantics &Sem) {
+    return FitsFP(Dividend, Sem) && FitsFP(Divisor, Sem);
+  };
+  // i8/i16/i32: the operands fit the float mantissa
+  // exactly so one float divide recovers the exact quotient.
+  if (EltBits <= 32) {
+    // f32 recovers the quotient exactly when both operands fit in 24 bits
+    MVT FPSclVT = MVT::f64;
+    if (EltBits <= 16 || BothFitFP(APFloat::IEEEsingle()))
+      FPSclVT = MVT::f32;
+    EVT FPVT = VT.changeVectorElementType(*DAG.getContext(), FPSclVT);
+
+    // Nothing will split an illegal FP type after type legalization, so halve
+    // the divide while the integer halves stay legal.
+    if (!DCI.isBeforeLegalize() &&
+        !DAG.getTargetLoweringInfo().isTypeLegal(FPVT)) {
+      EVT HalfVT = VT.getHalfNumVectorElementsVT(*DAG.getContext());
+      if (DAG.getTargetLoweringInfo().isTypeLegal(HalfVT))
+        return splitVectorIntBinary(SDValue(N, 0), DAG, DL);
+      return SDValue();
+    }
+
+    bool IsStrict = DAG.getMachineFunction().getFunction().hasFnAttribute(
+        Attribute::StrictFP);
+    if (IsStrict) {
+      // The SAE forms are 512-bit only. Inputs widen into a zmm below, which
+      // requires 512-bit types to be legal.
+      if (!Subtarget.useAVX512Regs())
+        return SDValue();
+      // More lanes than one zmm divide can hold so split the divide.
+      if (FPVT.getSizeInBits() > 512)
+        return splitVectorIntBinary(SDValue(N, 0), DAG, DL);
----------------
as4230 wrote:

Yes it is possible and it actually caused sdiv <9 x i32> under strictfp to hit the assert in splitVectorIntBinary since 288 bits is neither 256 nor 512.

Non pow2 shapes now return SDValue() but I couldnt do that for everything because the combine round after type legalization seems to only run if the legalizer changed something, so functions where the types are already legal never get it. strict sdiv <16 x i32> drops from 2 vdivpd to 16 idivl if I just bail on it.

So I moved the halving down below the gates instead and both paths share it now. The strict branch has no split of its own anymore and the shared one only runs when the type is something splitVectorIntBinary accepts.

https://github.com/llvm/llvm-project/pull/205263