[llvm-dev] ROTR availability detected inconsistently

[Harold Aptroot via llvm-dev]

Hi,

Apologies for the bit of a repost, last time was in the holidays and
did not receive much attention.
In short, using isOperationLegalOrCustom(ISD::ROTR, VT) in
TargetLowering returns an inconsistent result.
The context of this is the optimization of divisibility tests, where a
test of the form "less than c and the k lowest bits are zero" is used,
which can be rolled into one test by using a right rotate by k.
The attached path dumps the result of the ROTR check in err() in a
kind of lame way. It can prints things such as
HasROTR: 0
HasROTR: 1
HasROTR: 0
HasROTR: 1
HasROTR: 1
HasROTR: 0
HasROTR: 1
Which I find very odd, the legality of an instruction shouldn't be
changing like that, right?
What's going on and what should I do about it in the context of what I
am trying to do?

Regards,
Harold
-------------- next part --------------
Index: lib/CodeGen/SelectionDAG/TargetLowering.cpp
===================================================================
--- lib/CodeGen/SelectionDAG/TargetLowering.cpp	(revision 256471)
+++ lib/CodeGen/SelectionDAG/TargetLowering.cpp	(working copy)
@@ -1790,6 +1790,42 @@
         }
       }
     }
+
+    if ((Cond == ISD::SETEQ || Cond == ISD::SETNE) &&
+        C1.isMinValue() &&
+        N0.getOpcode() == ISD::UREM &&
+        N0.hasOneUse()) {
+      if (ConstantSDNode *RemRHS =
+            dyn_cast<ConstantSDNode>(N0.getOperand(1))) {
+        const APInt &RemRHSC = RemRHS->getAPIntValue();
+        if (!RemRHSC.isPowerOf2()) {
+          bool HasROTR = isOperationLegalOrCustom(ISD::ROTR, VT);
+          errs() << "HasROTR: " << HasROTR << "\n";
+          auto &DL = DAG.getDataLayout();
+          auto Ty = N0.getValueType();
+          auto shTy = getShiftAmountTy(Ty, DL);
+          auto NewCond = (Cond == ISD::SETEQ) ? ISD::SETULE : ISD::SETUGE;
+          unsigned rotAmt = RemRHSC.countTrailingZeros();
+          unsigned W = Ty.getSizeInBits();
+          APInt Mod(W + 1, 0);
+          Mod.setBit(W);
+          APInt Inverse = RemRHSC.lshr(rotAmt).zext(W + 1)
+                          .multiplicativeInverse(Mod).trunc(W);
+          APInt Maxval = APInt::getMaxValue(W).udiv(RemRHSC);
+          SDValue Max = DAG.getConstant(Maxval, dl, Ty);
+          SDValue Mul = DAG.getNode(ISD::MUL, dl, Ty, N0.getOperand(0),
+                                    DAG.getConstant(Inverse, dl, Ty));
+          if (rotAmt && HasROTR) {
+            SDValue Ror = DAG.getNode(ISD::ROTR, dl, Ty, Mul,
+                                      DAG.getConstant(rotAmt, dl, shTy));
+            return DAG.getSetCC(dl, VT, Ror, Max, NewCond);
+          }
+          else if (!rotAmt) {
+            return DAG.getSetCC(dl, VT, Mul, Max, NewCond);
+          }
+        }
+      }
+    }
   }
 
   if (isa<ConstantFPSDNode>(N0.getNode())) {