[llvm] [RISCV] Widen i1 AnyOf reductions (PR #134898)
Craig Topper via llvm-commits
llvm-commits at lists.llvm.org
Tue Apr 8 22:23:36 PDT 2025
================
@@ -103,6 +105,82 @@ bool RISCVCodeGenPrepare::visitAnd(BinaryOperator &BO) {
return true;
}
+// With EVL tail folding, an AnyOf reduction will generate an i1 vp.merge like
+// follows:
+//
+// loop:
+// %phi = phi <vscale x 4 x i1> [ zeroinitializer, %entry ], [ %rec, %loop ]
+// %cmp = icmp ...
+// %or = or <vscale x 4 x i1> %phi, %cmp
+// %rec = call <vscale x 4 x i1> @llvm.vp.merge(%mask, %or, %phi, %evl)
+// ...
+// middle:
+// %res = call i1 @llvm.vector.reduce.or(<vscale x 4 x i1> %rec)
+//
+// However RVV doesn't have any tail undisturbed mask instructions and so we
+// need a convoluted sequence of mask instructions to lower the i1 vp.merge: see
+// llvm/test/CodeGen/RISCV/rvv/vpmerge-sdnode.ll.
+//
+// To avoid that this widens the i1 vp.merge to an i8 vp.merge, which will
+// usually be folded into a masked vor.vv.
+//
+// loop:
+// %phi = phi <vscale x 4 x i8> [ zeroinitializer, %entry ], [ %rec, %loop ]
+// %cmp = icmp ...
+// %zext = zext <vscale x 4 x i1> %cmp to <vscale x 4 x i8>
+// %or = or <vscale x 4 x i8> %phi, %cmp
+// %rec = call <vscale x 4 x i8> @llvm.vp.merge(%mask, %or, %phi, %evl)
+// %trunc = trunc <vscale x 4 x i8> %rec to <vscale x 4 x i1>
+// ...
+// middle:
+// %res = call i1 @llvm.vector.reduce.or(<vscale x 4 x i1> %rec)
+//
+// The trunc will normally be sunk outside of the loop, but even if there are
+// users inside the loop it is still profitable.
+bool RISCVCodeGenPrepare::widenVPMerge(IntrinsicInst &II) {
+ if (!II.getType()->getScalarType()->isIntegerTy(1))
+ return false;
+
+ Value *Mask, *PhiV, *Cond, *EVL;
+
+ using namespace PatternMatch;
+ if (!match(&II,
+ m_Intrinsic<Intrinsic::vp_merge>(
+ m_Value(Mask), m_OneUse(m_c_Or(m_Value(PhiV), m_Value(Cond))),
+ m_Deferred(PhiV), m_Value(EVL))))
+ return false;
+
+ auto *Phi = dyn_cast<PHINode>(PhiV);
+ auto *Start = dyn_cast<Constant>(Phi->getIncomingValue(0));
----------------
topperc wrote:
I think it is possible to have a phi with no operands so I'm not sure you can assume value 0 exists.
https://github.com/llvm/llvm-project/pull/134898
More information about the llvm-commits
mailing list