[llvm] [VectorCombine] New folding pattern for extract/binop/shuffle chains (PR #145232)
Yingwei Zheng via llvm-commits
llvm-commits at lists.llvm.org
Sun Aug 17 10:53:02 PDT 2025
================
@@ -2988,6 +2989,305 @@ bool VectorCombine::foldShuffleFromReductions(Instruction &I) {
return foldSelectShuffle(*Shuffle, true);
}
+/// For a given chain of patterns of the following form:
+///
+/// ```
+/// %1 = shufflevector <n x ty1> %0, <n x ty1> poison <n x ty2> mask
+///
+/// %2 = tail call <n x ty1> llvm.<umin/umax/smin/smax>(<n x ty1> %0, <n x
+/// ty1> %1)
+/// OR
+/// %2 = add/mul/or/and/xor <n x ty1> %0, %1
+///
+/// %3 = shufflevector <n x ty1> %2, <n x ty1> poison <n x ty2> mask
+/// ...
+/// ...
+/// %(i - 1) = tail call <n x ty1> llvm.<umin/umax/smin/smax>(<n x ty1> %(i -
+/// 3), <n x ty1> %(i - 2)
+/// OR
+/// %(i - 1) = add/mul/or/and/xor <n x ty1> %(i - 3), %(i - 2)
+///
+/// %(i) = extractelement <n x ty1> %(i - 1), 0
+/// ```
+///
+/// Where:
+/// `mask` follows a partition pattern:
+///
+/// Ex:
+/// [n = 8, p = poison]
+///
+/// 4 5 6 7 | p p p p
+/// 2 3 | p p p p p p
+/// 1 | p p p p p p p
+///
+/// For powers of 2, there's a consistent pattern, but for other cases
+/// the parity of the current half value at each step decides the
+/// next partition half (see `ExpectedParityMask` for more logical details
+/// in generalising this).
+///
+/// Ex:
+/// [n = 6]
+///
+/// 3 4 5 | p p p
+/// 1 2 | p p p p
+/// 1 | p p p p p
+bool VectorCombine::foldShuffleChainsToReduce(Instruction &I) {
+ // Going bottom-up for the pattern.
+ auto *EEI = dyn_cast<ExtractElementInst>(&I);
+ if (!EEI)
+ return false;
+
+ std::queue<Value *> InstWorklist;
+ InstructionCost OrigCost = 0;
+
+ Value *InitEEV = nullptr;
+
+ // Common instruction operation after each shuffle op.
+ unsigned int CommonCallOp = 0;
+ Instruction::BinaryOps CommonBinOp = Instruction::BinaryOpsEnd;
+
+ bool IsFirstCallOrBinInst = true;
+ bool ShouldBeCallOrBinInst = true;
+
+ // This stores the last used instructions for shuffle/common op.
+ //
+ // PrevVecV[2] stores the first vector from extract element instruction,
+ // while PrevVecV[0] / PrevVecV[1] store the last two simultaneous
+ // instructions from either shuffle/common op.
+ SmallVector<Value *, 3> PrevVecV(3, nullptr);
+
+ Value *VecOp;
+ if (!match(&I, m_ExtractElt(m_Value(VecOp), m_Zero())))
+ return false;
+
+ auto *FVT = dyn_cast<FixedVectorType>(VecOp->getType());
+ if (!FVT)
+ return false;
+
+ int64_t VecSize = FVT->getNumElements();
+ if (VecSize < 2)
+ return false;
+
+ // Number of levels would be ~log2(n), considering we always partition
+ // by half for this fold pattern.
+ unsigned int NumLevels = Log2_64_Ceil(VecSize), VisitedCnt = 0;
+ int64_t ShuffleMaskHalf = 1, ExpectedParityMask = 0;
+
+ // This is how we generalise for all element sizes.
+ // At each step, if vector size is odd, we need non-poison
+ // values to cover the dominant half so we don't miss out on any element.
+ //
+ // This mask will help us retrieve this as we go from bottom to top:
+ //
+ // Mask Set -> N = N * 2 - 1
+ // Mask Unset -> N = N * 2
+ for (int Cur = VecSize, Mask = NumLevels - 1; Cur > 1;
+ Cur = (Cur + 1) / 2, --Mask) {
+ if (Cur & 1)
+ ExpectedParityMask |= (1ll << Mask);
+ }
+
+ PrevVecV[2] = VecOp;
+ InitEEV = EEI;
----------------
dtcxzyw wrote:
```suggestion
```
https://github.com/llvm/llvm-project/pull/145232
More information about the llvm-commits
mailing list