[llvm] [LoopVectorizer] Add support for partial reductions (PR #92418)

Graham Hunter via llvm-commits llvm-commits at lists.llvm.org
Thu Sep 12 03:19:43 PDT 2024


================
@@ -1556,7 +1556,105 @@ class LoopVectorizationCostModel {
   getReductionPatternCost(Instruction *I, ElementCount VF, Type *VectorTy,
                           TTI::TargetCostKind CostKind) const;
 
+  using PartialReductionList = DenseMap<Instruction *, PartialReductionChain>;
+
+  PartialReductionList getPartialReductionChains() {
+    return PartialReductionChains;
+  }
+
+  bool getInstructionsPartialReduction(Instruction *I,
+                                       PartialReductionChain &Chain) const {
+    auto PairIt = PartialReductionChains.find(I);
+    if (PairIt == PartialReductionChains.end())
+      return false;
+    Chain = PairIt->second;
+    return true;
+  }
+
+  void addPartialReductionIfSupported(Instruction *Instr, ElementCount VF) {
+    Value *ExpectedPhi;
+    Value *A, *B;
+
+    using namespace llvm::PatternMatch;
+    auto Pattern =
+        m_BinOp(m_OneUse(m_BinOp(m_OneUse(m_ZExtOrSExt(m_OneUse(m_Value(A)))),
+                                 m_OneUse(m_ZExtOrSExt(m_OneUse(m_Value(B)))))),
----------------
huntergr-arm wrote:

You need to make sure that the wider-than-legal vector type is never actually needed; m_OneUse on the extends was an easy way to guarantee that, but we could instead keep track of all uses of the extends to make sure they're accounted for with partial reductions.

The initial un-extended values never needed a m_OneUse, so those can indeed be removed outright.

https://github.com/llvm/llvm-project/pull/92418


More information about the llvm-commits mailing list