[llvm] [LV][AArch64] Provide option to use partial reductions by default (PR #216001)

Benjamin Maxwell via llvm-commits llvm-commits at lists.llvm.org
Wed Aug 26 06:18:07 PDT 2026


================
@@ -5230,13 +5240,56 @@ void VPlanTransforms::createPartialReductions(VPlan &Plan,
   MapVector<VPReductionPHIRecipe *, SmallVector<VPPartialReductionChain>>
       ChainsByPhi;
   VPBasicBlock *HeaderVPBB = Plan.getVectorLoopRegion()->getEntryBasicBlock();
+  SmallVector<VPReductionPHIRecipe *, 4> UnorderedReductions;
   for (VPRecipeBase &R : HeaderVPBB->phis()) {
     auto *RedPhiR = dyn_cast<VPReductionPHIRecipe>(&R);
     if (!RedPhiR)
       continue;
 
     if (auto Chains = getScaledReductions(RedPhiR))
       ChainsByPhi.try_emplace(RedPhiR, std::move(*Chains));
+    else if (UsePartialReduceByDefault &&
+             (RedPhiR->getRecurrenceKind() == RecurKind::Add ||
+              (RedPhiR->getRecurrenceKind() == RecurKind::FAdd &&
+               !RedPhiR->isOrdered() && !RedPhiR->isInLoop())))
+      UnorderedReductions.push_back(RedPhiR);
+  }
+
+  // For general unordered reductions which aren't part of a candidate
+  // chain for a scaled partial reduction, we can potentially still use
+  // the intrinsic to allow for more optimization later on.
+  for (auto *Rdx : UnorderedReductions) {
+    auto *Backedge = dyn_cast<VPWidenRecipe>(Rdx->getBackedgeValue());
+    VPValue *OtherOp;
+    if (!Backedge ||
+        !match(Backedge,
+               m_CombineOr(m_c_FAdd(m_Specific(Rdx), m_VPValue(OtherOp)),
+                           m_c_Add(m_Specific(Rdx), m_VPValue(OtherOp)))))
----------------
MacDue wrote:

Also, what about sub? I think we could support that too with a negation before the reduce. 

https://github.com/llvm/llvm-project/pull/216001


More information about the llvm-commits mailing list