[llvm] [LoopVectorizer] Add support for partial reductions (PR #92418)

Graham Hunter via llvm-commits llvm-commits at lists.llvm.org
Thu Sep 5 06:41:27 PDT 2024


================
@@ -6995,7 +7122,17 @@ LoopVectorizationPlanner::planInVPlanNativePath(ElementCount UserVF) {
 
 void LoopVectorizationPlanner::plan(ElementCount UserVF, unsigned UserIC) {
   assert(OrigLoop->isInnermost() && "Inner loop expected.");
-  CM.collectValuesToIgnore();
+
+  for (auto ReductionVar : Legal->getReductionVars()) {
+    auto *ReductionExitInstr = ReductionVar.second.getLoopExitInstr();
+    if (isInstrPartialReduction(ReductionExitInstr)) {
+      auto Chain = getPartialReductionInstrChain(ReductionExitInstr);
+      if (isPartialReductionChainValid(Chain, TTI))
----------------
huntergr-arm wrote:

The three functions called here seem to overlap a lot -- first check whether we have a possible partial reduction, then gather the operations in a Chain (blindly assuming an exact fit for the dot product sequence), then ask the target whether it's supported.

While we can certainly have static helpers if appropriate, I think there should be a single interface (in the cost model) which evaluates and records partial reduction candidates per VF. The presence of a partial reduction in a chosen vplan can then be used to prevent epilogue vectorization until we decide to support it properly.

https://github.com/llvm/llvm-project/pull/92418


More information about the llvm-commits mailing list