[llvm] [LoopVectorizer] Add support for partial reductions (PR #92418)
Sam Tebbs via llvm-commits
llvm-commits at lists.llvm.org
Mon Nov 18 09:14:20 PST 2024
================
@@ -8663,6 +8663,113 @@ VPReplicateRecipe *VPRecipeBuilder::handleReplication(Instruction *I,
return Recipe;
}
+void VPRecipeBuilder::collectScaledReductions(VFRange &Range) {
+ // Find all possible partial reductions
+ SmallVector<PartialReductionChain, 1> PartialReductionChains;
+ for (const auto &[Phi, RdxDesc] : Legal->getReductionVars())
+ if (std::optional<PartialReductionChain> Chain =
+ getScaledReduction(Phi, RdxDesc, Range))
+ PartialReductionChains.push_back(*Chain);
+
+ // A partial reduction is invalid if any of its extends are used by
+ // something that isn't another partial reduction. This is because the
+ // extends are intended to be lowered along with the reduction itself.
+
+ // Build up a set of partial reduction bin ops for efficient use checking
+ SmallSet<User *, 4> PartialReductionBinOps;
+ for (auto It : PartialReductionChains) {
+ if (It.BinOp)
+ PartialReductionBinOps.insert(It.BinOp);
+ }
+
+ auto ExtendIsOnlyUsedByPartialReductions =
+ [&PartialReductionBinOps](Instruction *Extend) {
+ return all_of(Extend->users(), [&](const User *U) {
+ return PartialReductionBinOps.contains(U);
+ });
+ };
+
+ // Check if each use of a chain's two extends is a partial reduction
+ // and only add those those that don't have non-partial reduction users
+ for (auto It : PartialReductionChains) {
+ PartialReductionChain Chain = It;
----------------
SamTebbs33 wrote:
Oh yeah you're right.
https://github.com/llvm/llvm-project/pull/92418
More information about the llvm-commits
mailing list