[llvm] [VPlan] Run narrowInterleaveGroups during general VPlan optimizations. (PR #149706)
David Sherwood via llvm-commits
llvm-commits at lists.llvm.org
Thu Sep 11 08:27:38 PDT 2025
================
@@ -4011,24 +4031,44 @@ void VPlanTransforms::narrowInterleaveGroups(VPlan &Plan, ElementCount VF,
auto *WideMember0 = dyn_cast_or_null<VPWidenRecipe>(
InterleaveR->getStoredValues()[0]->getDefiningRecipe());
if (!WideMember0)
- return;
+ return nullptr;
for (const auto &[I, V] : enumerate(InterleaveR->getStoredValues())) {
auto *R = dyn_cast_or_null<VPWidenRecipe>(V->getDefiningRecipe());
if (!R || R->getOpcode() != WideMember0->getOpcode() ||
R->getNumOperands() > 2)
- return;
+ return nullptr;
if (any_of(enumerate(R->operands()),
[WideMember0, Idx = I](const auto &P) {
const auto &[OpIdx, OpV] = P;
return !canNarrowLoad(WideMember0, OpIdx, OpV, Idx);
}))
- return;
+ return nullptr;
}
StoreGroups.push_back(InterleaveR);
}
if (StoreGroups.empty())
- return;
+ return nullptr;
+
+ // All interleave groups in Plan can be narrowed for VFToOptimize. Split the
+ // original Plan into 2: a) a new clone which contains all VFs of Plan, except
+ // VFToOptimize, and b) the original Plan with VFToOptimize as single VF.
+ std::unique_ptr<VPlan> NewPlan;
+ if (size(Plan.vectorFactors()) != 1) {
+ NewPlan = std::unique_ptr<VPlan>(Plan.duplicate());
+ Plan.setVF(*VFToOptimize);
+ bool First = true;
+ for (ElementCount VF : NewPlan->vectorFactors()) {
+ if (VF == VFToOptimize)
+ continue;
+ if (First) {
+ NewPlan->setVF(VF);
+ First = false;
+ continue;
+ }
+ NewPlan->addVF(VF);
----------------
david-arm wrote:
This feels a bit cumbersome. It would be nice if `addVF` could be made to work without any existing VF, then you could rewrite the loop as:
```
for (ElementCount VF : NewPlan->vectorFactors())
if (VF != VFToOptimize)
NewPlan->addVF(VF);
```
https://github.com/llvm/llvm-project/pull/149706
More information about the llvm-commits
mailing list