[llvm] [LV] Support tail-folded epilogue loops (PR #208764)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Tue Sep 29 05:53:32 PDT 2026
================
@@ -5401,22 +5414,30 @@ void LoopVectorizationPlanner::plan(ElementCount UserVF, unsigned UserIC) {
// Collect the instructions (and their associated costs) that will be more
// profitable to scalarize.
CM->collectNonVectorizedAndSetWideningDecisions(UserVF);
- buildVPlans(*VPlan1, UserVF, UserVF);
- ElementCount EpilogueUserVF = EpilogueVectorizationForceVF;
- if (EpilogueUserVF.isVector() &&
- ElementCount::isKnownLT(EpilogueUserVF, UserVF)) {
- CM->collectNonVectorizedAndSetWideningDecisions(EpilogueUserVF);
- buildVPlans(*VPlan1, EpilogueUserVF, EpilogueUserVF);
- }
- if (!VPlans.empty() && VPlans.front()->getSingleVF() == UserVF) {
- // For scalar VF, skip VPlan cost check as VPlan cost is designed for
- // vector VFs only.
- if (UserVF.isScalar() ||
- cost(*VPlans.front(), UserVF, /*RU=*/nullptr).isValid()) {
- LLVM_DEBUG(dbgs() << "LV: Using user VF " << UserVF << ".\n");
- LLVM_DEBUG(printPlans(dbgs()));
- return;
+ // Build the main-loop VPlan firstly because if epilogue tail-folding is
+ // enabled, it will be built later, so we keep the epilogue vplans at the
+ // end.
----------------
fhahn wrote:
nit: I think this can be removed.
```suggestion
```
https://github.com/llvm/llvm-project/pull/208764
More information about the llvm-commits
mailing list