[llvm] [LV] Add -predicated-epilogue option for tail-folded epilogue (PR #190697)
David Sherwood via llvm-commits
llvm-commits at lists.llvm.org
Fri Apr 10 03:46:13 PDT 2026
================
@@ -8574,6 +8592,58 @@ static ScalarEpilogueLowering getScalarEpilogueLowering(
return CM_ScalarEpilogueAllowed;
}
+/// Check if we can apply tail-folding on the scalar loop of the vectorized
+/// epilogue loop, so that we can have vectorized unpredicated main loop along
+/// with vectorized predicated epilogue loop.
+static bool isEpilogueTailFoldingAllowed(const LoopVectorizationCostModel &CM,
+ const LoopVectorizationLegality &LVL) {
+ // Epilogue TF is only enabled when explicitly requested via command line.
+ if (!PreferPredicateOverEpilogue.getNumOccurrences() ||
+ PreferPredicateOverEpilogue != PreferPredicateTy::PredicatedEpilogue)
+ return false;
+
+ if (!EnableEpilogueVectorization) {
+ LLVM_DEBUG(dbgs() << "LV: Options conflict, epilogue vectorization is "
+ "disallowed while epilogue predication allowed!\n");
+ LLVM_DEBUG(dbgs() << "LV: Disallow epilogue predication\n");
+ return false;
+ }
+
+ // If scalar epilogue is explicitly required, we can't apply TF.
+ if (CM.requiresScalarEpilogue(/*IsVectorizing*/ true)) {
+ LLVM_DEBUG(dbgs() << "LV: Epilogue tail folding can't be applied because "
+ "scalar epilogue is required\n");
+ return false;
+ }
+
+ // If having epilogue is NOT allowed, then no epilogue to apply TF for.
+ if (!CM.isScalarEpilogueAllowed()) {
----------------
david-arm wrote:
OK well I think this code needs deleting in that case because isScalarEpilogueAllowed always returns true and we'll never return false here.
https://github.com/llvm/llvm-project/pull/190697
More information about the llvm-commits
mailing list