[llvm] [LV] Add -predicated-epilogue option for tail-folded epilogue (PR #190697)

David Sherwood via llvm-commits llvm-commits at lists.llvm.org
Fri Apr 10 03:46:13 PDT 2026


================
@@ -8574,6 +8592,58 @@ static ScalarEpilogueLowering getScalarEpilogueLowering(
   return CM_ScalarEpilogueAllowed;
 }
 
+/// Check if we can apply tail-folding on the scalar loop of the vectorized
+/// epilogue loop, so that we can have vectorized unpredicated main loop along
+/// with vectorized predicated epilogue loop.
+static bool isEpilogueTailFoldingAllowed(const LoopVectorizationCostModel &CM,
+                                         const LoopVectorizationLegality &LVL) {
+  // Epilogue TF is only enabled when explicitly requested via command line.
+  if (!PreferPredicateOverEpilogue.getNumOccurrences() ||
+      PreferPredicateOverEpilogue != PreferPredicateTy::PredicatedEpilogue)
+    return false;
+
+  if (!EnableEpilogueVectorization) {
+    LLVM_DEBUG(dbgs() << "LV: Options conflict, epilogue vectorization is "
+                         "disallowed while epilogue predication allowed!\n");
+    LLVM_DEBUG(dbgs() << "LV: Disallow epilogue predication\n");
+    return false;
+  }
+
+  // If scalar epilogue is explicitly required, we can't apply TF.
+  if (CM.requiresScalarEpilogue(/*IsVectorizing*/ true)) {
+    LLVM_DEBUG(dbgs() << "LV: Epilogue tail folding can't be applied because "
+                         "scalar epilogue is required\n");
+    return false;
+  }
+
+  // If having epilogue is NOT allowed, then no epilogue to apply TF for.
+  if (!CM.isScalarEpilogueAllowed()) {
----------------
david-arm wrote:

OK well I think this code needs deleting in that case because isScalarEpilogueAllowed always returns true and we'll never return false here.

https://github.com/llvm/llvm-project/pull/190697


More information about the llvm-commits mailing list