[llvm] [LV] Add tests for excluded cases in epiloge tail-folding (PR #217364)

David Sherwood via llvm-commits llvm-commits at lists.llvm.org
Mon Aug 24 05:57:25 PDT 2026


================
@@ -7223,23 +7225,64 @@ getEpilogueTailLowering(const LoopVectorizationCostModel &MainCM, const Loop *L,
   if (!EnableEpilogueVectorization) {
     reportVectorizationInfo(
         "Options conflict, epilogue vectorization is disallowed while "
-        "epilogue tail-folding allowed!\n",
+        "epilogue tail-folding allowed!",
         "UnsupportedEpilogueTailFoldingPolicy", ORE, L);
     return CM_EpilogueAllowed;
   }
 
+  if (!Hints.getWidth() || !hasForcedEpilogueVF()) {
+    reportVectorizationInfo("For now, Epilogue tail-folding can't be "
+                            "applied without forced main/epilogue loop VF",
+                            "UnsupportedEpilogueTailFoldingPolicy", ORE, L);
+    return CM_EpilogueAllowed;
+  }
+
+  if (!L->isInnermost()) {
+    reportVectorizationInfo(
+        "Epilogue tail-folding is not supported for outer loop",
+        "InvalidTailFoldedEpilogue", ORE, L);
+    return CM_EpilogueAllowed;
+  }
+
   // If scalar epilogue is explicitly required, we can't apply TF.
   if (MainCM.requiresScalarEpilogue(/*IsVectorizing*/ true)) {
-    LLVM_DEBUG(dbgs() << "LV: Epilogue tail-folding can't be applied because "
-                         "scalar epilogue is required\n"
-                         "LV: Fall back to a normal epilogue\n");
+    reportVectorizationInfo(
+        "Epilogue tail-folding can't be applied because scalar epilogue is "
+        "required. Fall back to a normal epilogue",
+        "InvalidTailFoldedEpilogue", ORE, L);
     return CM_EpilogueAllowed;
   }
 
   // If having epilogue is NOT allowed, then no epilogue to apply TF for.
   if (!MainCM.isEpilogueAllowed()) {
-    LLVM_DEBUG(dbgs() << "LV: No epilogue to apply tail-folding for.\n"
-                         "LV: Fall back to a normal epilogue\n");
+    reportVectorizationInfo("No epilogue to apply tail-folding for. Fall back "
+                            "to a normal epilogue",
+                            "InvalidTailFoldedEpilogue", ORE, L);
+    return CM_EpilogueAllowed;
+  }
+
+  // For now epilogue TF is not supported for some kinds of reductions that
+  // results in more overhead
+  bool HasReductions = !LVL.getReductionVars().empty();
+  bool HasSelectCmpReductions =
+      HasReductions &&
+      any_of(LVL.getReductionVars(), [](auto &Reduction) -> bool {
+        const RecurrenceDescriptor &RdxDesc = Reduction.second;
+        RecurKind RK = RdxDesc.getRecurrenceKind();
+        return RecurrenceDescriptor::isAnyOfRecurrenceKind(RK) ||
+               RecurrenceDescriptor::isFindIVRecurrenceKind(RK);
+      });
+  if (HasSelectCmpReductions) {
+    reportVectorizationInfo(
+        "Epilogue tail-folding is not supported yet for select-cmp Reductions",
+        "InvalidTailFoldedEpilogue", ORE, L);
+    return CM_EpilogueAllowed;
+  }
+
+  if (L->getExitingBlock() != L->getLoopLatch()) {
----------------
david-arm wrote:

Unfortunately, this isn't a strong enough test due to https://github.com/llvm/llvm-project/pull/205109, which may land soon. You probably need something like:

```
  if (L->getExitingBlock() != L->getLoopLatch() ||
      Legal->hasUncountableEarlyExit())
```

Can you also add a test similar to the one in llvm/test/Transforms/LoopVectorize/early_exit_combined_exits_epilogue.ll added by that PR?


https://github.com/llvm/llvm-project/pull/217364


More information about the llvm-commits mailing list