[llvm] [LV] Don't require scalar epilogue for unsupported IAG with tail (PR #96544)

Florian Hahn via llvm-commits llvm-commits at lists.llvm.org
Sun Jul 14 12:56:43 PDT 2024


================
@@ -1446,24 +1446,42 @@ class LoopVectorizationCostModel {
 
   /// Returns true if we're required to use a scalar epilogue for at least
   /// the final iteration of the original loop.
-  bool requiresScalarEpilogue(bool IsVectorizing) const {
+  bool requiresScalarEpilogue(ElementCount VF) const {
     if (!isScalarEpilogueAllowed()) {
-      LLVM_DEBUG(dbgs() << "LV: Loop does not require scalar epilogue\n");
+      LLVM_DEBUG(dbgs() << "LV: Loop with VF = " << VF
+                        << " does not require scalar epilogue\n");
       return false;
     }
     // If we might exit from anywhere but the latch, must run the exiting
     // iteration in scalar form.
     if (TheLoop->getExitingBlock() != TheLoop->getLoopLatch()) {
-      LLVM_DEBUG(
-          dbgs() << "LV: Loop requires scalar epilogue: multiple exits\n");
+      LLVM_DEBUG(dbgs() << "LV: Loop with VF = " << VF
+                        << " requires scalar epilogue: multiple exits\n");
       return true;
     }
-    if (IsVectorizing && InterleaveInfo.requiresScalarEpilogue()) {
-      LLVM_DEBUG(dbgs() << "LV: Loop requires scalar epilogue: "
-                           "interleaved group requires scalar epilogue\n");
-      return true;
+    if (VF.isVector()) {
+      if (InterleaveInfo.requiresScalarEpilogue()) {
+        // Make sure interleaved groups that require scalar epilogue will be
+        // widened.
+        if (any_of(InterleaveInfo.getInterleaveGroups(), [&](auto *Group) {
+              if (!Group->requiresScalarEpilogue())
+                return false;
+
+              Instruction *I = Group->getMember(0);
+              InstWidening Decision = getWideningDecision(I, VF);
+              return Decision == CM_Interleave ||
+                     (Decision == CM_Unknown &&
----------------
fhahn wrote:

Is it possible that the decision later changes and the result of `requiresScalarEpilogue` changes? as mentioned for https://github.com/llvm/llvm-project/pull/96681 would it be possible to compute whether a scalar epilogue is needed once and set a flag to avoid it potentially changing later?

https://github.com/llvm/llvm-project/pull/96544


More information about the llvm-commits mailing list