[llvm] [LoopVectorize] Improve Vectorization of Low Trip Count Loops (PR #195823)
David Sherwood via llvm-commits
llvm-commits at lists.llvm.org
Mon Jun 22 06:15:05 PDT 2026
================
@@ -6134,6 +6198,40 @@ LoopVectorizationPlanner::computeBestVF() {
}
VPlan *PlanForBestVF = &FirstPlan;
+ ElementCount ExactTC = getSmallConstantTripCount(PSE.getSE(), OrigLoop);
+ auto IsUnprofitableOneScalarTail =
+ [&](const VectorizationFactor &CurrentFactor, bool HasTail) {
+ if (ForceVectorization || !HasTail || !ExactTC.isFixed() ||
+ CurrentFactor.Width.isScalable())
+ return false;
+
+ unsigned TC = ExactTC.getFixedValue();
+ if (TC == 0 || TC > TTI.getMinTripCountTailFoldingThreshold())
+ return false;
+
+ unsigned EstimatedWidth = estimateElementCount(
+ CurrentFactor.Width, Config.getVScaleForTuning());
+ if (TC != EstimatedWidth + 1)
+ return false;
+
+ InstructionCost VectorCost =
+ getCostForKnownTripCount(CurrentFactor, TC, HasTail);
+ InstructionCost ScalarCostForTC =
+ getCostForKnownTripCount(ScalarFactor, TC, /*HasTail=*/false);
+ // Be conservative for the one-scalar-tail shape. It introduces extra
+ // control flow and a scalar epilogue for a single element, so require
+ // the vectorized form to save at least one scalar iteration.
+ InstructionCost AdjustedVectorCost = VectorCost + ScalarCost;
+ if (!VectorCost.isValid() || !ScalarCostForTC.isValid() ||
----------------
david-arm wrote:
I think you can just do `!AdjustedVectorCost.isValid()` because the Invalid state should be inherited from LHS and propagated from the RHS of the `+`. Similarly for the code above on line 6147.
https://github.com/llvm/llvm-project/pull/195823
More information about the llvm-commits
mailing list