[llvm] [LoopVectorize] Improve Vectorization of Low Trip Count Loops (PR #195823)

Jack Styles via llvm-commits llvm-commits at lists.llvm.org
Tue Jul 21 01:54:28 PDT 2026


================
@@ -5790,14 +5814,78 @@ LoopVectorizationPlanner::computeBestVF() {
     return {VectorizationFactor::Disabled(), nullptr};
   // If there is a single VPlan with a single VF, return it directly.
   VPlan &FirstPlan = *VPlans[0];
+  auto IsUnprofitableOneScalarTail =
+      [&](const VectorizationFactor &CurrentFactor, bool HasTail,
+          bool ForceVectorization, const ElementCount &ExactTC,
+          const VectorizationFactor &ScalarFactor,
+          const InstructionCost &ScalarCost, unsigned int UserIC) {
+        if (ForceVectorization || !HasTail || !ExactTC.isFixed() ||
+            CurrentFactor.Width.isScalable())
+          return false;
+
+        unsigned TC = ExactTC.getFixedValue();
+        if (TC == 0 || TC > TTI.getMinTripCountTailFoldingThreshold())
+          return false;
+
+        unsigned EstimatedWidth = estimateElementCount(
+            CurrentFactor.Width, Config.getVScaleForTuning());
+        if (TC != (EstimatedWidth * UserIC) + 1)
+          return false;
+
+        InstructionCost VectorCost =
+            getCostForKnownTripCount(CurrentFactor, TC, HasTail);
+        InstructionCost ScalarCostForTC =
+            getCostForKnownTripCount(ScalarFactor, TC, /*HasTail=*/false);
+        // Be conservative for the one-scalar-tail shape. It introduces extra
+        // control flow and a scalar epilogue for a single element, so require
+        // the vectorized form to save at least one scalar iteration.
+        InstructionCost AdjustedVectorCost = VectorCost + ScalarCost;
----------------
Stylie777 wrote:

Thats correct, because we want VectorCost to reflect the cost for the vector iteration(s) and the one remaining scalar cost (I have added a comment here to explain).

Ok, understood. Apologies I got how the costs are done mixed up.

I still think we should be making clear savings. There does seem to be an issue with the costing of `smin/smax/umin/umax` intrinsics when the type needs promoting, but even with that fixed (see, INSERT PR NUMBER FOR SMIN FIX) the costs for `tc3_smin_i8_reject` still just accept the vector cost over the scalar cost (29 vector < 30 scalar) so the function will get vectorised, which as we have already discussed we know is slower for performance as per the micro benchmark.


https://github.com/llvm/llvm-project/pull/195823


More information about the llvm-commits mailing list