[llvm] [LV] Change loops' interleave count computation (PR #73766)

Florian Hahn via llvm-commits llvm-commits at lists.llvm.org
Thu Nov 30 06:23:46 PST 2023


================
@@ -5737,10 +5743,19 @@ LoopVectorizationCostModel::selectInterleaveCount(ElementCount VF,
   // the InterleaveCount as if vscale is '1', although if some information about
   // the vector is known (e.g. min vector size), we can make a better decision.
   if (BestKnownTC) {
-    MaxInterleaveCount =
-        std::min(*BestKnownTC / VF.getKnownMinValue(), MaxInterleaveCount);
-    // Make sure MaxInterleaveCount is greater than 0.
-    MaxInterleaveCount = std::max(1u, MaxInterleaveCount);
+    unsigned EstimatedVF = VF.getKnownMinValue();
+    if (VF.isScalable()) {
+      if (std::optional<unsigned> VScale = getVScaleForTuning(L, TTI))
+        EstimatedVF *= *VScale;
+    }
+    if (InterleaveSmallLoopScalarReduction || (*BestKnownTC % EstimatedVF == 0))
+      MaxInterleaveCount =
+          std::min(*BestKnownTC / EstimatedVF, MaxInterleaveCount);
+    else
+      MaxInterleaveCount =
+          std::min(*BestKnownTC / (EstimatedVF * 2), MaxInterleaveCount);
+    // Make sure MaxInterleaveCount is greater than 0 & a power of 2.
+    MaxInterleaveCount = llvm::bit_floor(std::max(1u, MaxInterleaveCount));
----------------
fhahn wrote:

nit: drop `llvm::`

https://github.com/llvm/llvm-project/pull/73766


More information about the llvm-commits mailing list