[llvm] [LV] Change loops' interleave count computation (PR #73766)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Thu Nov 30 06:23:46 PST 2023
================
@@ -5737,10 +5743,19 @@ LoopVectorizationCostModel::selectInterleaveCount(ElementCount VF,
// the InterleaveCount as if vscale is '1', although if some information about
// the vector is known (e.g. min vector size), we can make a better decision.
if (BestKnownTC) {
- MaxInterleaveCount =
- std::min(*BestKnownTC / VF.getKnownMinValue(), MaxInterleaveCount);
- // Make sure MaxInterleaveCount is greater than 0.
- MaxInterleaveCount = std::max(1u, MaxInterleaveCount);
+ unsigned EstimatedVF = VF.getKnownMinValue();
+ if (VF.isScalable()) {
+ if (std::optional<unsigned> VScale = getVScaleForTuning(L, TTI))
+ EstimatedVF *= *VScale;
+ }
+ if (InterleaveSmallLoopScalarReduction || (*BestKnownTC % EstimatedVF == 0))
+ MaxInterleaveCount =
+ std::min(*BestKnownTC / EstimatedVF, MaxInterleaveCount);
+ else
+ MaxInterleaveCount =
+ std::min(*BestKnownTC / (EstimatedVF * 2), MaxInterleaveCount);
+ // Make sure MaxInterleaveCount is greater than 0 & a power of 2.
+ MaxInterleaveCount = llvm::bit_floor(std::max(1u, MaxInterleaveCount));
----------------
fhahn wrote:
nit: drop `llvm::`
https://github.com/llvm/llvm-project/pull/73766
More information about the llvm-commits
mailing list