[llvm] [LV]Enable max safe distance in predicated DataWithEVL vectorization mode. (PR #100755)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Tue Nov 26 13:40:01 PST 2024
================
@@ -3890,11 +3914,28 @@ FixedScalableVFPair LoopVectorizationCostModel::computeFeasibleMaxVF(
// It is computed by MaxVF * sizeOf(type) * 8, where type is taken from
// the memory accesses that is most restrictive (involved in the smallest
// dependence distance).
- unsigned MaxSafeElements =
- llvm::bit_floor(Legal->getMaxSafeVectorWidthInBits() / WidestType);
-
- auto MaxSafeFixedVF = ElementCount::getFixed(MaxSafeElements);
- auto MaxSafeScalableVF = getMaxLegalScalableVF(MaxSafeElements);
+ unsigned MaxSafeElements = Legal->getMaxSafeVectorWidthInBits() / WidestType;
+ if (Legal->isSafeForAnyVectorWidth())
+ MaxSafeElements = bit_ceil(MaxSafeElements);
+ unsigned MaxSafeElementsPowerOf2 = bit_floor(std::gcd(
+ MaxSafeElements, Legal->getMaxStoreLoadForwardSafeVFPowerOf2().value_or(
+ 1ULL << countr_zero(MaxSafeElements))));
+
+ auto MaxSafeFixedVF = ElementCount::getFixed(MaxSafeElementsPowerOf2);
+ auto MaxSafeScalableVF = getMaxLegalScalableVF(MaxSafeElementsPowerOf2);
+ if (FoldTailByMasking && AllowNonPowerOf2SafeDist) {
+ if (std::optional<unsigned> SafeStoreLoadForwardingDist =
+ Legal->getMaxStoreLoadForwardSafeVFNonPowerOf2()) {
+ unsigned SafeDist = *SafeStoreLoadForwardingDist;
+ if (MaxSafeElements >= SafeDist)
+ MaxSafeElements = SafeDist;
+ else
+ MaxSafeElements = std::max(std::gcd(MaxSafeElements, SafeDist),
+ MaxSafeElementsPowerOf2);
+ }
+ MaxSafeScalableVF =
+ getMaxLegalScalableVF(std::numeric_limits<unsigned>::max());
----------------
fhahn wrote:
Why ignore MaxSafeElements here?
This will somehow set the MaxSaclableVF to non-power-of-2?
https://github.com/llvm/llvm-project/pull/100755
More information about the llvm-commits
mailing list