[llvm] [LV]Initial support for safe distance in predicated DataWithEVL vectorization mode. (PR #102897)
Alexey Bataev via llvm-commits
llvm-commits at lists.llvm.org
Mon Sep 30 12:55:49 PDT 2024
================
@@ -4071,15 +4093,25 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
InterleaveInfo.invalidateGroupsRequiringScalarEpilogue();
}
- FixedScalableVFPair MaxFactors = computeFeasibleMaxVF(MaxTC, UserVF, true);
+ // If we don't know the precise trip count, or if the trip count that we
+ // found modulo the vectorization factor is not zero, try to fold the tail
+ // by masking.
+ // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
+ setTailFoldingStyles(UserIC);
+ FixedScalableVFPair MaxFactors =
+ computeFeasibleMaxVF(MaxTC, UserVF, foldTailByMasking());
// Avoid tail folding if the trip count is known to be a multiple of any VF
// we choose.
std::optional<unsigned> MaxPowerOf2RuntimeVF =
MaxFactors.FixedVF.getFixedValue();
if (MaxFactors.ScalableVF) {
std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
- if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo()) {
+ // If scalable vector factor is not set or power-of-2 or tail folding with
+ // EVL is not set, try to avoid tail folding if the trip count is known to
+ // be a multiple of any chosen VF.
----------------
alexey-bataev wrote:
If MaxEVLSafeElements is not power-of-2, cannot set MaxPowerOf2RuntimeVF and need to use regular vectorization with min(AVL, MaxSafeElements). With power of 2 can avoid this by disabling tail folding completely.
https://github.com/llvm/llvm-project/pull/102897
More information about the llvm-commits
mailing list