[llvm] [LV]Initial support for safe distance in predicated DataWithEVL vectorization mode. (PR #102897)
Florian Hahn via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 27 01:37:38 PDT 2024
================
@@ -4070,15 +4092,22 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
InterleaveInfo.invalidateGroupsRequiringScalarEpilogue();
}
- FixedScalableVFPair MaxFactors = computeFeasibleMaxVF(MaxTC, UserVF, true);
+ // If we don't know the precise trip count, or if the trip count that we
+ // found modulo the vectorization factor is not zero, try to fold the tail
+ // by masking.
+ // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
+ setTailFoldingStyles(UserIC);
+ FixedScalableVFPair MaxFactors =
+ computeFeasibleMaxVF(MaxTC, UserVF, foldTailByMasking());
// Avoid tail folding if the trip count is known to be a multiple of any VF
// we choose.
std::optional<unsigned> MaxPowerOf2RuntimeVF =
MaxFactors.FixedVF.getFixedValue();
if (MaxFactors.ScalableVF) {
std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
- if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo()) {
+ if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo() &&
+ (!foldTailWithEVL() || isPowerOf2_32(MaxEVLSafeElements.value_or(0)))) {
----------------
fhahn wrote:
Might be good to document the logic here?
https://github.com/llvm/llvm-project/pull/102897
More information about the llvm-commits
mailing list