[llvm] [LV]Initial support for safe distance in predicated DataWithEVL vectorization mode. (PR #102897)

Florian Hahn via llvm-commits llvm-commits at lists.llvm.org
Tue Aug 27 01:37:38 PDT 2024


================
@@ -4070,15 +4092,22 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
     InterleaveInfo.invalidateGroupsRequiringScalarEpilogue();
   }
 
-  FixedScalableVFPair MaxFactors = computeFeasibleMaxVF(MaxTC, UserVF, true);
+  // If we don't know the precise trip count, or if the trip count that we
+  // found modulo the vectorization factor is not zero, try to fold the tail
+  // by masking.
+  // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
+  setTailFoldingStyles(UserIC);
+  FixedScalableVFPair MaxFactors =
+      computeFeasibleMaxVF(MaxTC, UserVF, foldTailByMasking());
 
   // Avoid tail folding if the trip count is known to be a multiple of any VF
   // we choose.
   std::optional<unsigned> MaxPowerOf2RuntimeVF =
       MaxFactors.FixedVF.getFixedValue();
   if (MaxFactors.ScalableVF) {
     std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
-    if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo()) {
+    if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo() &&
+        (!foldTailWithEVL() || isPowerOf2_32(MaxEVLSafeElements.value_or(0)))) {
----------------
fhahn wrote:

Might be good to document the logic here?

https://github.com/llvm/llvm-project/pull/102897


More information about the llvm-commits mailing list