[llvm] [LV]Initial support for safe distance in predicated DataWithEVL vectorization mode. (PR #102897)

Alexey Bataev via llvm-commits llvm-commits at lists.llvm.org
Mon Sep 30 12:55:49 PDT 2024


================
@@ -4071,15 +4093,25 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
     InterleaveInfo.invalidateGroupsRequiringScalarEpilogue();
   }
 
-  FixedScalableVFPair MaxFactors = computeFeasibleMaxVF(MaxTC, UserVF, true);
+  // If we don't know the precise trip count, or if the trip count that we
+  // found modulo the vectorization factor is not zero, try to fold the tail
+  // by masking.
+  // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
+  setTailFoldingStyles(UserIC);
+  FixedScalableVFPair MaxFactors =
+      computeFeasibleMaxVF(MaxTC, UserVF, foldTailByMasking());
 
   // Avoid tail folding if the trip count is known to be a multiple of any VF
   // we choose.
   std::optional<unsigned> MaxPowerOf2RuntimeVF =
       MaxFactors.FixedVF.getFixedValue();
   if (MaxFactors.ScalableVF) {
     std::optional<unsigned> MaxVScale = getMaxVScale(*TheFunction, TTI);
-    if (MaxVScale && TTI.isVScaleKnownToBeAPowerOfTwo()) {
+    // If scalable vector factor is not set or power-of-2 or tail folding with
+    // EVL is not set, try to avoid tail folding if the trip count is known to
+    // be a multiple of any chosen VF.
----------------
alexey-bataev wrote:

If MaxEVLSafeElements is not power-of-2, cannot set MaxPowerOf2RuntimeVF and need to use regular vectorization with min(AVL, MaxSafeElements). With power of 2 can avoid this by disabling tail folding completely.

https://github.com/llvm/llvm-project/pull/102897


More information about the llvm-commits mailing list