[llvm] [LV, VP]VP intrinsics support for the Loop Vectorizer + adding new tail-folding mode using EVL. (PR #76172)

Florian Hahn via llvm-commits llvm-commits at lists.llvm.org
Fri Feb 2 12:48:27 PST 2024


================
@@ -4733,6 +4755,36 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
   // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
   if (Legal->prepareToFoldTailByMasking()) {
     CanFoldTailByMasking = true;
+    if (getTailFoldingStyle() == TailFoldingStyle::None)
+      return MaxFactors;
+
+    if (UserIC > 1) {
+      LLVM_DEBUG(dbgs() << "LV: Preference for VP intrinsics indicated. Will "
+                           "not generate VP intrinsics since interleave count "
+                           "specified is greater than 1.\n");
+      return MaxFactors;
+    }
+
+    // FIXME: use actual opcode/data type for analysis here.
+    PreferEVL = MaxFactors.ScalableVF.isScalable() &&
+                getTailFoldingStyle() == TailFoldingStyle::DataWithEVL &&
+                TTI.hasActiveVectorLength(0, nullptr, Align());
+    if (!PreferEVL) {
+      LLVM_DEBUG(dbgs() << "LV: Preference for VP intrinsics indicated. Will "
----------------
fhahn wrote:

IIUC if `!PreferEVL` we fall back to `TailFoldingStyle::DataWithoutLaneMask` making the value returned by `getTailFoldingStyle()` incorrect, right? This may be confusing and we also ignore the selected tail folding style due to any restrictions in `useVPIWithVPEVLVectorization`.  

AFAICT this isn't an issue with any of the other tail folding styles, as they can always be used if tail folding is valid. For EVL, I assume that even if the restrictions due to current implementation limitations are dropped, we still may not be able to use EVL, because there may be operations in the loop which don't support active-vector-length?

If that's the case, it would probably be cleaner to store the tail-folding style instead of `PreferEVL` and check the selected style early and adjust it if needed. After this point, it should be enough to check the set tail-folding style.

https://github.com/llvm/llvm-project/pull/76172


More information about the llvm-commits mailing list