[llvm] [LV, VP]VP intrinsics support for the Loop Vectorizer + adding new tail-folding mode using EVL. (PR #76172)

Alexey Bataev via llvm-commits llvm-commits at lists.llvm.org
Mon Feb 5 07:46:56 PST 2024


================
@@ -4733,6 +4755,36 @@ LoopVectorizationCostModel::computeMaxVF(ElementCount UserVF, unsigned UserIC) {
   // FIXME: look for a smaller MaxVF that does divide TC rather than masking.
   if (Legal->prepareToFoldTailByMasking()) {
     CanFoldTailByMasking = true;
+    if (getTailFoldingStyle() == TailFoldingStyle::None)
+      return MaxFactors;
+
+    if (UserIC > 1) {
+      LLVM_DEBUG(dbgs() << "LV: Preference for VP intrinsics indicated. Will "
+                           "not generate VP intrinsics since interleave count "
+                           "specified is greater than 1.\n");
+      return MaxFactors;
+    }
+
+    // FIXME: use actual opcode/data type for analysis here.
+    PreferEVL = MaxFactors.ScalableVF.isScalable() &&
+                getTailFoldingStyle() == TailFoldingStyle::DataWithEVL &&
+                TTI.hasActiveVectorLength(0, nullptr, Align());
+    if (!PreferEVL) {
+      LLVM_DEBUG(dbgs() << "LV: Preference for VP intrinsics indicated. Will "
+                           "not try to generate VP Intrinsics since the target "
+                           "does not support vector length predication.\n");
+      return MaxFactors;
+    }
+
+    LLVM_DEBUG(dbgs() << "LV: Preference for VP intrinsics indicated. Will "
+                         "try to generate VP Intrinsics.\n");
+
+    // Tail folded loop using VP intrinsics restricts the VF to be scalable
+    // for now.
+    // TODO: extend it for fixed vectors, if required.
+    assert(MaxFactors.ScalableVF.isScalable() &&
+           "Expected scalable vector factor.");
+    MaxFactors.FixedVF = ElementCount::getFixed(1);
----------------
alexey-bataev wrote:

Added

https://github.com/llvm/llvm-project/pull/76172


More information about the llvm-commits mailing list