[llvm] [LV] Avoid collapsing VF to zero when a loop has no memory ops or reductions (PR #216266)

[via llvm-commits]

llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-risc-v

@llvm/pr-subscribers-vectorizers

Author: Pengcheng Wang (wangpc-pp)

<details>
<summary>Changes</summary>

getSmallestAndWidestTypes() initializes MinWidth to the -1U sentinel and only
updates it from loads, stores, and reduction recurrences. A loop with none of
these (for example an early-exit search loop) therefore reports SmallestType as
the sentinel value.

The max-bandwidth VF computation in getMaximizedVFForTarget() divides the widest
register width by SmallestType. With the sentinel this underflows to zero, so
MaxVF collapses to an empty ElementCount and vectorization is disabled. As a
result -vectorizer-maximize-bandwidth, which is meant to allow a larger VF, can
instead turn a loop that vectorizes by default into a scalar loop.

Restore the SmallestType <= WidestType invariant by falling back to MaxWidth
when no element type was observed. This keeps the default path unchanged and
makes the max-bandwidth path pick the same VF for such loops.

Assisted-by: TRAE CLI (Opus 4.8)


---
Full diff: https://github.com/llvm/llvm-project/pull/216266.diff


2 Files Affected:

- (modified) llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp (+9) 
- (added) llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll (+52) 


``````````diff
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
index fd7fd2e011a83..d9ce5b5e7ebf4 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
@@ -527,6 +527,15 @@ VFSelectionContext::getSmallestAndWidestTypes() const {
           MaxWidth, DL.getTypeSizeInBits(T->getScalarType()).getFixedValue());
     }
   }
+
+  // If the loop has no loads/stores or reductions (e.g. a search loop with an
+  // early exit), MinWidth is never updated and is left at its sentinel value.
+  // Fall back to MaxWidth to keep the SmallestType <= WidestType invariant, so
+  // callers such as the max-bandwidth VF computation don't divide by the
+  // sentinel and collapse the VF to zero.
+  if (MinWidth == -1U)
+    MinWidth = MaxWidth;
+
   return {MinWidth, MaxWidth};
 }
 
diff --git a/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
new file mode 100644
index 0000000000000..d5a4e06508258
--- /dev/null
+++ b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
@@ -0,0 +1,52 @@
+; REQUIRES: asserts
+; RUN: opt -passes=loop-vectorize -mtriple riscv64 -mattr=+v \
+; RUN:   -debug-only=loop-vectorize -disable-output -S < %s 2>&1 | \
+; RUN:   FileCheck %s --check-prefix=DEFAULT
+; RUN: opt -passes=loop-vectorize -mtriple riscv64 -mattr=+v \
+; RUN:   -vectorizer-maximize-bandwidth \
+; RUN:   -debug-only=loop-vectorize -disable-output -S < %s 2>&1 | \
+; RUN:   FileCheck %s --check-prefix=MAXBW
+
+; This is an early-exit search loop with no loads/stores and no reductions, so
+; getSmallestAndWidestTypes() cannot observe any element type. The smallest type
+; is reported as equal to the widest type instead of the -1U sentinel, so the
+; max-bandwidth VF computation does not divide the register width by the
+; sentinel and collapse the VF to zero. With and without
+; -vectorizer-maximize-bandwidth we pick the same VF.
+
+; DEFAULT-LABEL: Checking a loop in 'find_first_ge'
+; DEFAULT: LV: The Smallest and Widest types: 8 / 8 bits.
+; DEFAULT: LV: Selecting VF: vscale x 16.
+
+; MAXBW-LABEL: Checking a loop in 'find_first_ge'
+; MAXBW: LV: The Smallest and Widest types: 8 / 8 bits.
+; MAXBW: LV: Selecting VF: vscale x 16.
+
+define i32 @find_first_ge(i32 %n, i32 %target) {
+entry:
+  %cmp.not = icmp eq i32 %n, 0
+  br i1 %cmp.not, label %cleanup, label %for.body
+
+for.body:                                         ; preds = %entry, %for.inc
+  %i = phi i32 [ %inc, %for.inc ], [ 0, %entry ]
+  %shl = shl i32 %i, 1
+  %add = add i32 %shl, 512
+  %cmp1.not = icmp ult i32 %add, %target
+  br i1 %cmp1.not, label %for.inc, label %cleanup.loopexit
+
+for.inc:                                          ; preds = %for.body
+  %inc = add nuw i32 %i, 1
+  %exitcond.not = icmp eq i32 %inc, %n
+  br i1 %exitcond.not, label %cleanup.loopexit, label %for.body
+
+cleanup.loopexit:                                 ; preds = %for.body, %for.inc
+  %i.lcssa.ph = phi i32 [ %n, %for.inc ], [ %i, %for.body ]
+  %umin = tail call i32 @llvm.umin.i32(i32 %i.lcssa.ph, i32 %n)
+  br label %cleanup
+
+cleanup:                                          ; preds = %cleanup.loopexit, %entry
+  %i.lcssa = phi i32 [ 0, %entry ], [ %umin, %cleanup.loopexit ]
+  ret i32 %i.lcssa
+}
+
+declare i32 @llvm.umin.i32(i32, i32)

``````````

</details>


https://github.com/llvm/llvm-project/pull/216266