[llvm] [LV] Avoid collapsing VF to zero when a loop has no memory ops or reductions (PR #216266)
Pengcheng Wang via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 14 00:33:28 PDT 2026
https://github.com/wangpc-pp created https://github.com/llvm/llvm-project/pull/216266
getSmallestAndWidestTypes() initializes MinWidth to the -1U sentinel and only
updates it from loads, stores, and reduction recurrences. A loop with none of
these (for example an early-exit search loop) therefore reports SmallestType as
the sentinel value.
The max-bandwidth VF computation in getMaximizedVFForTarget() divides the widest
register width by SmallestType. With the sentinel this underflows to zero, so
MaxVF collapses to an empty ElementCount and vectorization is disabled. As a
result -vectorizer-maximize-bandwidth, which is meant to allow a larger VF, can
instead turn a loop that vectorizes by default into a scalar loop.
Restore the SmallestType <= WidestType invariant by falling back to MaxWidth
when no element type was observed. This keeps the default path unchanged and
makes the max-bandwidth path pick the same VF for such loops.
Assisted-by: TRAE CLI (Opus 4.8)
>From a28ff8adddf41e87be7613d048dfb30c3e759e1a Mon Sep 17 00:00:00 2001
From: Pengcheng Wang <wangpengcheng.pp at bytedance.com>
Date: Fri, 14 Aug 2026 15:20:09 +0800
Subject: [PATCH 1/2] [LV] Add test for VF selection in loops with no memory
ops or reductions
Add a RISC-V early-exit search loop that has no loads, stores, or reductions.
getSmallestAndWidestTypes() cannot observe an element type for such a loop and
reports the smallest type as the -1U sentinel. This captures the current
behavior where -vectorizer-maximize-bandwidth leaves the loop scalar even
though it is vectorized by default.
Assisted-by: TRAE CLI (Opus 4.8)
---
.../RISCV/smallest-and-widest-types.ll | 54 +++++++++++++++++++
1 file changed, 54 insertions(+)
create mode 100644 llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
diff --git a/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
new file mode 100644
index 0000000000000..22991ad77fc1a
--- /dev/null
+++ b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
@@ -0,0 +1,54 @@
+; REQUIRES: asserts
+; RUN: opt -passes=loop-vectorize -mtriple riscv64 -mattr=+v \
+; RUN: -debug-only=loop-vectorize -disable-output -S < %s 2>&1 | \
+; RUN: FileCheck %s --check-prefix=DEFAULT
+; RUN: opt -passes=loop-vectorize -mtriple riscv64 -mattr=+v \
+; RUN: -vectorizer-maximize-bandwidth \
+; RUN: -debug-only=loop-vectorize -disable-output -S < %s 2>&1 | \
+; RUN: FileCheck %s --check-prefix=MAXBW
+
+; This is an early-exit search loop with no loads/stores and no reductions, so
+; getSmallestAndWidestTypes() cannot observe any element type and reports the
+; smallest type as the -1U sentinel (4294967295).
+;
+; FIXME: The sentinel makes the max-bandwidth VF computation divide the register
+; width by it, collapse the VF to zero, and disable vectorization. So with
+; -vectorizer-maximize-bandwidth this loop is left scalar even though it is
+; vectorized by default.
+
+; DEFAULT-LABEL: Checking a loop in 'find_first_ge'
+; DEFAULT: LV: The Smallest and Widest types: 4294967295 / 8 bits.
+; DEFAULT: LV: Selecting VF: vscale x 16.
+
+; MAXBW-LABEL: Checking a loop in 'find_first_ge'
+; MAXBW: LV: The Smallest and Widest types: 4294967295 / 8 bits.
+; MAXBW: LV: Vectorization is possible but not beneficial.
+
+define i32 @find_first_ge(i32 %n, i32 %target) {
+entry:
+ %cmp.not = icmp eq i32 %n, 0
+ br i1 %cmp.not, label %cleanup, label %for.body
+
+for.body: ; preds = %entry, %for.inc
+ %i = phi i32 [ %inc, %for.inc ], [ 0, %entry ]
+ %shl = shl i32 %i, 1
+ %add = add i32 %shl, 512
+ %cmp1.not = icmp ult i32 %add, %target
+ br i1 %cmp1.not, label %for.inc, label %cleanup.loopexit
+
+for.inc: ; preds = %for.body
+ %inc = add nuw i32 %i, 1
+ %exitcond.not = icmp eq i32 %inc, %n
+ br i1 %exitcond.not, label %cleanup.loopexit, label %for.body
+
+cleanup.loopexit: ; preds = %for.body, %for.inc
+ %i.lcssa.ph = phi i32 [ %n, %for.inc ], [ %i, %for.body ]
+ %umin = tail call i32 @llvm.umin.i32(i32 %i.lcssa.ph, i32 %n)
+ br label %cleanup
+
+cleanup: ; preds = %cleanup.loopexit, %entry
+ %i.lcssa = phi i32 [ 0, %entry ], [ %umin, %cleanup.loopexit ]
+ ret i32 %i.lcssa
+}
+
+declare i32 @llvm.umin.i32(i32, i32)
>From 81f69cf5f098996472b89b3d67f0bfbfae4a847c Mon Sep 17 00:00:00 2001
From: Pengcheng Wang <wangpengcheng.pp at bytedance.com>
Date: Fri, 14 Aug 2026 15:22:14 +0800
Subject: [PATCH 2/2] [LV] Avoid collapsing VF to zero when a loop has no
memory ops or reductions
getSmallestAndWidestTypes() initializes MinWidth to the -1U sentinel and only
updates it from loads, stores, and reduction recurrences. A loop with none of
these (for example an early-exit search loop) therefore reports SmallestType as
the sentinel value.
The max-bandwidth VF computation in getMaximizedVFForTarget() divides the widest
register width by SmallestType. With the sentinel this underflows to zero, so
MaxVF collapses to an empty ElementCount and vectorization is disabled. As a
result -vectorizer-maximize-bandwidth, which is meant to allow a larger VF, can
instead turn a loop that vectorizes by default into a scalar loop.
Restore the SmallestType <= WidestType invariant by falling back to MaxWidth
when no element type was observed. This keeps the default path unchanged and
makes the max-bandwidth path pick the same VF for such loops.
Assisted-by: TRAE CLI (Opus 4.8)
---
.../Vectorize/LoopVectorizationPlanner.cpp | 9 +++++++++
.../RISCV/smallest-and-widest-types.ll | 18 ++++++++----------
2 files changed, 17 insertions(+), 10 deletions(-)
diff --git a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
index fd7fd2e011a83..d9ce5b5e7ebf4 100644
--- a/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
+++ b/llvm/lib/Transforms/Vectorize/LoopVectorizationPlanner.cpp
@@ -527,6 +527,15 @@ VFSelectionContext::getSmallestAndWidestTypes() const {
MaxWidth, DL.getTypeSizeInBits(T->getScalarType()).getFixedValue());
}
}
+
+ // If the loop has no loads/stores or reductions (e.g. a search loop with an
+ // early exit), MinWidth is never updated and is left at its sentinel value.
+ // Fall back to MaxWidth to keep the SmallestType <= WidestType invariant, so
+ // callers such as the max-bandwidth VF computation don't divide by the
+ // sentinel and collapse the VF to zero.
+ if (MinWidth == -1U)
+ MinWidth = MaxWidth;
+
return {MinWidth, MaxWidth};
}
diff --git a/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
index 22991ad77fc1a..d5a4e06508258 100644
--- a/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
+++ b/llvm/test/Transforms/LoopVectorize/RISCV/smallest-and-widest-types.ll
@@ -8,21 +8,19 @@
; RUN: FileCheck %s --check-prefix=MAXBW
; This is an early-exit search loop with no loads/stores and no reductions, so
-; getSmallestAndWidestTypes() cannot observe any element type and reports the
-; smallest type as the -1U sentinel (4294967295).
-;
-; FIXME: The sentinel makes the max-bandwidth VF computation divide the register
-; width by it, collapse the VF to zero, and disable vectorization. So with
-; -vectorizer-maximize-bandwidth this loop is left scalar even though it is
-; vectorized by default.
+; getSmallestAndWidestTypes() cannot observe any element type. The smallest type
+; is reported as equal to the widest type instead of the -1U sentinel, so the
+; max-bandwidth VF computation does not divide the register width by the
+; sentinel and collapse the VF to zero. With and without
+; -vectorizer-maximize-bandwidth we pick the same VF.
; DEFAULT-LABEL: Checking a loop in 'find_first_ge'
-; DEFAULT: LV: The Smallest and Widest types: 4294967295 / 8 bits.
+; DEFAULT: LV: The Smallest and Widest types: 8 / 8 bits.
; DEFAULT: LV: Selecting VF: vscale x 16.
; MAXBW-LABEL: Checking a loop in 'find_first_ge'
-; MAXBW: LV: The Smallest and Widest types: 4294967295 / 8 bits.
-; MAXBW: LV: Vectorization is possible but not beneficial.
+; MAXBW: LV: The Smallest and Widest types: 8 / 8 bits.
+; MAXBW: LV: Selecting VF: vscale x 16.
define i32 @find_first_ge(i32 %n, i32 %target) {
entry:
More information about the llvm-commits
mailing list