[Mlir-commits] [mlir] [mlir][vector] Support fixed-size masks in `eliminateVectorMasks` (PR #221595)
Dhairyashil R G
llvmlistbot at llvm.org
Mon Sep 21 07:04:11 PDT 2026
https://github.com/dhairyashilRG updated https://github.com/llvm/llvm-project/pull/221595
>From 13240544aa0abfdb946ee55f3d545a35b8058969 Mon Sep 17 00:00:00 2001
From: Dhairyashil R G <dhairyashil25 at gmail.com>
Date: Sun, 6 Sep 2026 23:16:56 +0530
Subject: [PATCH 1/3] [mlir][vector] Support fixed-size masks in
eliminateVectorMasks
`eliminateVectorMasks` returned immediately unless a `VscaleRange` was
supplied, under a TODO arguing the fixed-size case was unlikely to be
useful because "for fixed-size code dimensions are all static so masks
tend to fold away".
That holds only when the mask operand is constant. When it is derived
from an induction variable, which is the shape tiling produces, the
operand is not constant and no fold removes the mask, even though
value-bounds analysis can prove it all-true. For a `vector<4xi1>` mask of
`1024 - %i` inside `scf.for %i = 0 to 1024 step 4`, canonicalization
leaves the `create_mask` alone.
The algorithm itself was never scalable-specific; only the bound query
was. Make `vscaleRange` genuinely optional: without one, compute an
ordinary constant lower bound with `ValueBoundsConstraintSet` and compare
it against the mask dimension. Scalable dimensions still require a range
and bail without one, so a fixed-size caller cannot prove a dimension
whose runtime size is `vscale` times the size in the type. The scalable
path is unchanged.
Note this changes what an omitted `vscaleRange` means. It previously made
the call a no-op; it now runs the fixed-size analysis. No in-tree caller
relies on the old meaning, but downstream ones may: IREE calls this
unconditionally in `GenericVectorization.cpp`, passing a range that is
`std::nullopt` for every target other than AArch64+SVE.
Assisted-by: Claude
---
.../Vector/Transforms/VectorTransforms.h | 9 +-
.../Transforms/VectorMaskElimination.cpp | 38 +++++---
.../Vector/eliminate-masks-fixed-size.mlir | 97 +++++++++++++++++++
.../Dialect/Vector/TestVectorTransforms.cpp | 10 +-
4 files changed, 135 insertions(+), 19 deletions(-)
create mode 100644 mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
diff --git a/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h b/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
index e815e026305fa..e88058fe1ff70 100644
--- a/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
+++ b/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
@@ -124,11 +124,10 @@ struct VscaleRange {
/// Attempts to eliminate redundant vector masks by replacing them with all-true
/// constants at the top of the function (which results in the masks folding
-/// away). Note: Currently, this only runs for vector.create_mask ops and
-/// requires `vscaleRange`. If `vscaleRange` is not provided this transform does
-/// nothing. This is because these redundant masks are much more likely for
-/// scalable code which requires memref/tensor dynamic sizes, whereas fixed-size
-/// code has static sizes, so simpler folds remove the masks.
+/// away). Note: Currently, this only runs for vector.create_mask ops.
+///
+/// `vscaleRange` is only needed to prove that a scalable mask dimension is
+/// all-true; without it, only fixed-size dimensions are considered.
void eliminateVectorMasks(IRRewriter &rewriter, FunctionOpInterface function,
std::optional<VscaleRange> vscaleRange = {});
diff --git a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
index 6f75ce7a04511..f8f0ac8e33183 100644
--- a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
+++ b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
@@ -15,11 +15,14 @@ using namespace mlir;
using namespace mlir::vector;
namespace {
-/// Attempts to resolve a (scalable) CreateMaskOp to an all-true constant mask.
-/// All-true masks can then be eliminated by simple folds.
-LogicalResult resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
- vector::CreateMaskOp createMaskOp,
- VscaleRange vscaleRange) {
+/// Attempts to resolve a CreateMaskOp to an all-true constant mask. All-true
+/// masks can then be eliminated by simple folds. `vscaleRange` is required to
+/// reason about scalable dimensions; without it only fixed-size dimensions can
+/// be proven all-true.
+LogicalResult
+resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
+ vector::CreateMaskOp createMaskOp,
+ std::optional<VscaleRange> vscaleRange) {
auto maskType = createMaskOp.getVectorType();
auto maskTypeDimScalableFlags = maskType.getScalableDims();
auto maskTypeDimSizes = maskType.getShape();
@@ -54,9 +57,25 @@ LogicalResult resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
for (auto [i, dimSize] : unknownDims) {
// Compute the lower bound for the unknown dimension (i.e. the smallest
// value it could be).
+ if (!vscaleRange) {
+ // A constant bound cannot prove a scalable dim, whose runtime size is
+ // `vscale` times the size in the type. Checked first to skip the query.
+ if (maskTypeDimScalableFlags[i])
+ return failure();
+ FailureOr<int64_t> dimLowerBound =
+ ValueBoundsConstraintSet::computeConstantBound(
+ presburger::BoundType::LB, dimSize);
+ if (failed(dimLowerBound))
+ return failure();
+ // If LB < the mask dim size then this dim is not all-true.
+ if (*dimLowerBound < maskTypeDimSizes[i])
+ return failure();
+ continue;
+ }
+
FailureOr<ConstantOrScalableBound> dimLowerBound =
vector::ScalableValueBoundsConstraintSet::computeScalableBound(
- dimSize, {}, vscaleRange.vscaleMin, vscaleRange.vscaleMax,
+ dimSize, {}, vscaleRange->vscaleMin, vscaleRange->vscaleMax,
presburger::BoundType::LB);
if (failed(dimLowerBound))
return failure();
@@ -94,11 +113,6 @@ namespace mlir::vector {
void eliminateVectorMasks(IRRewriter &rewriter, FunctionOpInterface function,
std::optional<VscaleRange> vscaleRange) {
- // TODO: Support fixed-size case. This is less likely to be useful as for
- // fixed-size code dimensions are all static so masks tend to fold away.
- if (!vscaleRange)
- return;
-
// Early exit for functions without a body.
if (function.isExternal())
return;
@@ -114,7 +128,7 @@ void eliminateVectorMasks(IRRewriter &rewriter, FunctionOpInterface function,
rewriter.setInsertionPointToStart(&function.front());
for (auto mask : worklist)
- (void)resolveAllTrueCreateMaskOp(rewriter, mask, *vscaleRange);
+ (void)resolveAllTrueCreateMaskOp(rewriter, mask, vscaleRange);
}
} // namespace mlir::vector
diff --git a/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir b/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
new file mode 100644
index 0000000000000..192285ccc512a
--- /dev/null
+++ b/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
@@ -0,0 +1,97 @@
+// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks=fixed-size | FileCheck %s
+
+// The mask bound comes from the induction variable, so it is not constant and
+// no fold removes it. Value-bounds analysis proves `%i <= 1020`, hence
+// `1024 - %i >= 4`, so the mask is all-true.
+
+// CHECK-LABEL: @eliminate_mask_bounded_by_induction_variable
+// CHECK: %[[ALL_TRUE:.*]] = vector.constant_mask [4] : vector<4xi1>
+// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE]]
+func.func @eliminate_mask_bounded_by_induction_variable(%t: tensor<1024xf32>) -> f32 {
+ %c0 = arith.constant 0 : index
+ %c4 = arith.constant 4 : index
+ %c1024 = arith.constant 1024 : index
+ %f0 = arith.constant 0.0 : f32
+ %r = scf.for %i = %c0 to %c1024 step %c4 iter_args(%acc = %f0) -> f32 {
+ %rem = arith.subi %c1024, %i : index
+ %mask = vector.create_mask %rem : vector<4xi1>
+ %v = vector.transfer_read %t[%i], %f0, %mask : tensor<1024xf32>, vector<4xf32>
+ %s = vector.reduction <add>, %v : vector<4xf32> into f32
+ %n = arith.addf %acc, %s : f32
+ scf.yield %n : f32
+ }
+ return %r : f32
+}
+
+// -----
+
+// `max(%n, 4) >= 4` for any `%n`, so the mask is all-true even though the
+// operand is dynamic.
+
+// CHECK-LABEL: @eliminate_mask_with_dynamic_operand
+// CHECK: %[[ALL_TRUE:.*]] = vector.constant_mask [4] : vector<4xi1>
+// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE]]
+func.func @eliminate_mask_with_dynamic_operand(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
+ %c0 = arith.constant 0 : index
+ %c4 = arith.constant 4 : index
+ %f0 = arith.constant 0.0 : f32
+ %m = arith.maxsi %n, %c4 : index
+ %mask = vector.create_mask %m : vector<4xi1>
+ %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
+ return %v : vector<4xf32>
+}
+
+// -----
+
+// Nothing bounds `%n` from below, so the mask cannot be proven all-true.
+
+// CHECK-LABEL: @negative_unbounded_operand
+// CHECK-NOT: vector.constant_mask
+// CHECK: vector.create_mask
+func.func @negative_unbounded_operand(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
+ %c0 = arith.constant 0 : index
+ %f0 = arith.constant 0.0 : f32
+ %mask = vector.create_mask %n : vector<4xi1>
+ %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
+ return %v : vector<4xf32>
+}
+
+// -----
+
+// The lower bound of `max(%n, 2)` is 2, which is less than the mask dimension,
+// so the mask is not all-true. Unlike the unbounded case above, a bound *is*
+// computed here; it is the comparison against the mask size that rejects it.
+
+// CHECK-LABEL: @negative_lower_bound_too_small
+// CHECK-NOT: vector.constant_mask
+// CHECK: vector.create_mask
+func.func @negative_lower_bound_too_small(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
+ %c0 = arith.constant 0 : index
+ %c2 = arith.constant 2 : index
+ %f0 = arith.constant 0.0 : f32
+ %m = arith.maxsi %n, %c2 : index
+ %mask = vector.create_mask %m : vector<4xi1>
+ %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
+ return %v : vector<4xf32>
+}
+
+// -----
+
+// `max(%n, 4 * vscale) >= 4 * vscale`, which is exactly the runtime size of
+// `vector<[4]xi1>`, so with a vscale range this mask is provably all-true.
+// Without one a scalable dimension cannot be proven, so it must be left alone.
+
+// CHECK-LABEL: @negative_scalable_dim_without_vscale_range
+// CHECK-NOT: vector.constant_mask
+// CHECK: vector.create_mask
+func.func @negative_scalable_dim_without_vscale_range(%t: tensor<?xf32>, %n: index) -> vector<[4]xf32> {
+ %c0 = arith.constant 0 : index
+ %c4 = arith.constant 4 : index
+ %f0 = arith.constant 0.0 : f32
+ %vscale = vector.vscale
+ %c4_vscale = arith.muli %vscale, %c4 : index
+ %m = arith.maxsi %n, %c4_vscale : index
+ %mask = vector.create_mask %m : vector<[4]xi1>
+ %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<[4]xf32>
+ return %v : vector<[4]xf32>
+}
diff --git a/mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp b/mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp
index c0f8a85206cdf..773cb5e29c1a9 100644
--- a/mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp
+++ b/mlir/test/lib/Dialect/Vector/TestVectorTransforms.cpp
@@ -1055,6 +1055,10 @@ struct TestEliminateVectorMasks
Option<unsigned> vscaleMax{
*this, "vscale-max", llvm::cl::desc("Maximum possible value of vscale."),
llvm::cl::init(16)};
+ Option<bool> fixedSize{
+ *this, "fixed-size",
+ llvm::cl::desc("Run without a vscale range, as fixed-size code would."),
+ llvm::cl::init(false)};
StringRef getArgument() const final { return "test-eliminate-vector-masks"; }
StringRef getDescription() const final {
@@ -1062,8 +1066,10 @@ struct TestEliminateVectorMasks
}
void runOnOperation() override {
IRRewriter rewriter(&getContext());
- eliminateVectorMasks(rewriter, getOperation(),
- VscaleRange{vscaleMin, vscaleMax});
+ std::optional<VscaleRange> vscaleRange;
+ if (!fixedSize)
+ vscaleRange = VscaleRange{vscaleMin, vscaleMax};
+ eliminateVectorMasks(rewriter, getOperation(), vscaleRange);
}
};
>From 3d63d9c4108db0098415e77b157fd43d638ad96f Mon Sep 17 00:00:00 2001
From: Dhairyashil R G <dhairyashil25 at gmail.com>
Date: Tue, 8 Sep 2026 21:56:56 +0530
Subject: [PATCH 2/3] Address review comments
- Merge the fixed-size tests into `eliminate-masks.mlir` rather than keeping
a separate file, and run that file twice: once with a vscale range and once
without. Each scalable test now has a fixed-size counterpart, so both widths
get the same coverage. Two cases genuinely differ between the runs and use
`WITH-RANGE`/`NO-RANGE` prefixes; everything else behaves identically with
and without a range.
Two of the scalable tests have no fixed-size counterpart by construction:
`negative_constant_vscale_multiple_not_all_true` exercises `vscale` multiple
matching, and `negative_value_bounds_scalable_dim_not_all_true` needs a
scalable dimension. Both say so in a comment.
- Use `notifyMatchFailure` instead of bare `failure()`. Converted all ten sites
in `resolveAllTrueCreateMaskOp`, not just the new ones, so the function does
not end up half converted. Note this only records a reason for an attached
listener; the current caller passes a plain `IRRewriter`, which has none.
- Rename the new `dimLowerBound` to `constantLowerBound` so it does not share a
name with the differently-typed bound in the scalable branch. Renamed the new
variable rather than the pre-existing one to keep the diff off untouched code.
- Reorder the `eliminateVectorMasks` doc comment as suggested.
Assisted-by: Claude
---
.../Vector/Transforms/VectorTransforms.h | 5 +-
.../Transforms/VectorMaskElimination.cpp | 38 +++--
.../Vector/eliminate-masks-fixed-size.mlir | 97 ------------
mlir/test/Dialect/Vector/eliminate-masks.mlir | 140 +++++++++++++++++-
4 files changed, 160 insertions(+), 120 deletions(-)
delete mode 100644 mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
diff --git a/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h b/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
index e88058fe1ff70..f98e0a47f39ee 100644
--- a/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
+++ b/mlir/include/mlir/Dialect/Vector/Transforms/VectorTransforms.h
@@ -124,10 +124,11 @@ struct VscaleRange {
/// Attempts to eliminate redundant vector masks by replacing them with all-true
/// constants at the top of the function (which results in the masks folding
-/// away). Note: Currently, this only runs for vector.create_mask ops.
-///
+/// away).
/// `vscaleRange` is only needed to prove that a scalable mask dimension is
/// all-true; without it, only fixed-size dimensions are considered.
+///
+/// Note: Currently, this only runs for vector.create_mask ops.
void eliminateVectorMasks(IRRewriter &rewriter, FunctionOpInterface function,
std::optional<VscaleRange> vscaleRange = {});
diff --git a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
index f8f0ac8e33183..9844f129d2e77 100644
--- a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
+++ b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
@@ -43,11 +43,13 @@ resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
if (auto intSize = getConstantIntValue(dimSize)) {
// Mask not all-true for this dim.
if (maskTypeDimScalableFlags[i] || intSize < maskTypeDimSizes[i])
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "constant mask dimension is not all-true");
} else if (auto vscaleMultiplier = getConstantVscaleMultiplier(dimSize)) {
// Mask not all-true for this dim.
if (vscaleMultiplier < maskTypeDimSizes[i])
- return failure();
+ return rewriter.notifyMatchFailure(createMaskOp,
+ "`vscale` multiple is not all-true");
} else {
// Unknown (without further analysis).
unknownDims.push_back(UnknownMaskDim{i, dimSize});
@@ -61,15 +63,18 @@ resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
// A constant bound cannot prove a scalable dim, whose runtime size is
// `vscale` times the size in the type. Checked first to skip the query.
if (maskTypeDimScalableFlags[i])
- return failure();
- FailureOr<int64_t> dimLowerBound =
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "scalable dimension requires a `vscale` range");
+ FailureOr<int64_t> constantLowerBound =
ValueBoundsConstraintSet::computeConstantBound(
presburger::BoundType::LB, dimSize);
- if (failed(dimLowerBound))
- return failure();
+ if (failed(constantLowerBound))
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "no constant lower bound for mask dimension");
// If LB < the mask dim size then this dim is not all-true.
- if (*dimLowerBound < maskTypeDimSizes[i])
- return failure();
+ if (*constantLowerBound < maskTypeDimSizes[i])
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "lower bound is less than the mask dimension size");
continue;
}
@@ -78,23 +83,30 @@ resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
dimSize, {}, vscaleRange->vscaleMin, vscaleRange->vscaleMax,
presburger::BoundType::LB);
if (failed(dimLowerBound))
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "no scalable lower bound for mask dimension");
auto dimLowerBoundSize = dimLowerBound->getSize();
if (failed(dimLowerBoundSize))
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "scalable lower bound has no known size");
if (dimLowerBoundSize->scalable) {
// 1. The lower bound, LB, is scalable. If LB is < the mask dim size then
// this dim is not all-true.
if (dimLowerBoundSize->baseSize < maskTypeDimSizes[i])
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "scalable lower bound is less than the mask "
+ "dimension size");
} else {
// 2. The lower bound, LB, is a constant.
// - If the mask dim size is scalable then this dim is not all-true.
if (maskTypeDimScalableFlags[i])
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "constant lower bound cannot prove a scalable "
+ "mask dimension");
// - If LB < the _fixed-size_ mask dim size then this dim is not all-true.
if (dimLowerBoundSize->baseSize < maskTypeDimSizes[i])
- return failure();
+ return rewriter.notifyMatchFailure(
+ createMaskOp, "lower bound is less than the mask dimension size");
}
}
diff --git a/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir b/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
deleted file mode 100644
index 192285ccc512a..0000000000000
--- a/mlir/test/Dialect/Vector/eliminate-masks-fixed-size.mlir
+++ /dev/null
@@ -1,97 +0,0 @@
-// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks=fixed-size | FileCheck %s
-
-// The mask bound comes from the induction variable, so it is not constant and
-// no fold removes it. Value-bounds analysis proves `%i <= 1020`, hence
-// `1024 - %i >= 4`, so the mask is all-true.
-
-// CHECK-LABEL: @eliminate_mask_bounded_by_induction_variable
-// CHECK: %[[ALL_TRUE:.*]] = vector.constant_mask [4] : vector<4xi1>
-// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE]]
-func.func @eliminate_mask_bounded_by_induction_variable(%t: tensor<1024xf32>) -> f32 {
- %c0 = arith.constant 0 : index
- %c4 = arith.constant 4 : index
- %c1024 = arith.constant 1024 : index
- %f0 = arith.constant 0.0 : f32
- %r = scf.for %i = %c0 to %c1024 step %c4 iter_args(%acc = %f0) -> f32 {
- %rem = arith.subi %c1024, %i : index
- %mask = vector.create_mask %rem : vector<4xi1>
- %v = vector.transfer_read %t[%i], %f0, %mask : tensor<1024xf32>, vector<4xf32>
- %s = vector.reduction <add>, %v : vector<4xf32> into f32
- %n = arith.addf %acc, %s : f32
- scf.yield %n : f32
- }
- return %r : f32
-}
-
-// -----
-
-// `max(%n, 4) >= 4` for any `%n`, so the mask is all-true even though the
-// operand is dynamic.
-
-// CHECK-LABEL: @eliminate_mask_with_dynamic_operand
-// CHECK: %[[ALL_TRUE:.*]] = vector.constant_mask [4] : vector<4xi1>
-// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE]]
-func.func @eliminate_mask_with_dynamic_operand(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
- %c0 = arith.constant 0 : index
- %c4 = arith.constant 4 : index
- %f0 = arith.constant 0.0 : f32
- %m = arith.maxsi %n, %c4 : index
- %mask = vector.create_mask %m : vector<4xi1>
- %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
- return %v : vector<4xf32>
-}
-
-// -----
-
-// Nothing bounds `%n` from below, so the mask cannot be proven all-true.
-
-// CHECK-LABEL: @negative_unbounded_operand
-// CHECK-NOT: vector.constant_mask
-// CHECK: vector.create_mask
-func.func @negative_unbounded_operand(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
- %c0 = arith.constant 0 : index
- %f0 = arith.constant 0.0 : f32
- %mask = vector.create_mask %n : vector<4xi1>
- %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
- return %v : vector<4xf32>
-}
-
-// -----
-
-// The lower bound of `max(%n, 2)` is 2, which is less than the mask dimension,
-// so the mask is not all-true. Unlike the unbounded case above, a bound *is*
-// computed here; it is the comparison against the mask size that rejects it.
-
-// CHECK-LABEL: @negative_lower_bound_too_small
-// CHECK-NOT: vector.constant_mask
-// CHECK: vector.create_mask
-func.func @negative_lower_bound_too_small(%t: tensor<?xf32>, %n: index) -> vector<4xf32> {
- %c0 = arith.constant 0 : index
- %c2 = arith.constant 2 : index
- %f0 = arith.constant 0.0 : f32
- %m = arith.maxsi %n, %c2 : index
- %mask = vector.create_mask %m : vector<4xi1>
- %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<4xf32>
- return %v : vector<4xf32>
-}
-
-// -----
-
-// `max(%n, 4 * vscale) >= 4 * vscale`, which is exactly the runtime size of
-// `vector<[4]xi1>`, so with a vscale range this mask is provably all-true.
-// Without one a scalable dimension cannot be proven, so it must be left alone.
-
-// CHECK-LABEL: @negative_scalable_dim_without_vscale_range
-// CHECK-NOT: vector.constant_mask
-// CHECK: vector.create_mask
-func.func @negative_scalable_dim_without_vscale_range(%t: tensor<?xf32>, %n: index) -> vector<[4]xf32> {
- %c0 = arith.constant 0 : index
- %c4 = arith.constant 4 : index
- %f0 = arith.constant 0.0 : f32
- %vscale = vector.vscale
- %c4_vscale = arith.muli %vscale, %c4 : index
- %m = arith.maxsi %n, %c4_vscale : index
- %mask = vector.create_mask %m : vector<[4]xi1>
- %v = vector.transfer_read %t[%c0], %f0, %mask : tensor<?xf32>, vector<[4]xf32>
- return %v : vector<[4]xf32>
-}
diff --git a/mlir/test/Dialect/Vector/eliminate-masks.mlir b/mlir/test/Dialect/Vector/eliminate-masks.mlir
index 88be7e529bb9e..e968288eb3e32 100644
--- a/mlir/test/Dialect/Vector/eliminate-masks.mlir
+++ b/mlir/test/Dialect/Vector/eliminate-masks.mlir
@@ -1,13 +1,23 @@
-// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks --split-input-file | FileCheck %s
+// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks | FileCheck %s --check-prefixes=CHECK,WITH-RANGE
+// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks=fixed-size | FileCheck %s --check-prefixes=CHECK,NO-RANGE
+
+// Each scalable test below is paired with a fixed-size equivalent, so that both
+// widths get the same coverage. Only the two cases that need a vscale range to
+// be resolved differ between the two runs; everything else behaves identically
+// with and without one.
// This tests a general pattern the vectorizer tends to emit.
// CHECK-LABEL: @eliminate_redundant_masks_through_insert_and_extracts
-// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<[4]xi1>
-// CHECK: vector.transfer_read {{.*}} %[[ALL_TRUE_MASK]]
-// CHECK: vector.mask %[[ALL_TRUE_MASK:.*]] {
-// CHECK-SAME: vector.outerproduct
-// CHECK: vector.transfer_write {{.*}} %[[ALL_TRUE_MASK]]
+// WITH-RANGE: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<[4]xi1>
+// WITH-RANGE: vector.transfer_read {{.*}} %[[ALL_TRUE_MASK]]
+// WITH-RANGE: vector.mask %[[ALL_TRUE_MASK:.*]] {
+// WITH-RANGE-SAME: vector.outerproduct
+// WITH-RANGE: vector.transfer_write {{.*}} %[[ALL_TRUE_MASK]]
+// The mask dimension is scalable, so without a vscale range it cannot be
+// resolved.
+// NO-RANGE-NOT: vector.constant_mask
+// NO-RANGE: vector.create_mask
#map = affine_map<()[s0] -> (-(1080 mod s0) + 1080)>
func.func @eliminate_redundant_masks_through_insert_and_extracts(%tensor: tensor<1x1000xf32>, %rhs : f32) {
@@ -45,6 +55,31 @@ func.func @eliminate_redundant_masks_through_insert_and_extracts(%tensor: tensor
// -----
+// Fixed-size equivalent of the above. The mask bound comes from the induction
+// variable, so it is not constant and no fold removes it, but value bounds
+// proves `%i <= 1020`, hence `1024 - %i >= 4`.
+
+// CHECK-LABEL: @eliminate_redundant_masks_fixed_size
+// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<4xi1>
+// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE_MASK]]
+func.func @eliminate_redundant_masks_fixed_size(%tensor: tensor<1024xf32>) -> f32 {
+ %c0 = arith.constant 0 : index
+ %c4 = arith.constant 4 : index
+ %c1024 = arith.constant 1024 : index
+ %c0_f32 = arith.constant 0.0 : f32
+ %result = scf.for %i = %c0 to %c1024 step %c4 iter_args(%acc = %c0_f32) -> f32 {
+ %rem = arith.subi %c1024, %i : index
+ %mask = vector.create_mask %rem : vector<4xi1>
+ %vec = vector.transfer_read %tensor[%i], %c0_f32, %mask : tensor<1024xf32>, vector<4xf32>
+ %sum = vector.reduction <add>, %vec : vector<4xf32> into f32
+ %new_acc = arith.addf %acc, %sum : f32
+ scf.yield %new_acc : f32
+ }
+ return %result : f32
+}
+
+// -----
+
// Test to ensure that functions without a body are skipped.
// CHECK-LABEL: func.func private @negative_no_func_body()
func.func private @negative_no_func_body()
@@ -88,7 +123,9 @@ func.func @trivially_all_true_case(%tensor: tensor<2x?xf32>)
%c4 = arith.constant 4 : index
%vscale = vector.vscale
%c4_vscale = arith.muli %vscale, %c4 : index
- // Is found to be all true _without_ value bounds analysis.
+ // Is found to be all true _without_ value bounds analysis. A `vscale`
+ // multiple is all-true whenever the multiplier is at least the mask dim
+ // size, whatever `vscale` turns out to be, so no range is needed.
%mask = vector.create_mask %c2, %c4_vscale : vector<2x[4]xi1>
"test.some_use"(%mask) : (vector<2x[4]xi1>) -> ()
return
@@ -96,6 +133,21 @@ func.func @trivially_all_true_case(%tensor: tensor<2x?xf32>)
// -----
+// CHECK-LABEL: @trivially_all_true_case_fixed_size
+// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [2, 4] : vector<2x4xi1>
+// CHECK: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<2x4xi1>) -> ()
+func.func @trivially_all_true_case_fixed_size()
+{
+ %c2 = arith.constant 2 : index
+ %c4 = arith.constant 4 : index
+ // Also found to be all true _without_ value bounds analysis.
+ %mask = vector.create_mask %c2, %c4 : vector<2x4xi1>
+ "test.some_use"(%mask) : (vector<2x4xi1>) -> ()
+ return
+}
+
+// -----
+
// CHECK-LABEL: @negative_constant_dim_not_all_true
// CHECK-NOT: vector.constant_mask
// CHECK: %[[MASK:.*]] = vector.create_mask
@@ -115,6 +167,22 @@ func.func @negative_constant_dim_not_all_true()
// -----
+// CHECK-LABEL: @negative_constant_dim_not_all_true_fixed_size
+// CHECK-NOT: vector.constant_mask
+// CHECK: %[[MASK:.*]] = vector.create_mask
+// CHECK: "test.some_use"(%[[MASK]]) : (vector<2x4xi1>) -> ()
+func.func @negative_constant_dim_not_all_true_fixed_size()
+{
+ %c1 = arith.constant 1 : index
+ %c4 = arith.constant 4 : index
+ // As above, found not to be all-true via simple pattern matching.
+ %mask = vector.create_mask %c1, %c4 : vector<2x4xi1>
+ "test.some_use"(%mask) : (vector<2x4xi1>) -> ()
+ return
+}
+
+// -----
+
// CHECK-LABEL: @negative_constant_vscale_multiple_not_all_true
// CHECK-NOT: vector.constant_mask
// CHECK: %[[MASK:.*]] = vector.create_mask
@@ -125,7 +193,8 @@ func.func @negative_constant_vscale_multiple_not_all_true() {
%vscale = vector.vscale
%c3_vscale = arith.muli %vscale, %c3 : index
// Since %c3_vscale is a constant vscale multiple, this will be found not to
- // be all-true via simple pattern matching.
+ // be all-true via simple pattern matching. There is no fixed-size equivalent
+ // of this case: it exercises the `vscale` multiple matching specifically.
%mask = vector.create_mask %c2, %c3_vscale : vector<2x[4]xi1>
"test.some_use"(%mask) : (vector<2x[4]xi1>) -> ()
return
@@ -153,6 +222,24 @@ func.func @negative_value_bounds_fixed_dim_not_all_true(%tensor: tensor<2x?xf32>
// -----
+// CHECK-LABEL: @negative_value_bounds_fixed_dim_not_all_true_fixed_size
+// CHECK-NOT: vector.constant_mask
+// CHECK: %[[MASK:.*]] = vector.create_mask
+// CHECK: "test.some_use"(%[[MASK]]) : (vector<3x4xi1>) -> ()
+func.func @negative_value_bounds_fixed_dim_not_all_true_fixed_size(%tensor: tensor<2x?xf32>)
+{
+ %c0 = arith.constant 0 : index
+ %c4 = arith.constant 4 : index
+ // As above: a bound is found for %dim (2), but it is smaller than the mask
+ // dimension (3).
+ %dim = tensor.dim %tensor, %c0 : tensor<2x?xf32>
+ %mask = vector.create_mask %dim, %c4 : vector<3x4xi1>
+ "test.some_use"(%mask) : (vector<3x4xi1>) -> ()
+ return
+}
+
+// -----
+
// CHECK-LABEL: @negative_value_bounds_scalable_dim_not_all_true
// CHECK-NOT: vector.constant_mask
// CHECK: %[[MASK:.*]] = vector.create_mask
@@ -164,8 +251,45 @@ func.func @negative_value_bounds_scalable_dim_not_all_true(%tensor: tensor<2x100
%c3_vscale = arith.muli %vscale, %c3 : index
%slice = tensor.extract_slice %tensor[0, 0] [2, %c3_vscale] [1, 1] : tensor<2x100xf32> to tensor<2x?xf32>
// Another simple example, but value bounds will be used to resolve the tensor.dim.
+ // There is no fixed-size equivalent: the dimension under test is scalable.
%dim = tensor.dim %slice, %c1 : tensor<2x?xf32>
%mask = vector.create_mask %c3, %dim : vector<3x[4]xi1>
"test.some_use"(%mask) : (vector<3x[4]xi1>) -> ()
return
}
+
+// -----
+
+// Nothing bounds %n from below, so no bound is found at all.
+
+// CHECK-LABEL: @negative_unbounded_operand_fixed_size
+// CHECK-NOT: vector.constant_mask
+// CHECK: %[[MASK:.*]] = vector.create_mask
+// CHECK: "test.some_use"(%[[MASK]]) : (vector<4xi1>) -> ()
+func.func @negative_unbounded_operand_fixed_size(%n: index) {
+ %mask = vector.create_mask %n : vector<4xi1>
+ "test.some_use"(%mask) : (vector<4xi1>) -> ()
+ return
+}
+
+// -----
+
+// A mask dimension that is only provably all-true once a vscale range is known.
+// `max(%n, 4 * vscale) >= 4 * vscale`, which is exactly the runtime size of
+// `vector<[4]xi1>`.
+
+// CHECK-LABEL: @scalable_dim_requires_vscale_range
+// WITH-RANGE: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<[4]xi1>
+// WITH-RANGE: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<[4]xi1>) -> ()
+// NO-RANGE-NOT: vector.constant_mask
+// NO-RANGE: %[[MASK:.*]] = vector.create_mask
+// NO-RANGE: "test.some_use"(%[[MASK]]) : (vector<[4]xi1>) -> ()
+func.func @scalable_dim_requires_vscale_range(%n: index) {
+ %c4 = arith.constant 4 : index
+ %vscale = vector.vscale
+ %c4_vscale = arith.muli %vscale, %c4 : index
+ %m = arith.maxsi %n, %c4_vscale : index
+ %mask = vector.create_mask %m : vector<[4]xi1>
+ "test.some_use"(%mask) : (vector<[4]xi1>) -> ()
+ return
+}
>From d337dff83d968d177dd2272f637275c9e8854ba3 Mon Sep 17 00:00:00 2001
From: Dhairyashil R G <dhairyashil25 at gmail.com>
Date: Mon, 21 Sep 2026 19:31:04 +0530
Subject: [PATCH 3/3] Address review comments
- Mark the two cases in `resolveAllTrueCreateMaskOp` with headings. The
pre-existing "compute the lower bound" comment was written for the scalable
query and now sits above the fixed-width branch inserted under it, so it read
as if it introduced that branch.
- Revert `notifyMatchFailure` back to bare `failure()` at all ten sites, as
requested. It only records a reason for an attached listener and the sole
caller passes a plain `IRRewriter`, so it was documentation rather than
diagnostics. This also restores the style the function had before this patch.
- Rename the shared FileCheck prefix from `CHECK` to `ALL`.
- Replace `@scalable_dim_requires_vscale_range` with
`@negative_scalable_dim_constant_lower_bound`. The old test duplicated the
coverage of `@eliminate_redundant_masks_through_insert_and_extracts`, and it
did not test the guard it was named after: deleting the scalable-dimension
bail on the fixed-width path left it passing, because its `max(%n, 4 * vscale)`
bound is declined by the constant-bound path anyway. With `max(%n, 4)` the
mask is folded to an all-true `vector.constant_mask [4] : vector<[4]xi1>` once
that guard is removed, which is wrong for every `vscale > 1`, so the test now
fails without it. Both runs behave the same, so it uses the shared prefix.
Only one case now differs between the runs, and three scalable tests have no
fixed-size counterpart.
Assisted-by: Claude
---
.../Transforms/VectorMaskElimination.cpp | 37 +++---
mlir/test/Dialect/Vector/eliminate-masks.mlir | 121 +++++++++---------
2 files changed, 75 insertions(+), 83 deletions(-)
diff --git a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
index 9844f129d2e77..e6ee2b047de07 100644
--- a/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
+++ b/mlir/lib/Dialect/Vector/Transforms/VectorMaskElimination.cpp
@@ -43,13 +43,11 @@ resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
if (auto intSize = getConstantIntValue(dimSize)) {
// Mask not all-true for this dim.
if (maskTypeDimScalableFlags[i] || intSize < maskTypeDimSizes[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "constant mask dimension is not all-true");
+ return failure();
} else if (auto vscaleMultiplier = getConstantVscaleMultiplier(dimSize)) {
// Mask not all-true for this dim.
if (vscaleMultiplier < maskTypeDimSizes[i])
- return rewriter.notifyMatchFailure(createMaskOp,
- "`vscale` multiple is not all-true");
+ return failure();
} else {
// Unknown (without further analysis).
unknownDims.push_back(UnknownMaskDim{i, dimSize});
@@ -59,54 +57,49 @@ resolveAllTrueCreateMaskOp(IRRewriter &rewriter,
for (auto [i, dimSize] : unknownDims) {
// Compute the lower bound for the unknown dimension (i.e. the smallest
// value it could be).
+
+ // Fixed-width case: without a `vscale` range the bound is a plain constant,
+ // which can only prove a fixed-size dimension all-true.
if (!vscaleRange) {
// A constant bound cannot prove a scalable dim, whose runtime size is
// `vscale` times the size in the type. Checked first to skip the query.
if (maskTypeDimScalableFlags[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "scalable dimension requires a `vscale` range");
+ return failure();
FailureOr<int64_t> constantLowerBound =
ValueBoundsConstraintSet::computeConstantBound(
presburger::BoundType::LB, dimSize);
if (failed(constantLowerBound))
- return rewriter.notifyMatchFailure(
- createMaskOp, "no constant lower bound for mask dimension");
+ return failure();
// If LB < the mask dim size then this dim is not all-true.
if (*constantLowerBound < maskTypeDimSizes[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "lower bound is less than the mask dimension size");
+ return failure();
continue;
}
+ // Scalable case: with a `vscale` range the bound has the form
+ // `base + n * vscale`, which can prove either kind of dimension all-true.
FailureOr<ConstantOrScalableBound> dimLowerBound =
vector::ScalableValueBoundsConstraintSet::computeScalableBound(
dimSize, {}, vscaleRange->vscaleMin, vscaleRange->vscaleMax,
presburger::BoundType::LB);
if (failed(dimLowerBound))
- return rewriter.notifyMatchFailure(
- createMaskOp, "no scalable lower bound for mask dimension");
+ return failure();
auto dimLowerBoundSize = dimLowerBound->getSize();
if (failed(dimLowerBoundSize))
- return rewriter.notifyMatchFailure(
- createMaskOp, "scalable lower bound has no known size");
+ return failure();
if (dimLowerBoundSize->scalable) {
// 1. The lower bound, LB, is scalable. If LB is < the mask dim size then
// this dim is not all-true.
if (dimLowerBoundSize->baseSize < maskTypeDimSizes[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "scalable lower bound is less than the mask "
- "dimension size");
+ return failure();
} else {
// 2. The lower bound, LB, is a constant.
// - If the mask dim size is scalable then this dim is not all-true.
if (maskTypeDimScalableFlags[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "constant lower bound cannot prove a scalable "
- "mask dimension");
+ return failure();
// - If LB < the _fixed-size_ mask dim size then this dim is not all-true.
if (dimLowerBoundSize->baseSize < maskTypeDimSizes[i])
- return rewriter.notifyMatchFailure(
- createMaskOp, "lower bound is less than the mask dimension size");
+ return failure();
}
}
diff --git a/mlir/test/Dialect/Vector/eliminate-masks.mlir b/mlir/test/Dialect/Vector/eliminate-masks.mlir
index e968288eb3e32..002c502dc5fdb 100644
--- a/mlir/test/Dialect/Vector/eliminate-masks.mlir
+++ b/mlir/test/Dialect/Vector/eliminate-masks.mlir
@@ -1,14 +1,14 @@
-// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks | FileCheck %s --check-prefixes=CHECK,WITH-RANGE
-// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks=fixed-size | FileCheck %s --check-prefixes=CHECK,NO-RANGE
+// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks | FileCheck %s --check-prefixes=ALL,WITH-RANGE
+// RUN: mlir-opt %s -split-input-file -test-eliminate-vector-masks=fixed-size | FileCheck %s --check-prefixes=ALL,NO-RANGE
// Each scalable test below is paired with a fixed-size equivalent, so that both
-// widths get the same coverage. Only the two cases that need a vscale range to
-// be resolved differ between the two runs; everything else behaves identically
+// widths get the same coverage. Only the one case that needs a vscale range to
+// be resolved differs between the two runs; everything else behaves identically
// with and without one.
// This tests a general pattern the vectorizer tends to emit.
-// CHECK-LABEL: @eliminate_redundant_masks_through_insert_and_extracts
+// ALL-LABEL: @eliminate_redundant_masks_through_insert_and_extracts
// WITH-RANGE: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<[4]xi1>
// WITH-RANGE: vector.transfer_read {{.*}} %[[ALL_TRUE_MASK]]
// WITH-RANGE: vector.mask %[[ALL_TRUE_MASK:.*]] {
@@ -59,9 +59,9 @@ func.func @eliminate_redundant_masks_through_insert_and_extracts(%tensor: tensor
// variable, so it is not constant and no fold removes it, but value bounds
// proves `%i <= 1020`, hence `1024 - %i >= 4`.
-// CHECK-LABEL: @eliminate_redundant_masks_fixed_size
-// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<4xi1>
-// CHECK: vector.transfer_read {{.*}}, %[[ALL_TRUE_MASK]]
+// ALL-LABEL: @eliminate_redundant_masks_fixed_size
+// ALL: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<4xi1>
+// ALL: vector.transfer_read {{.*}}, %[[ALL_TRUE_MASK]]
func.func @eliminate_redundant_masks_fixed_size(%tensor: tensor<1024xf32>) -> f32 {
%c0 = arith.constant 0 : index
%c4 = arith.constant 4 : index
@@ -81,15 +81,15 @@ func.func @eliminate_redundant_masks_fixed_size(%tensor: tensor<1024xf32>) -> f3
// -----
// Test to ensure that functions without a body are skipped.
-// CHECK-LABEL: func.func private @negative_no_func_body()
+// ALL-LABEL: func.func private @negative_no_func_body()
func.func private @negative_no_func_body()
// -----
-// CHECK-LABEL: @negative_extract_slice_size_shrink
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<[4]xi1>) -> ()
+// ALL-LABEL: @negative_extract_slice_size_shrink
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<[4]xi1>) -> ()
func.func @negative_extract_slice_size_shrink(%tensor: tensor<1000xf32>) {
%c0 = arith.constant 0 : index
%c4 = arith.constant 4 : index
@@ -114,9 +114,9 @@ func.func @negative_extract_slice_size_shrink(%tensor: tensor<1000xf32>) {
// -----
-// CHECK-LABEL: @trivially_all_true_case
-// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [2, 4] : vector<2x[4]xi1>
-// CHECK: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<2x[4]xi1>) -> ()
+// ALL-LABEL: @trivially_all_true_case
+// ALL: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [2, 4] : vector<2x[4]xi1>
+// ALL: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<2x[4]xi1>) -> ()
func.func @trivially_all_true_case(%tensor: tensor<2x?xf32>)
{
%c2 = arith.constant 2 : index
@@ -133,9 +133,9 @@ func.func @trivially_all_true_case(%tensor: tensor<2x?xf32>)
// -----
-// CHECK-LABEL: @trivially_all_true_case_fixed_size
-// CHECK: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [2, 4] : vector<2x4xi1>
-// CHECK: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<2x4xi1>) -> ()
+// ALL-LABEL: @trivially_all_true_case_fixed_size
+// ALL: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [2, 4] : vector<2x4xi1>
+// ALL: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<2x4xi1>) -> ()
func.func @trivially_all_true_case_fixed_size()
{
%c2 = arith.constant 2 : index
@@ -148,10 +148,10 @@ func.func @trivially_all_true_case_fixed_size()
// -----
-// CHECK-LABEL: @negative_constant_dim_not_all_true
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<2x[4]xi1>) -> ()
+// ALL-LABEL: @negative_constant_dim_not_all_true
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<2x[4]xi1>) -> ()
func.func @negative_constant_dim_not_all_true()
{
%c1 = arith.constant 1 : index
@@ -167,10 +167,10 @@ func.func @negative_constant_dim_not_all_true()
// -----
-// CHECK-LABEL: @negative_constant_dim_not_all_true_fixed_size
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<2x4xi1>) -> ()
+// ALL-LABEL: @negative_constant_dim_not_all_true_fixed_size
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<2x4xi1>) -> ()
func.func @negative_constant_dim_not_all_true_fixed_size()
{
%c1 = arith.constant 1 : index
@@ -183,10 +183,10 @@ func.func @negative_constant_dim_not_all_true_fixed_size()
// -----
-// CHECK-LABEL: @negative_constant_vscale_multiple_not_all_true
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<2x[4]xi1>) -> ()
+// ALL-LABEL: @negative_constant_vscale_multiple_not_all_true
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<2x[4]xi1>) -> ()
func.func @negative_constant_vscale_multiple_not_all_true() {
%c2 = arith.constant 2 : index
%c3 = arith.constant 3 : index
@@ -202,10 +202,10 @@ func.func @negative_constant_vscale_multiple_not_all_true() {
// -----
-// CHECK-LABEL: @negative_value_bounds_fixed_dim_not_all_true
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<3x[4]xi1>) -> ()
+// ALL-LABEL: @negative_value_bounds_fixed_dim_not_all_true
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<3x[4]xi1>) -> ()
func.func @negative_value_bounds_fixed_dim_not_all_true(%tensor: tensor<2x?xf32>)
{
%c0 = arith.constant 0 : index
@@ -222,10 +222,10 @@ func.func @negative_value_bounds_fixed_dim_not_all_true(%tensor: tensor<2x?xf32>
// -----
-// CHECK-LABEL: @negative_value_bounds_fixed_dim_not_all_true_fixed_size
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<3x4xi1>) -> ()
+// ALL-LABEL: @negative_value_bounds_fixed_dim_not_all_true_fixed_size
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<3x4xi1>) -> ()
func.func @negative_value_bounds_fixed_dim_not_all_true_fixed_size(%tensor: tensor<2x?xf32>)
{
%c0 = arith.constant 0 : index
@@ -240,10 +240,10 @@ func.func @negative_value_bounds_fixed_dim_not_all_true_fixed_size(%tensor: tens
// -----
-// CHECK-LABEL: @negative_value_bounds_scalable_dim_not_all_true
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<3x[4]xi1>) -> ()
+// ALL-LABEL: @negative_value_bounds_scalable_dim_not_all_true
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<3x[4]xi1>) -> ()
func.func @negative_value_bounds_scalable_dim_not_all_true(%tensor: tensor<2x100xf32>) {
%c1 = arith.constant 1 : index
%c3 = arith.constant 3 : index
@@ -262,10 +262,10 @@ func.func @negative_value_bounds_scalable_dim_not_all_true(%tensor: tensor<2x100
// Nothing bounds %n from below, so no bound is found at all.
-// CHECK-LABEL: @negative_unbounded_operand_fixed_size
-// CHECK-NOT: vector.constant_mask
-// CHECK: %[[MASK:.*]] = vector.create_mask
-// CHECK: "test.some_use"(%[[MASK]]) : (vector<4xi1>) -> ()
+// ALL-LABEL: @negative_unbounded_operand_fixed_size
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<4xi1>) -> ()
func.func @negative_unbounded_operand_fixed_size(%n: index) {
%mask = vector.create_mask %n : vector<4xi1>
"test.some_use"(%mask) : (vector<4xi1>) -> ()
@@ -274,21 +274,20 @@ func.func @negative_unbounded_operand_fixed_size(%n: index) {
// -----
-// A mask dimension that is only provably all-true once a vscale range is known.
-// `max(%n, 4 * vscale) >= 4 * vscale`, which is exactly the runtime size of
-// `vector<[4]xi1>`.
-
-// CHECK-LABEL: @scalable_dim_requires_vscale_range
-// WITH-RANGE: %[[ALL_TRUE_MASK:.*]] = vector.constant_mask [4] : vector<[4]xi1>
-// WITH-RANGE: "test.some_use"(%[[ALL_TRUE_MASK]]) : (vector<[4]xi1>) -> ()
-// NO-RANGE-NOT: vector.constant_mask
-// NO-RANGE: %[[MASK:.*]] = vector.create_mask
-// NO-RANGE: "test.some_use"(%[[MASK]]) : (vector<[4]xi1>) -> ()
-func.func @scalable_dim_requires_vscale_range(%n: index) {
+// A constant lower bound can never prove a scalable dimension all-true, in
+// either run: `max(%n, 4) >= 4`, but `vector<[4]xi1>` holds `4 * vscale`
+// elements, so the mask is only all-true when `vscale` is 1. Without the
+// scalable-dimension check on the fixed-width path this is folded to an
+// all-true mask, which is wrong for every `vscale > 1`.
+
+// ALL-LABEL: @negative_scalable_dim_constant_lower_bound
+// ALL-NOT: vector.constant_mask
+// ALL: %[[MASK:.*]] = vector.create_mask
+// ALL: "test.some_use"(%[[MASK]]) : (vector<[4]xi1>) -> ()
+func.func @negative_scalable_dim_constant_lower_bound(%n: index) {
%c4 = arith.constant 4 : index
- %vscale = vector.vscale
- %c4_vscale = arith.muli %vscale, %c4 : index
- %m = arith.maxsi %n, %c4_vscale : index
+ // There is no fixed-size equivalent: the dimension under test is scalable.
+ %m = arith.maxsi %n, %c4 : index
%mask = vector.create_mask %m : vector<[4]xi1>
"test.some_use"(%mask) : (vector<[4]xi1>) -> ()
return
More information about the Mlir-commits
mailing list