[all-commits] [llvm/llvm-project] 7e526f: [mlir][affine] emit `in_bounds` on `transfer_read`...

[Federico Bruzzone via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 7e526f7bc4ace92236d7d07cb5771031347f2ce4
      https://github.com/llvm/llvm-project/commit/7e526f7bc4ace92236d7d07cb5771031347f2ce4
  Author: Federico Bruzzone <federico.bruzzone.i at gmail.com>
  Date:   2026-06-09 (Tue, 09 Jun 2026)

  Changed paths:
    M mlir/lib/Dialect/Affine/Transforms/SuperVectorize.cpp
    M mlir/test/Dialect/Affine/SuperVectorize/vector_utils.mlir
    M mlir/test/Dialect/Affine/SuperVectorize/vectorize_1d.mlir
    M mlir/test/Dialect/Affine/SuperVectorize/vectorize_2d.mlir
    A mlir/test/Dialect/Affine/SuperVectorize/vectorize_2d_inbounds.mlir
    M mlir/test/Dialect/Affine/SuperVectorize/vectorize_affine_apply.mlir
    M mlir/test/Dialect/Affine/SuperVectorize/vectorize_reduction.mlir

  Log Message:
  -----------
  [mlir][affine] emit `in_bounds` on `transfer_read`/`write` when statically provable in `affine-super-vectorize` (#201180)

This patch fixes an issue reported on the MLIR Discourse ([May
2026](https://discourse.llvm.org/t/mlir-affine-affine-super-vectorize-does-not-set-in-bounds-on-transfer-ops-for-statically-divisible-shapes/90785/3)),
that I also came across during a study I reported in [my blog
post](https://federicobruzzone.github.io/posts/mlir-study.html).

`affine-super-vectorize` always creates `vector.transfer_read` and
`vector.transfer_write` without an `in_bounds` attribute, even when it
is statically provable that every access stays within bounds. This
forces downstream lowering to unconditionally emit
`llvm.intr.masked.load`/`llvm.intr.masked.store`: masked intrinsics that
carry $\sim3\times$ overhead on AArch64/NEON and prevent
auto-vectorization (see the MLIR Discourse).

## Root cause

`vectorizeAffineLoad` and `vectorizeAffineStore` in `SuperVectorize.cpp`
forwarded neither the `in_bounds` mask nor any analysis of it when
constructing the transfer ops. The fix computes the mask at
vectorization time by inspecting the permutation map and the memref type
via a new `computeInBoundsMask` helper:
- `AffineDimExpr`: the vector dimension maps to a concrete memref
dimension. If that dimension is static and divisible by the vector
width, the accesses are guaranteed in-bounds.
- `AffineConstantExpr`: a broadcast (the dimension is collapsed to a
constant index). A broadcast can never be out-of-bounds.
- Everything else (e.g., AffineAddExpr on dynamic dimensions):
conservatively left false.

<details>
<summary>Reproduction</summary>

Run the lowering pipeline on any static-size affine copy loop:

```mlir
// file: copy.mlir
func.func @copy(%A: memref<512x512xf32>, %B: memref<512x512xf32>) {
  affine.for %i = 0 to 512 {
    affine.for %j = 0 to 512 {
      %v = affine.load %A[%i, %j] : memref<512x512xf32>
      affine.store %v, %B[%i, %j] : memref<512x512xf32>
    }
  }
  return
}
```

```
mlir-opt copy.mlir \
  --affine-super-vectorize="virtual-vector-size=4" \
  --convert-vector-to-llvm \
  --finalize-memref-to-llvm \
  --convert-func-to-llvm
```

Before this patch: masked intrinsics despite fully static, divisible
dimensions:
```
%31 = llvm.intr.masked.load %30, %25, %16 {alignment = 4 : i32} : (!llvm.ptr, vector<4xi1>, vector<4xf32>) -> vector<4xf32>
llvm.intr.masked.store %31, %43, %38 {alignment = 4 : i32} : vector<4xf32>, vector<4xi1> into !llvm.ptr
```
After this patch: plain vector load/store:
```
%23 = llvm.load %22 {alignment = 4 : i64} : !llvm.ptr -> vector<4xf32>
llvm.store %23, %28 {alignment = 4 : i64} : vector<4xf32>, !llvm.ptr
```
</details>

AI Disclaimer: I used AI for the tests.

---------

Signed-off-by: Federico Bruzzone <federico.bruzzone.i at gmail.com>
Co-authored-by: Artem Gindinson <gindinson at roofline.ai>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications