[Mlir-commits] [mlir] [mlir][ArmSME] Add test-lower-to-arm-sme pipeline (PR #81732)
Cullen Rhodes
llvmlistbot at llvm.org
Thu Feb 22 00:55:10 PST 2024
https://github.com/c-rhodes updated https://github.com/llvm/llvm-project/pull/81732
>From c8af435c6bd0f8bf934e60dd3b5667287d3ad410 Mon Sep 17 00:00:00 2001
From: Cullen Rhodes <cullen.rhodes at arm.com>
Date: Tue, 13 Feb 2024 12:22:59 +0000
Subject: [PATCH 1/2] [mlir][ArmSME] Add test-lower-to-arm-sme pipeline
The ArmSME compilation pipeline has evolved significantly and is now
sufficiently complex enough that it warrants a proper lowering pipeline
that encapsulates the various passes and orderings. Currently the
pipeline is loosely defined in our integration tests, but these have
diverged and are not using the same passes or ordering everywhere.
This patch introduces a test-lower-to-arm-sme pipeline mirroring
test-lower-to-llvm that provides some sanity when running e2e examples
and can be used a reference for targeting ArmSME in MLIR.
All the integration tests are updated to use this pipeline. The
intention is to productize the pipeline once it becomes more mature.
---
.../Dialect/Linalg/CPU/ArmSME/fill-2d.mlir | 11 +-
.../Linalg/CPU/ArmSME/matmul-transpose-a.mlir | 8 +-
.../Dialect/Linalg/CPU/ArmSME/matmul.mlir | 8 +-
.../Linalg/CPU/ArmSME/multi-tile-matmul.mlir | 7 +-
.../Linalg/CPU/ArmSME/use-too-many-tiles.mlir | 7 +-
.../CPU/ArmSME/load-store-128-bit-tile.mlir | 6 +-
.../Vector/CPU/ArmSME/test-load-vertical.mlir | 6 +-
.../CPU/ArmSME/test-multi-tile-transpose.mlir | 8 +-
.../ArmSME/test-outerproduct-f16f16f32.mlir | 11 +-
.../CPU/ArmSME/test-outerproduct-f32.mlir | 7 +-
.../CPU/ArmSME/test-outerproduct-f64.mlir | 7 +-
.../CPU/ArmSME/test-outerproduct-i8i8i32.mlir | 8 +-
.../CPU/ArmSME/test-transfer-read-2d.mlir | 6 +-
.../CPU/ArmSME/test-transfer-write-2d.mlir | 7 +-
.../Vector/CPU/ArmSME/test-transpose.mlir | 6 +-
.../Dialect/Vector/CPU/ArmSME/tile_fill.mlir | 6 +-
.../Vector/CPU/ArmSME/vector-load-store.mlir | 6 +-
.../Dialect/Vector/CPU/ArmSME/vector-ops.mlir | 5 +-
mlir/test/lib/Dialect/ArmSME/CMakeLists.txt | 27 +++
.../lib/Dialect/ArmSME/TestLowerToArmSME.cpp | 157 ++++++++++++++++++
mlir/test/lib/Dialect/CMakeLists.txt | 1 +
mlir/tools/mlir-opt/CMakeLists.txt | 1 +
mlir/tools/mlir-opt/mlir-opt.cpp | 2 +
23 files changed, 209 insertions(+), 109 deletions(-)
create mode 100644 mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
create mode 100644 mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
index 44ff1afe76d383..8724e09ba0bd98 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
@@ -1,13 +1,6 @@
// RUN: mlir-opt %s \
-// RUN: -transform-interpreter \
-// RUN: -test-transform-dialect-erase-schedule \
-// RUN: -lower-vector-mask \
-// RUN: -one-shot-bufferize="bufferize-function-boundaries" \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// RUN: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// RUN: -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -convert-arm-sme-to-llvm -cse -canonicalize \
-// RUN: -test-lower-to-llvm | \
+// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
+// RUN: -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=entry -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
index c781d5e0af846e..656b04815c8562 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
@@ -1,12 +1,6 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -one-shot-bufferize="bufferize-function-boundaries" -canonicalize \
-// RUN: -convert-vector-to-arm-sme -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// RUN: -convert-vector-to-scf -cse -arm-sve-legalize-vector-storage \
-// RUN: -convert-arm-sme-to-llvm \
-// RUN: -convert-vector-to-llvm=enable-arm-sve \
-// RUN: -cse -canonicalize -test-lower-to-llvm | \
+// RUN: -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
index 31c3202c3fc57b..2ac0bb591e442a 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
@@ -1,12 +1,6 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -canonicalize \
-// RUN: -convert-vector-to-arm-sme -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// RUN: -convert-vector-to-scf -cse -arm-sve-legalize-vector-storage \
-// RUN: -convert-arm-sme-to-llvm \
-// RUN: -convert-vector-to-llvm=enable-arm-sve \
-// RUN: -cse -canonicalize -test-lower-to-llvm | \
+// RUN: -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
index d5c35068ccb32e..2a55e91b316043 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
@@ -1,11 +1,6 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -one-shot-bufferize="bufferize-function-boundaries" -canonicalize \
-// RUN: -arm-sme-vector-legalization -canonicalize -cse \
-// RUN: -convert-vector-to-arm-sme -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// RUN: -convert-vector-to-scf=full-unroll -convert-arm-sme-to-llvm \
-// RUN: -test-lower-to-llvm | \
+// RUN: -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
index 42fe21cccd48a7..9c84a36fb17a22 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
@@ -1,10 +1,5 @@
// RUN: mlir-opt %s \
-// RUN: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// RUN: -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// RUN: -convert-vector-to-scf -cse -arm-sve-legalize-vector-storage \
-// RUN: -convert-arm-sme-to-llvm -convert-vector-to-llvm=enable-arm-sve -cse \
-// RUN: -canonicalize -test-lower-to-llvm -verify-diagnostics | \
+// RUN: -test-lower-to-arm-sme -verify-diagnostics | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
index 59b4a7e6a52f9b..8bf2c87b848a1a 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
@@ -1,9 +1,5 @@
// DEFINE: %{entry_point} = test_load_store_zaq0
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arm-sme-to-scf \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -allocate-arm-sme-tiles -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
index 064141c349241e..0ef15353e9248e 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
@@ -1,9 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
index 0827d9b7464add..b088cd135b36a4 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
@@ -1,10 +1,4 @@
-// RUN: mlir-opt %s -arm-sme-vector-legalization -cse -canonicalize \
-// RUN: -convert-vector-to-arm-sme -allocate-arm-sme-tiles -convert-arm-sme-to-scf \
-// RUN: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// RUN: -convert-vector-to-scf -cse -arm-sve-legalize-vector-storage \
-// RUN: -convert-arm-sme-to-llvm \
-// RUN: -convert-vector-to-llvm=enable-arm-sve \
-// RUN: -cse -canonicalize -test-lower-to-llvm | \
+// RUN: mlir-opt %s -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
index f081838300a9a4..d2153e8241d572 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
@@ -1,11 +1,6 @@
+// DEFINE: %{opts} =
// DEFINE: %{entry} = main
-// DEFINE: %{fusion_opts} = -arm-sme-outer-product-fusion
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme %{fusion_opts} \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm -o %t
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme=%{opts} -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry} -entry-point-result=void \
@@ -18,7 +13,7 @@
// Check result is the same when outerproducts are not combined into widening
// variant.
-// REDEFINE: %{fusion_opts} =
+// REDEFINE: %{opts} = fuse-outer-products=false
// RUN: %{run} | FileCheck %s
func.func @main() {
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
index 5f41b37560e760..0e07e2299bdc81 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
@@ -1,10 +1,5 @@
// DEFINE: %{entry_point} = test_outerproduct_no_accumulator_4x4xf32
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm -o %t
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
index a1bb9b7d6f80ec..8fb4864895b63a 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
@@ -1,10 +1,5 @@
// DEFINE: %{entry_point} = test_outerproduct_no_accumulator_2x2xf64
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm -o %t
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme-f64f64 \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
index 1770e579f0bd68..befd78e92a161a 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
@@ -1,11 +1,5 @@
// DEFINE: %{entry} = main
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// DEFINE: -arm-sme-outer-product-fusion \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
index 6e028d5fb83614..387531122dcc16 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
@@ -1,9 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -convert-vector-to-arm-sme -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
index c0c1f55d7ddd1a..415dd2ffae97a2 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
@@ -1,10 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za only-if-required-by-ops" \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
index eee3c56351d81e..ec344caeb24cad 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
@@ -1,9 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
index 223bc8ce74343b..85509a88ca8964 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
@@ -1,8 +1,4 @@
-// RUN: mlir-opt %s -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// RUN: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// RUN: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// RUN: -convert-arm-sme-to-llvm -cse -canonicalize \
-// RUN: -test-lower-to-llvm | \
+// RUN: mlir-opt %s -test-lower-to-arm-sme | \
// RUN: %mcr_aarch64_cmd \
// RUN: -march=aarch64 -mattr=+sve,+sme \
// RUN: -e entry -entry-point-result=i32 \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
index 2f151e2ec72fb7..6a1f3b49889f00 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
@@ -1,9 +1,5 @@
// DEFINE: %{entry_point} = za0_d_f64
-// DEFINE: %{compile} = mlir-opt %s \
-// DEFINE: -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -cse -canonicalize \
-// DEFINE: -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=i32 \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
index f28bf19b299934..15c892a5b57294 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
@@ -1,8 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -enable-arm-streaming="streaming-mode=streaming-locally za-mode=new-za" \
-// DEFINE: -convert-vector-to-arm-sme -convert-arith-to-arm-sme \
-// DEFINE: -convert-arm-sme-to-scf -allocate-arm-sme-tiles \
-// DEFINE: -convert-arm-sme-to-llvm -test-lower-to-llvm
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=i32 \
diff --git a/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt b/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
new file mode 100644
index 00000000000000..40442d9a0405dd
--- /dev/null
+++ b/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
@@ -0,0 +1,27 @@
+# Exclude tests from libMLIR.so
+add_mlir_library(MLIRArmSMETestPasses
+ TestLowerToArmSME.cpp
+
+ EXCLUDE_FROM_LIBMLIR
+
+ LINK_LIBS PUBLIC
+ MLIRAffineToStandard
+ MLIRArithToArmSME
+ MLIRArmSMEToLLVM
+ MLIRArmSMEToSCF
+ MLIRFuncDialect
+ MLIRFuncToLLVM
+ MLIRIR
+ MLIRIndexToLLVM
+ MLIRLLVMDialect
+ MLIRLinalgTransforms
+ MLIRMathToLLVM
+ MLIRMemRefToLLVM
+ MLIRMemRefTransforms
+ MLIRPass
+ MLIRReconcileUnrealizedCasts
+ MLIRSCFToControlFlow
+ MLIRTransforms
+ MLIRVectorToLLVMPass
+ MLIRVectorToSCF
+ )
diff --git a/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp b/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
new file mode 100644
index 00000000000000..bbeb5ca2b4c532
--- /dev/null
+++ b/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
@@ -0,0 +1,157 @@
+//===- TestLowerToArmSME.cpp - Test lowering to ArmSME as a sink pass -----===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements a pass for testing the lowering to ArmSME as a
+// generally usable sink pass.
+//
+//===----------------------------------------------------------------------===//
+
+#include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
+#include "mlir/Conversion/ArithToArmSME/ArithToArmSME.h"
+#include "mlir/Conversion/ArmSMEToLLVM/ArmSMEToLLVM.h"
+#include "mlir/Conversion/ArmSMEToSCF/ArmSMEToSCF.h"
+#include "mlir/Conversion/FuncToLLVM/ConvertFuncToLLVMPass.h"
+#include "mlir/Conversion/IndexToLLVM/IndexToLLVM.h"
+#include "mlir/Conversion/MathToLLVM/MathToLLVM.h"
+#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
+#include "mlir/Conversion/ReconcileUnrealizedCasts/ReconcileUnrealizedCasts.h"
+#include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h"
+#include "mlir/Conversion/VectorToArmSME/VectorToArmSME.h"
+#include "mlir/Conversion/VectorToLLVM/ConvertVectorToLLVMPass.h"
+#include "mlir/Conversion/VectorToSCF/VectorToSCF.h"
+#include "mlir/Dialect/ArmSME/Transforms/Passes.h"
+#include "mlir/Dialect/ArmSVE/Transforms/Passes.h"
+#include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h"
+#include "mlir/Dialect/Bufferization/Transforms/Passes.h"
+#include "mlir/Dialect/Func/IR/FuncOps.h"
+#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
+#include "mlir/Dialect/Linalg/Passes.h"
+#include "mlir/Dialect/MemRef/Transforms/Passes.h"
+#include "mlir/Dialect/Vector/Transforms/Passes.h"
+#include "mlir/IR/DialectRegistry.h"
+#include "mlir/Pass/Pass.h"
+#include "mlir/Pass/PassManager.h"
+#include "mlir/Pass/PassOptions.h"
+#include "mlir/Transforms/Passes.h"
+
+using namespace mlir;
+
+namespace {
+struct TestLowerToArmSMEOptions
+ : public PassPipelineOptions<TestLowerToArmSMEOptions> {
+ PassOptions::Option<bool> fuseOuterProducts{
+ *this, "fuse-outer-products",
+ llvm::cl::desc("Fuse outer product operations via "
+ "'-arm-sme-outer-product-fusion' pass"),
+ llvm::cl::init(true)};
+};
+
+void buildTestLowerToArmSME(OpPassManager &pm,
+ const TestLowerToArmSMEOptions &options) {
+ // Lower 'vector.mask' operations.
+ pm.addPass(vector::createLowerVectorMaskPass());
+
+ // One shot bufferize. Convert ops with tensor semantics to ops with memref
+ // semantics in a single pass.
+ bufferization::OneShotBufferizationOptions bufferizationOptions;
+ bufferizationOptions.bufferizeFunctionBoundaries = true;
+ pm.addPass(bufferization::createOneShotBufferizePass(bufferizationOptions));
+
+ // Sprinkle some cleanups.
+ pm.addPass(createCanonicalizerPass());
+
+ // Legalize vector operations so they can be converted to ArmSME.
+ pm.addPass(arm_sme::createVectorLegalizationPass());
+
+ // Sprinkle some cleanups.
+ pm.addPass(createCanonicalizerPass());
+ pm.addPass(createCSEPass());
+
+ // Passes that convert operations on vectors to ArmSME operations.
+
+ // Convert Arith to ArmSME.
+ pm.addPass(createArithToArmSMEConversionPass());
+ // Convert Vector to ArmSME.
+ pm.addPass(createConvertVectorToArmSMEPass());
+
+ // Fuse outer products.
+ if (options.fuseOuterProducts)
+ pm.addPass(arm_sme::createOuterProductFusionPass());
+
+ // Convert operations on high-level vectors to loops.
+
+ // Convert ArmSME to SCF.
+ pm.addPass(createConvertArmSMEToSCFPass());
+ // Convert Vector to SCF.
+ pm.addPass(createConvertVectorToSCFPass());
+
+ // Allocate tiles for ArmSME operations.
+ //
+ // Later passes may create further ArmSME ops that implement the
+ // ArmSMETileOpInterface, but tiles are allocated for root operations,
+ // all of which should now exist.
+ pm.addPass(arm_sme::createTileAllocationPass());
+
+ // Configure PSTATE.SM and PSTATE.ZA.
+ pm.addPass(arm_sme::createEnableArmStreamingPass(
+ arm_sme::ArmStreamingMode::StreamingLocally, arm_sme::ArmZaMode::NewZA,
+ /*onlyIfRequiredByOps=*/true));
+
+ // Legalize operations on SVE vector types.
+ pm.addPass(arm_sve::createLegalizeVectorStoragePass());
+
+ // Convert ArmSME to LLVM.
+ pm.addPass(createConvertArmSMEToLLVMPass());
+
+ // Sprinkle some cleanups.
+ pm.addPass(createCanonicalizerPass());
+ pm.addPass(createCSEPass());
+
+ // The remaining passes are taken from the -test-lower-to-llvm pipeline.
+
+ // Blanket-convert any remaining linalg ops to loops if any remain.
+ pm.addNestedPass<func::FuncOp>(createConvertLinalgToLoopsPass());
+ // Blanket-convert any remaining affine ops if any remain.
+ pm.addPass(createLowerAffinePass());
+ // Convert SCF to CF (always needed).
+ pm.addPass(createConvertSCFToCFPass());
+ // Sprinkle some cleanups.
+ pm.addPass(createCanonicalizerPass());
+ pm.addPass(createCSEPass());
+ // Convert vector to LLVM (always needed).
+ ConvertVectorToLLVMPassOptions lowerVectorToLLVMOptions{};
+ lowerVectorToLLVMOptions.armSVE = true;
+ pm.addPass(createConvertVectorToLLVMPass(lowerVectorToLLVMOptions));
+ // Convert Math to LLVM (always needed).
+ pm.addNestedPass<func::FuncOp>(createConvertMathToLLVMPass());
+ // Expand complicated MemRef operations before lowering them.
+ pm.addPass(memref::createExpandStridedMetadataPass());
+ // The expansion may create affine expressions. Get rid of them.
+ pm.addPass(createLowerAffinePass());
+ // Convert MemRef to LLVM (always needed).
+ pm.addPass(createFinalizeMemRefToLLVMConversionPass());
+ // Convert Func to LLVM (always needed).
+ pm.addPass(createConvertFuncToLLVMPass());
+ // Convert Index to LLVM (always needed).
+ pm.addPass(createConvertIndexToLLVMPass());
+ // Convert remaining unrealized_casts (always needed).
+ pm.addPass(createReconcileUnrealizedCastsPass());
+}
+} // namespace
+
+namespace mlir {
+namespace test {
+void registerTestLowerToArmSME() {
+ PassPipelineRegistration<TestLowerToArmSMEOptions>(
+ "test-lower-to-arm-sme",
+ "An example pipeline to lower operations on vectors (arith, vector) to "
+ "LLVM via ArmSME.",
+ buildTestLowerToArmSME);
+}
+} // namespace test
+} // namespace mlir
diff --git a/mlir/test/lib/Dialect/CMakeLists.txt b/mlir/test/lib/Dialect/CMakeLists.txt
index 30a17c201ff763..e20cd4473a3580 100644
--- a/mlir/test/lib/Dialect/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/CMakeLists.txt
@@ -1,5 +1,6 @@
add_subdirectory(Affine)
add_subdirectory(Arith)
+add_subdirectory(ArmSME)
add_subdirectory(Bufferization)
add_subdirectory(ControlFlow)
add_subdirectory(DLTI)
diff --git a/mlir/tools/mlir-opt/CMakeLists.txt b/mlir/tools/mlir-opt/CMakeLists.txt
index 68aa6bad5f92c5..701fc461b3b4e9 100644
--- a/mlir/tools/mlir-opt/CMakeLists.txt
+++ b/mlir/tools/mlir-opt/CMakeLists.txt
@@ -17,6 +17,7 @@ if(MLIR_INCLUDE_TESTS)
MLIRTestFuncToLLVM
MLIRAffineTransformsTestPasses
MLIRArithTestPasses
+ MLIRArmSMETestPasses
MLIRBufferizationTestPasses
MLIRControlFlowTestPasses
MLIRDLTITestPasses
diff --git a/mlir/tools/mlir-opt/mlir-opt.cpp b/mlir/tools/mlir-opt/mlir-opt.cpp
index cec1e5225d5a6c..06d60bf44b2246 100644
--- a/mlir/tools/mlir-opt/mlir-opt.cpp
+++ b/mlir/tools/mlir-opt/mlir-opt.cpp
@@ -109,6 +109,7 @@ void registerTestLoopFusion();
void registerTestCFGLoopInfoPass();
void registerTestLoopMappingPass();
void registerTestLoopUnrollingPass();
+void registerTestLowerToArmSME();
void registerTestLowerToLLVM();
void registerTestMakeIsolatedFromAbovePass();
void registerTestMatchReductionPass();
@@ -233,6 +234,7 @@ void registerTestPasses() {
mlir::test::registerTestCFGLoopInfoPass();
mlir::test::registerTestLoopMappingPass();
mlir::test::registerTestLoopUnrollingPass();
+ mlir::test::registerTestLowerToArmSME();
mlir::test::registerTestLowerToLLVM();
mlir::test::registerTestMakeIsolatedFromAbovePass();
mlir::test::registerTestMatchReductionPass();
>From b147919659f20ea119cd84484325c5060ca09fbd Mon Sep 17 00:00:00 2001
From: Cullen Rhodes <cullen.rhodes at arm.com>
Date: Thu, 22 Feb 2024 08:36:01 +0000
Subject: [PATCH 2/2] remove test-lower-to-llvm passes
---
.../Dialect/Linalg/CPU/ArmSME/fill-2d.mlir | 4 +-
.../Linalg/CPU/ArmSME/matmul-transpose-a.mlir | 3 +-
.../Dialect/Linalg/CPU/ArmSME/matmul.mlir | 2 +-
.../Linalg/CPU/ArmSME/multi-tile-matmul.mlir | 3 +-
.../Linalg/CPU/ArmSME/use-too-many-tiles.mlir | 2 +-
.../CPU/ArmSME/load-store-128-bit-tile.mlir | 2 +-
.../Vector/CPU/ArmSME/test-load-vertical.mlir | 2 +-
.../CPU/ArmSME/test-multi-tile-transpose.mlir | 2 +-
.../ArmSME/test-outerproduct-f16f16f32.mlir | 3 +-
.../CPU/ArmSME/test-outerproduct-f32.mlir | 3 +-
.../CPU/ArmSME/test-outerproduct-f64.mlir | 3 +-
.../CPU/ArmSME/test-outerproduct-i8i8i32.mlir | 2 +-
.../CPU/ArmSME/test-transfer-read-2d.mlir | 2 +-
.../CPU/ArmSME/test-transfer-write-2d.mlir | 2 +-
.../Vector/CPU/ArmSME/test-transpose.mlir | 2 +-
.../Dialect/Vector/CPU/ArmSME/tile_fill.mlir | 2 +-
.../Vector/CPU/ArmSME/vector-load-store.mlir | 2 +-
.../Dialect/Vector/CPU/ArmSME/vector-ops.mlir | 2 +-
mlir/test/lib/Dialect/ArmSME/CMakeLists.txt | 13 +---
.../lib/Dialect/ArmSME/TestLowerToArmSME.cpp | 68 ++-----------------
20 files changed, 31 insertions(+), 93 deletions(-)
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
index 8724e09ba0bd98..12f13e8dbc4a9f 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/fill-2d.mlir
@@ -1,6 +1,8 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -test-lower-to-arm-sme | \
+// RUN: -lower-vector-mask \
+// RUN: -one-shot-bufferize="bufferize-function-boundaries" \
+// RUN: -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=entry -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
index 656b04815c8562..34c5351c8703d4 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul-transpose-a.mlir
@@ -1,6 +1,7 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -test-lower-to-arm-sme | \
+// RUN: -one-shot-bufferize="bufferize-function-boundaries" \
+// RUN: -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
index 2ac0bb591e442a..2bfdaa8e8a2be0 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/matmul.mlir
@@ -1,6 +1,6 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -test-lower-to-arm-sme | \
+// RUN: -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
index 2a55e91b316043..e376bdde24a15d 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/multi-tile-matmul.mlir
@@ -1,6 +1,7 @@
// RUN: mlir-opt %s \
// RUN: -transform-interpreter -test-transform-dialect-erase-schedule \
-// RUN: -test-lower-to-arm-sme | \
+// RUN: -one-shot-bufferize="bufferize-function-boundaries" -canonicalize \
+// RUN: -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
index 9c84a36fb17a22..ee3866de303e03 100644
--- a/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
+++ b/mlir/test/Integration/Dialect/Linalg/CPU/ArmSME/use-too-many-tiles.mlir
@@ -1,5 +1,5 @@
// RUN: mlir-opt %s \
-// RUN: -test-lower-to-arm-sme -verify-diagnostics | \
+// RUN: -test-lower-to-arm-sme -test-lower-to-llvm -verify-diagnostics | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
index 8bf2c87b848a1a..06b1c107cb2c16 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/load-store-128-bit-tile.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = test_load_store_zaq0
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
index 0ef15353e9248e..27be801252b812 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-load-vertical.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
index b088cd135b36a4..9d836d93c85bb7 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-multi-tile-transpose.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -test-lower-to-arm-sme | \
+// RUN: mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -e=main -entry-point-result=void \
// RUN: -march=aarch64 -mattr="+sve,+sme" \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
index d2153e8241d572..a06ad37b054e45 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f16f16f32.mlir
@@ -1,6 +1,7 @@
// DEFINE: %{opts} =
// DEFINE: %{entry} = main
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme=%{opts} -o %t
+// DEFINE: %{compile} = mlir-opt %s \
+// DEFINE: -test-lower-to-arm-sme=%{opts} -test-lower-to-llvm -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
index 0e07e2299bdc81..7e7869d1c957aa 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f32.mlir
@@ -1,5 +1,6 @@
// DEFINE: %{entry_point} = test_outerproduct_no_accumulator_4x4xf32
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -o %t
+// DEFINE: %{compile} = mlir-opt %s \
+// DEFINE: -test-lower-to-arm-sme -test-lower-to-llvm -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
index 8fb4864895b63a..46bf799232ae3a 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-f64.mlir
@@ -1,5 +1,6 @@
// DEFINE: %{entry_point} = test_outerproduct_no_accumulator_2x2xf64
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -o %t
+// DEFINE: %{compile} = mlir-opt %s \
+// DEFINE: -test-lower-to-arm-sme -test-lower-to-llvm -o %t
// DEFINE: %{run} = %mcr_aarch64_cmd %t \
// DEFINE: -march=aarch64 -mattr=+sve,+sme-f64f64 \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
index befd78e92a161a..9a353ec2d2f666 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-outerproduct-i8i8i32.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry} = main
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
index 387531122dcc16..52f56883cad9c1 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-read-2d.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
index 415dd2ffae97a2..710cc6672f0057 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transfer-write-2d.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
index ec344caeb24cad..88bc0d0709d489 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/test-transpose.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=void \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
index 85509a88ca8964..e14917486d845d 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/tile_fill.mlir
@@ -1,4 +1,4 @@
-// RUN: mlir-opt %s -test-lower-to-arm-sme | \
+// RUN: mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm | \
// RUN: %mcr_aarch64_cmd \
// RUN: -march=aarch64 -mattr=+sve,+sme \
// RUN: -e entry -entry-point-result=i32 \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
index 6a1f3b49889f00..b29790db14ddc4 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-load-store.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = za0_d_f64
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=i32 \
diff --git a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
index 15c892a5b57294..c8c401bed14469 100644
--- a/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
+++ b/mlir/test/Integration/Dialect/Vector/CPU/ArmSME/vector-ops.mlir
@@ -1,5 +1,5 @@
// DEFINE: %{entry_point} = entry
-// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme
+// DEFINE: %{compile} = mlir-opt %s -test-lower-to-arm-sme -test-lower-to-llvm
// DEFINE: %{run} = %mcr_aarch64_cmd \
// DEFINE: -march=aarch64 -mattr=+sve,+sme \
// DEFINE: -e %{entry_point} -entry-point-result=i32 \
diff --git a/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt b/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
index 40442d9a0405dd..de4971ff7eb3d1 100644
--- a/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
+++ b/mlir/test/lib/Dialect/ArmSME/CMakeLists.txt
@@ -5,23 +5,12 @@ add_mlir_library(MLIRArmSMETestPasses
EXCLUDE_FROM_LIBMLIR
LINK_LIBS PUBLIC
- MLIRAffineToStandard
MLIRArithToArmSME
MLIRArmSMEToLLVM
MLIRArmSMEToSCF
- MLIRFuncDialect
- MLIRFuncToLLVM
MLIRIR
- MLIRIndexToLLVM
- MLIRLLVMDialect
- MLIRLinalgTransforms
- MLIRMathToLLVM
- MLIRMemRefToLLVM
- MLIRMemRefTransforms
MLIRPass
- MLIRReconcileUnrealizedCasts
- MLIRSCFToControlFlow
MLIRTransforms
- MLIRVectorToLLVMPass
+ MLIRVectorToArmSME
MLIRVectorToSCF
)
diff --git a/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp b/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
index bbeb5ca2b4c532..48d4a5859f8a08 100644
--- a/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
+++ b/mlir/test/lib/Dialect/ArmSME/TestLowerToArmSME.cpp
@@ -11,28 +11,13 @@
//
//===----------------------------------------------------------------------===//
-#include "mlir/Conversion/AffineToStandard/AffineToStandard.h"
#include "mlir/Conversion/ArithToArmSME/ArithToArmSME.h"
#include "mlir/Conversion/ArmSMEToLLVM/ArmSMEToLLVM.h"
#include "mlir/Conversion/ArmSMEToSCF/ArmSMEToSCF.h"
-#include "mlir/Conversion/FuncToLLVM/ConvertFuncToLLVMPass.h"
-#include "mlir/Conversion/IndexToLLVM/IndexToLLVM.h"
-#include "mlir/Conversion/MathToLLVM/MathToLLVM.h"
-#include "mlir/Conversion/MemRefToLLVM/MemRefToLLVM.h"
-#include "mlir/Conversion/ReconcileUnrealizedCasts/ReconcileUnrealizedCasts.h"
-#include "mlir/Conversion/SCFToControlFlow/SCFToControlFlow.h"
#include "mlir/Conversion/VectorToArmSME/VectorToArmSME.h"
-#include "mlir/Conversion/VectorToLLVM/ConvertVectorToLLVMPass.h"
#include "mlir/Conversion/VectorToSCF/VectorToSCF.h"
#include "mlir/Dialect/ArmSME/Transforms/Passes.h"
#include "mlir/Dialect/ArmSVE/Transforms/Passes.h"
-#include "mlir/Dialect/Bufferization/Transforms/OneShotAnalysis.h"
-#include "mlir/Dialect/Bufferization/Transforms/Passes.h"
-#include "mlir/Dialect/Func/IR/FuncOps.h"
-#include "mlir/Dialect/LLVMIR/LLVMDialect.h"
-#include "mlir/Dialect/Linalg/Passes.h"
-#include "mlir/Dialect/MemRef/Transforms/Passes.h"
-#include "mlir/Dialect/Vector/Transforms/Passes.h"
#include "mlir/IR/DialectRegistry.h"
#include "mlir/Pass/Pass.h"
#include "mlir/Pass/PassManager.h"
@@ -53,18 +38,6 @@ struct TestLowerToArmSMEOptions
void buildTestLowerToArmSME(OpPassManager &pm,
const TestLowerToArmSMEOptions &options) {
- // Lower 'vector.mask' operations.
- pm.addPass(vector::createLowerVectorMaskPass());
-
- // One shot bufferize. Convert ops with tensor semantics to ops with memref
- // semantics in a single pass.
- bufferization::OneShotBufferizationOptions bufferizationOptions;
- bufferizationOptions.bufferizeFunctionBoundaries = true;
- pm.addPass(bufferization::createOneShotBufferizePass(bufferizationOptions));
-
- // Sprinkle some cleanups.
- pm.addPass(createCanonicalizerPass());
-
// Legalize vector operations so they can be converted to ArmSME.
pm.addPass(arm_sme::createVectorLegalizationPass());
@@ -87,8 +60,10 @@ void buildTestLowerToArmSME(OpPassManager &pm,
// Convert ArmSME to SCF.
pm.addPass(createConvertArmSMEToSCFPass());
- // Convert Vector to SCF.
- pm.addPass(createConvertVectorToSCFPass());
+
+ // Convert Vector to SCF (with full unroll enabled).
+ pm.addPass(createConvertVectorToSCFPass(
+ VectorTransferToSCFOptions().enableFullUnroll()));
// Allocate tiles for ArmSME operations.
//
@@ -97,50 +72,17 @@ void buildTestLowerToArmSME(OpPassManager &pm,
// all of which should now exist.
pm.addPass(arm_sme::createTileAllocationPass());
- // Configure PSTATE.SM and PSTATE.ZA.
+ // Enable streaming-mode and ZA.
pm.addPass(arm_sme::createEnableArmStreamingPass(
arm_sme::ArmStreamingMode::StreamingLocally, arm_sme::ArmZaMode::NewZA,
/*onlyIfRequiredByOps=*/true));
- // Legalize operations on SVE vector types.
- pm.addPass(arm_sve::createLegalizeVectorStoragePass());
-
// Convert ArmSME to LLVM.
pm.addPass(createConvertArmSMEToLLVMPass());
// Sprinkle some cleanups.
pm.addPass(createCanonicalizerPass());
pm.addPass(createCSEPass());
-
- // The remaining passes are taken from the -test-lower-to-llvm pipeline.
-
- // Blanket-convert any remaining linalg ops to loops if any remain.
- pm.addNestedPass<func::FuncOp>(createConvertLinalgToLoopsPass());
- // Blanket-convert any remaining affine ops if any remain.
- pm.addPass(createLowerAffinePass());
- // Convert SCF to CF (always needed).
- pm.addPass(createConvertSCFToCFPass());
- // Sprinkle some cleanups.
- pm.addPass(createCanonicalizerPass());
- pm.addPass(createCSEPass());
- // Convert vector to LLVM (always needed).
- ConvertVectorToLLVMPassOptions lowerVectorToLLVMOptions{};
- lowerVectorToLLVMOptions.armSVE = true;
- pm.addPass(createConvertVectorToLLVMPass(lowerVectorToLLVMOptions));
- // Convert Math to LLVM (always needed).
- pm.addNestedPass<func::FuncOp>(createConvertMathToLLVMPass());
- // Expand complicated MemRef operations before lowering them.
- pm.addPass(memref::createExpandStridedMetadataPass());
- // The expansion may create affine expressions. Get rid of them.
- pm.addPass(createLowerAffinePass());
- // Convert MemRef to LLVM (always needed).
- pm.addPass(createFinalizeMemRefToLLVMConversionPass());
- // Convert Func to LLVM (always needed).
- pm.addPass(createConvertFuncToLLVMPass());
- // Convert Index to LLVM (always needed).
- pm.addPass(createConvertIndexToLLVMPass());
- // Convert remaining unrealized_casts (always needed).
- pm.addPass(createReconcileUnrealizedCastsPass());
}
} // namespace
More information about the Mlir-commits
mailing list