[Mlir-commits] [mlir] [mlir][SCFToControlFlow] Carry LLVM attributes through scf.parallel lowering (PR #219218)
llvmlistbot at llvm.org
llvmlistbot at llvm.org
Thu Aug 27 07:24:26 PDT 2026
https://github.com/khaki3 created https://github.com/llvm/llvm-project/pull/219218
`ParallelLowering` builds its `scf.for` nest without copying anything from the
`scf.parallel`, so an `llvm.loop_annotation` placed on a parallel loop is
silently dropped before `ForLowering` can move it onto the latch branch.
`scf.for` and `scf.while` already propagate LLVM-dialect attributes via
`propagateLoopAttrs`, so do the same for `scf.parallel`. A multi-dimensional
`scf.parallel` carries a single attribute dictionary but becomes several loops,
so the attributes go to the innermost one, whose latch is where `ForLowering`
attaches the loop metadata.
Tests cover the 1-D case and a 2-D nest, where the outer latch is checked to
stay unannotated. Verified that the new tests fail without the fix and pass with
it, and that the pre-existing expectations in `convert-to-cfg.mlir` are
unchanged.
>From 098353f4d8406cc594c99edc8fd8fd1a9d088b64 Mon Sep 17 00:00:00 2001
From: Kazuaki Matsumura <kmatsumura at nvidia.com>
Date: Thu, 20 Aug 2026 16:57:02 -0700
Subject: [PATCH 1/3] [mlir][SCFToControlFlow] Carry LLVM attributes through
scf.parallel lowering
ParallelLowering built its scf.for nest without copying anything from the
scf.parallel, so an llvm.loop_annotation placed on a parallel loop was
silently dropped before ForLowering could move it onto the latch branch.
scf.for and scf.while already propagate these attributes, so do the same
for scf.parallel by copying them to the innermost generated loop, which is
the one whose latch carries the loop metadata.
---
.../SCFToControlFlow/SCFToControlFlow.cpp | 22 ++++++++++++++-----
.../SCFToControlFlow/convert-to-cfg.mlir | 18 +++++++++++++++
2 files changed, 34 insertions(+), 6 deletions(-)
diff --git a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
index 2972d79c4302f..0de94894dcaf7 100644
--- a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
+++ b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
@@ -313,17 +313,20 @@ struct ForallLowering : public OpRewritePattern<mlir::scf::ForallOp> {
} // namespace
+static void copyLLVMDialectAttrs(Operation *from, Operation *to) {
+ SmallVector<NamedAttribute> llvmAttrs;
+ llvm::copy_if(from->getAttrs(), std::back_inserter(llvmAttrs), [](auto attr) {
+ return isa<LLVM::LLVMDialect>(attr.getValue().getDialect());
+ });
+ to->setDiscardableAttrs(llvmAttrs);
+}
+
static void propagateLoopAttrs(Operation *scfOp, Operation *brOp) {
// Let the CondBranchOp carry the LLVM attributes from the ForOp, such as the
// llvm.loop_annotation attribute.
// LLVM requires the loop metadata to be attached on the "latch" block. Which
// is the back-edge to the header block (conditionBlock)
- SmallVector<NamedAttribute> llvmAttrs;
- llvm::copy_if(scfOp->getAttrs(), std::back_inserter(llvmAttrs),
- [](auto attr) {
- return isa<LLVM::LLVMDialect>(attr.getValue().getDialect());
- });
- brOp->setDiscardableAttrs(llvmAttrs);
+ copyLLVMDialectAttrs(scfOp, brOp);
}
LogicalResult ForLowering::matchAndRewrite(ForOp forOp,
@@ -507,10 +510,12 @@ ParallelLowering::matchAndRewrite(ParallelOp parallelOp,
ivs.reserve(parallelOp.getNumLoops());
bool first = true;
SmallVector<Value, 4> loopResults(iterArgs);
+ ForOp innermostForOp;
for (auto [iv, lower, upper, step] :
llvm::zip(parallelOp.getInductionVars(), parallelOp.getLowerBound(),
parallelOp.getUpperBound(), parallelOp.getStep())) {
ForOp forOp = ForOp::create(rewriter, loc, lower, upper, step, iterArgs);
+ innermostForOp = forOp;
ivs.push_back(forOp.getInductionVar());
auto iterRange = forOp.getRegionIterArgs();
iterArgs.assign(iterRange.begin(), iterRange.end());
@@ -530,6 +535,11 @@ ParallelLowering::matchAndRewrite(ParallelOp parallelOp,
rewriter.setInsertionPointToStart(forOp.getBody());
}
+ // Carry LLVM attributes such as llvm.loop_annotation over to the innermost
+ // generated loop, which is the one whose latch will hold the loop metadata.
+ if (innermostForOp)
+ copyLLVMDialectAttrs(parallelOp, innermostForOp);
+
// First, merge reduction blocks into the main region.
SmallVector<Value> yieldOperands;
yieldOperands.reserve(parallelOp.getNumResults());
diff --git a/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir b/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
index 7ad8d594a23ab..eed1b914f9098 100644
--- a/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
+++ b/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
@@ -795,3 +795,21 @@ func.func @do_while_loops_annotation() {
return
}
+// -----
+
+// CHECK: #[[LOOP_UNROLL_DISABLE:.*]] = #llvm.loop_unroll<disable = true>
+// CHECK: #[[NO_UNROLL:.*]] = #llvm.loop_annotation<unroll = #[[LOOP_UNROLL_DISABLE]]>
+// CHECK: func @parallel_loop_annotation
+// CHECK: cf.cond_br
+// CHECK: cf.br {{.*}} {llvm.loop_annotation = #[[NO_UNROLL]]}
+// CHECK: return
+#no_unroll = #llvm.loop_annotation<unroll = <disable = true>>
+func.func @parallel_loop_annotation(%arg0 : index, %arg1 : index, %arg2 : index, %arg3 : memref<?xf32>) {
+ %cst = arith.constant 1.0 : f32
+ scf.parallel (%i) = (%arg0) to (%arg1) step (%arg2) {
+ memref.store %cst, %arg3[%i] : memref<?xf32>
+ scf.reduce
+ } {llvm.loop_annotation = #no_unroll}
+ return
+}
+
>From cefb14268489cd86bad94d7077acb229cd084d97 Mon Sep 17 00:00:00 2001
From: Kazuaki Matsumura <kmatsumura at nvidia.com>
Date: Mon, 24 Aug 2026 14:57:53 -0700
Subject: [PATCH 2/3] [mlir][SCFToControlFlow] Test annotation placement for a
parallel loop nest
A multi-dimensional scf.parallel carries one attribute dictionary but lowers
to a loop nest, so pin down that the loop metadata lands on the innermost
loop's latch and nowhere else.
---
.../SCFToControlFlow/SCFToControlFlow.cpp | 6 ++++--
.../SCFToControlFlow/convert-to-cfg.mlir | 21 +++++++++++++++++++
2 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
index 0de94894dcaf7..d1dacc17da24f 100644
--- a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
+++ b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
@@ -535,8 +535,10 @@ ParallelLowering::matchAndRewrite(ParallelOp parallelOp,
rewriter.setInsertionPointToStart(forOp.getBody());
}
- // Carry LLVM attributes such as llvm.loop_annotation over to the innermost
- // generated loop, which is the one whose latch will hold the loop metadata.
+ // Carry LLVM attributes such as llvm.loop_annotation over to the generated
+ // nest. A multi-dimensional scf.parallel has a single attribute dictionary
+ // but becomes several loops, so the attributes go to the innermost one, whose
+ // latch is where ForLowering will attach the loop metadata.
if (innermostForOp)
copyLLVMDialectAttrs(parallelOp, innermostForOp);
diff --git a/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir b/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
index eed1b914f9098..be89bfe9780ba 100644
--- a/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
+++ b/mlir/test/Conversion/SCFToControlFlow/convert-to-cfg.mlir
@@ -813,3 +813,24 @@ func.func @parallel_loop_annotation(%arg0 : index, %arg1 : index, %arg2 : index,
return
}
+// -----
+
+// A multi-dimensional scf.parallel lowers to a loop nest, and the metadata goes
+// to the innermost loop's latch only.
+// CHECK: #[[LOOP_UNROLL_DISABLE:.*]] = #llvm.loop_unroll<disable = true>
+// CHECK: #[[NO_UNROLL:.*]] = #llvm.loop_annotation<unroll = #[[LOOP_UNROLL_DISABLE]]>
+// CHECK: func @parallel_loop_annotation_2d
+// CHECK: cf.cond_br
+// CHECK: cf.cond_br
+// CHECK: cf.br {{.*}} {llvm.loop_annotation = #[[NO_UNROLL]]}
+// CHECK-NOT: llvm.loop_annotation
+// CHECK: return
+#no_unroll = #llvm.loop_annotation<unroll = <disable = true>>
+func.func @parallel_loop_annotation_2d(%arg0 : index, %arg1 : index, %arg2 : index, %arg3 : memref<?x?xf32>) {
+ %cst = arith.constant 1.0 : f32
+ scf.parallel (%i, %j) = (%arg0, %arg0) to (%arg1, %arg1) step (%arg2, %arg2) {
+ memref.store %cst, %arg3[%i, %j] : memref<?x?xf32>
+ scf.reduce
+ } {llvm.loop_annotation = #no_unroll}
+ return
+}
>From c695fe3ee600921c1dd72bed8efcdcfe63e93de8 Mon Sep 17 00:00:00 2001
From: Kazuaki Matsumura <kmatsumura at nvidia.com>
Date: Thu, 27 Aug 2026 07:22:17 -0700
Subject: [PATCH 3/3] [mlir][SCFToControlFlow] Clarify why parallel attributes
go to the inner loop
The comment jumped straight to latches while talking about scf.parallel, which
has none. Say that the conversion serializes the parallel loop into the for
nest first, so the rest follows.
---
mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
index d1dacc17da24f..4053915a40356 100644
--- a/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
+++ b/mlir/lib/Conversion/SCFToControlFlow/SCFToControlFlow.cpp
@@ -535,10 +535,9 @@ ParallelLowering::matchAndRewrite(ParallelOp parallelOp,
rewriter.setInsertionPointToStart(forOp.getBody());
}
- // Carry LLVM attributes such as llvm.loop_annotation over to the generated
- // nest. A multi-dimensional scf.parallel has a single attribute dictionary
- // but becomes several loops, so the attributes go to the innermost one, whose
- // latch is where ForLowering will attach the loop metadata.
+ // Serializing into the for nest would drop attributes such as
+ // llvm.loop_annotation. The loop above emits one scf.for per dimension, so
+ // the attributes go to the innermost one, which runs the body.
if (innermostForOp)
copyLLVMDialectAttrs(parallelOp, innermostForOp);
More information about the Mlir-commits
mailing list