[llvm-branch-commits] [llvm] [AMDGPU] Raise the MachinePipeliner MII cap via a target hook (PR #212537)
via llvm-branch-commits
llvm-branch-commits at lists.llvm.org
Wed Jul 29 09:51:33 PDT 2026
https://github.com/hjagasiaAMD updated https://github.com/llvm/llvm-project/pull/212537
>From ea86be1c21543701eaccf3a7d4c5fcb0cff6f965 Mon Sep 17 00:00:00 2001
From: Harsha Jagasia <harsha.jagasia at amd.com>
Date: Thu, 16 Jul 2026 16:19:27 +0000
Subject: [PATCH] [AMDGPU] Raise the MachinePipeliner MII cap via a target hook
The pipeliner rejects any loop whose minimum initiation interval (MII)
exceeds -pipeliner-max-mii, default 27. That is far too low for real
AMDGPU loops: resource-bound GEMM/attention bodies routinely have an MII
well above 27 and are dropped before scheduling even starts.
Add a PipelinerLoopInfo::getMaxMII() hook so a target can raise the cap,
and override it to 256 for AMDGPU. The generic default is unchanged, and
an explicit -pipeliner-max-mii still takes precedence.
The cap of 256 is chosen from the II distributions of two AMDGPU
workloads.
Composable Kernels:
II range count pct
0- 24 2305 36.7%
25- 49 1681 26.8%
50- 99 1582 25.3%
100-149 578 9.2%
150-299 64 1.0%
Triton:
II range count pct
96-128 19 50.0%
128-160 11 28.9%
160-192 2 5.3%
192-224 2 5.3%
224-256 4 10.5%
256 captures the bulk of both workloads while still rejecting
pathologically large loops up front.
---
llvm/include/llvm/CodeGen/TargetInstrInfo.h | 5 +++++
llvm/lib/CodeGen/MachinePipeliner.cpp | 13 +++++++++----
llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 4 ++++
.../CodeGen/AMDGPU/swp-amdgpu-pipeline-max-mii.ll | 11 +++++------
4 files changed, 23 insertions(+), 10 deletions(-)
diff --git a/llvm/include/llvm/CodeGen/TargetInstrInfo.h b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
index 4749d06501cb2..50c95a2208284 100644
--- a/llvm/include/llvm/CodeGen/TargetInstrInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
@@ -841,6 +841,11 @@ class LLVM_ABI TargetInstrInfo : public MCInstrInfo {
return true;
}
+ /// Return a target-specific cap on the minimum initiation interval (MII)
+ /// above which a loop is not pipelined, or nullopt to use the default cap.
+ /// An explicit -pipeliner-max-mii takes precedence over this hook.
+ virtual std::optional<unsigned> getMaxMII() const { return std::nullopt; }
+
/// Create a condition to determine if the trip count of the loop is greater
/// than TC, where TC is always one more than for the previous prologue or
/// 0 if this is being called for the outermost prologue.
diff --git a/llvm/lib/CodeGen/MachinePipeliner.cpp b/llvm/lib/CodeGen/MachinePipeliner.cpp
index a97e0702d1911..d715e1e966998 100644
--- a/llvm/lib/CodeGen/MachinePipeliner.cpp
+++ b/llvm/lib/CodeGen/MachinePipeliner.cpp
@@ -813,16 +813,21 @@ void SwingSchedulerDAG::schedule() {
}
// Don't pipeline large loops.
- if (SwpMaxMii != -1 && (int)MII > SwpMaxMii) {
- LLVM_DEBUG(dbgs() << "MII > " << SwpMaxMii
+ int MaxMII = SwpMaxMii;
+ if (SwpMaxMii.getNumOccurrences() == 0 && LoopPipelinerInfo)
+ if (std::optional<unsigned> TargetMaxMII = LoopPipelinerInfo->getMaxMII())
+ MaxMII = (int)*TargetMaxMII;
+
+ if (MaxMII != -1 && (int)MII > MaxMII) {
+ LLVM_DEBUG(dbgs() << "MII > " << MaxMII
<< ", we don't pipeline large loops\n");
NumFailLargeMaxMII++;
Pass.ORE->emit([&]() {
return MachineOptimizationRemarkAnalysis(
DEBUG_TYPE, "schedule", Loop.getStartLoc(), Loop.getHeader())
<< "Minimal Initiation Interval too large: "
- << ore::NV("MII", (int)MII) << " > "
- << ore::NV("SwpMaxMii", SwpMaxMii) << "."
+ << ore::NV("MII", (int)MII) << " > " << ore::NV("MaxMII", MaxMII)
+ << "."
<< "Refer to -pipeliner-max-mii.";
});
return;
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
index be49e5285be60..f76f67e4bdf91 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
@@ -3307,6 +3307,10 @@ class AMDGPUPipelinerLoopInfo : public TargetInstrInfo::PipelinerLoopInfo {
return CmpInst && MI == CmpInst;
}
+ // The generic default of 27 is too low for real AMDGPU loops; use a wider
+ // cap that still rejects pathologically large ones.
+ std::optional<unsigned> getMaxMII() const override { return 256; }
+
std::optional<bool> createTripCountGreaterCondition(
int TC, MachineBasicBlock &MBB,
SmallVectorImpl<MachineOperand> &CondParam) override {
diff --git a/llvm/test/CodeGen/AMDGPU/swp-amdgpu-pipeline-max-mii.ll b/llvm/test/CodeGen/AMDGPU/swp-amdgpu-pipeline-max-mii.ll
index f00a50d1d429e..b8a55af1a9941 100644
--- a/llvm/test/CodeGen/AMDGPU/swp-amdgpu-pipeline-max-mii.ll
+++ b/llvm/test/CodeGen/AMDGPU/swp-amdgpu-pipeline-max-mii.ll
@@ -1,12 +1,11 @@
; RUN: llc -mtriple=amdgpu9.50-amd-amdhsa -amdgpu-enable-pipeliner -pass-remarks-analysis=pipeliner %s -filetype=null 2>&1 | FileCheck %s --check-prefix=DEFAULT
-; RUN: llc -mtriple=amdgpu9.50-amd-amdhsa -amdgpu-enable-pipeliner -pipeliner-max-mii=64 -pass-remarks-analysis=pipeliner %s -filetype=null 2>&1 | FileCheck %s --check-prefix=RAISED
-
+; RUN: llc -mtriple=amdgpu9.50-amd-amdhsa -amdgpu-enable-pipeliner -pipeliner-max-mii=27 -pass-remarks-analysis=pipeliner %s -filetype=null 2>&1 | FileCheck %s --check-prefix=CAPPED
; This loop's MII is 32: two independent accumulators each issue one MFMA per
; iteration, and each MFMA holds the XDL pipe for 16 cycles (res=32). That
-; exceeds the generic default cap of 27. At default, pipeliner aborts and
-; only a raised cap lets it schedule.
-; DEFAULT: Minimal Initiation Interval too large: 32 > 27
-; RAISED: Schedule found with Initiation Interval
+; exceeds the generic default cap of 27. Loop pipelines at default but an
+; explicit -pipeliner-max-mii below the MII still rejects it.
+; DEFAULT: Schedule found with Initiation Interval
+; CAPPED: Minimal Initiation Interval too large: 32 > 27
define amdgpu_kernel void @swp_amdgpu_pipeline_max_mii(i32 %arg, ptr addrspace(3) %p) {
bb:
More information about the llvm-branch-commits
mailing list