[llvm] [AMDGPU] Replace amdgpu-num-* attributes with stress options (PR #213924)
Romanov Vlad via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 4 05:40:22 PDT 2026
https://github.com/romanovvlad updated https://github.com/llvm/llvm-project/pull/213924
>From c08092e0b2a32c4180c5ceda7a9bd3b4678c6455 Mon Sep 17 00:00:00 2001
From: Vlad <Vladislav.Romanov at amd.com>
Date: Fri, 31 Jul 2026 06:28:32 -0500
Subject: [PATCH 1/2] [AMDGPU] Replace amdgpu-num-vgpr/amdgpu-num-sgpr
attributes with stress options
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
In https://github.com/llvm/llvm-project/pull/210907, `-amdgpu-stress-*`
options were added to `SIRegisterInfo::getReservedRegs`. However, several
components that check register availability bypass reserved registers
entirely — for example, GCNNSAReassign and RewriteMFMAFormStage call
`getMaxNumVGPRs` or `getAddressableNumArchVGPRs` directly, so the stress
options had no effect on them.
This patch replaces amdgpu-num-* attribute handling with options and
refactors the `getMaxNum*` function hierarchy.
Before:
getMaxNumVGPRs(WavesPerEU) [hardware limit for given occupancy]
|
v
getBaseMaxNumVGPRs(F) [reads amdgpu-num-vgpr attribute, clamps]
|
v
getMaxNumVGPRs(F) [returns total budget as single number]
|
+---> getMaxNumVGPRs(MF) [forwards to F]
|
v
getMaxNumVectorRegs(F) [calls getMaxNumVGPRs(F), then splits
| result into (VGPRs, AGPRs) pair]
v
getReservedRegs(MF) [reserves phys regs above the limit;
| stress options applied HERE separately]
v
getNumAllocatableRegs(VGPR_32) [total physical - reserved; used by
scheduler ExcessLimit -- but NOT used by
RewriteMFMAFormStage or NSA Reassign]
The problem: stress options were applied as a separate override inside
getReservedRegs, on top of the values from getMaxNumVectorRegs. Anything
calling getMaxNumVGPRs or getMaxNumVectorRegs directly (NSA Reassign,
MFMA rewrite cost) never saw them.
After:
getMaxNumVGPRs(WavesPerEU) [hardware limit for given occupancy]
|
v
getMaxNumVectorRegs(F) [PRIMARY: computes occupancy budget,
| splits into (VGPRs, AGPRs) pair,
| applies stress overrides]
|
+---> getMaxNumVGPRs(F) [wrapper: VGPRs + AGPRs on gfx90a,
| | VGPRs only on gfx908/other]
| v
| getMaxNumVGPRs(MF) [forwards to F]
|
+---> getReservedRegs(MF) [reserves phys regs above the limit]
|
v
getNumAllocatableRegs [stress-aware via the chain above;
now also used by RewriteMFMAFormStage]
Now stress overrides are applied inside getMaxNumVectorRegs itself, so
every downstream consumer sees them - both the getMaxNumVGPRs chain and
the getReservedRegs -> getNumAllocatableRegs chain.
getBaseMaxNumVGPRs is removed -- it only existed to apply the attribute.
Additional changes:
- Some callers switched from getMaxNumSGPRs(Function) to
getMaxNumSGPRs(MachineFunction) to use the actual reserved SGPR count
(based on hasFlatScratchInit()) rather than the conservative
subtarget-level estimate (hasFlatAddressSpace()). The Function overload
is removed as it has no remaining callers.
- RewriteMFMAFormStage now uses getNumAllocatableRegs(VGPR_32) instead
of getAddressableNumArchVGPRs(), which always returned the hardware
constant (256) and was never stress-aware.
- Tests that used amdgpu-num-vgpr/amdgpu-num-sgpr attributes are updated
to use the corresponding stress options. Tests with multiple functions
using different per-function limits are split into separate files, since
the stress options are global.
---
llvm/lib/Target/AMDGPU/GCNRegPressure.cpp | 3 +-
llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp | 9 +-
llvm/lib/Target/AMDGPU/GCNSubtarget.cpp | 93 +-
llvm/lib/Target/AMDGPU/GCNSubtarget.h | 50 +-
llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 20 -
llvm/test/CodeGen/AMDGPU/agpr-remat.ll | 4 +-
.../CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll | 131 --
llvm/test/CodeGen/AMDGPU/attr-unparseable.ll | 21 -
llvm/test/CodeGen/AMDGPU/cc-update-stress.ll | 89 ++
llvm/test/CodeGen/AMDGPU/cc-update.ll | 83 --
.../frame-lowering-entry-all-sgpr-used.mir | 4 +-
.../gfx-callable-return-types-stress.ll | 1169 +++++++++++++++++
.../AMDGPU/gfx-callable-return-types.ll | 1166 ----------------
...a-metadata-kernel-code-props-sgpr-spill.ll | 56 +
...-metadata-kernel-code-props-vgpr-spill.ll} | 17 +-
.../AMDGPU/hsa-metadata-kernel-code-props.ll | 130 +-
...ine-scheduler-sink-trivial-remats-attr.mir | 420 ------
...eduler-sink-trivial-remats-sgpr-stress.mir | 330 +++++
...eduler-sink-trivial-remats-vgpr-stress.mir | 101 ++
.../CodeGen/AMDGPU/nsa-reassign-stress.ll | 24 +
llvm/test/CodeGen/AMDGPU/nsa-reassign.ll | 22 -
llvm/test/CodeGen/AMDGPU/nsa-reassign.mir | 8 +-
...al-regcopy-and-spill-missed-at-regalloc.ll | 10 +-
.../pei-build-spill-partial-agpr-vgpr1.mir | 93 ++
.../pei-build-spill-partial-agpr-vgpr3.mir | 109 ++
.../pei-build-spill-partial-agpr-vgpr4.mir | 68 +
.../pei-build-spill-partial-agpr-vgpr5.mir | 147 +++
.../AMDGPU/pei-build-spill-partial-agpr.mir | 399 ------
.../AMDGPU/preserve-wwm-copy-dst-reg.ll | 6 +-
.../AMDGPU/schedule-amdgpu-trackers-stress.ll | 38 +
.../AMDGPU/schedule-amdgpu-trackers.ll | 34 -
.../schedule-regpressure-limit-clustering.ll | 5 +-
.../si-lower-sgpr-spills-cycle-header.ll | 4 +-
llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr10.ll | 81 ++
llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr12.ll | 135 ++
llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr32.ll | 186 +++
llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr6.ll | 130 ++
llvm/test/CodeGen/AMDGPU/spill-agpr.ll | 493 -------
.../AMDGPU/spill-offset-calculation-sgpr16.ll | 90 ++
.../AMDGPU/spill-offset-calculation-sgpr18.ll | 91 ++
.../AMDGPU/spill-offset-calculation.ll | 166 ---
.../AMDGPU/spill-sgpr-stack-no-sgpr.ll | 4 +-
.../CodeGen/AMDGPU/spill-vector-superclass.ll | 4 +-
.../CodeGen/AMDGPU/spill-vgpr-stress10.ll | 27 +
.../CodeGen/AMDGPU/spill-vgpr-stress11.ll | 56 +
.../test/CodeGen/AMDGPU/spill-vgpr-to-agpr.ll | 4 +-
llvm/test/CodeGen/AMDGPU/spill-vgpr.ll | 74 --
llvm/test/CodeGen/AMDGPU/splitkit.mir | 10 +-
.../AMDGPU/undefined-physreg-sgpr-spill.mir | 4 +-
.../vgpr-mark-last-scratch-load-stress11.ll | 214 +++
.../vgpr-mark-last-scratch-load-stress6.ll | 69 +
.../AMDGPU/vgpr-mark-last-scratch-load.ll | 280 ----
.../AMDGPU/vgpr-mark-last-scratch-load.mir | 10 +-
.../AMDGPU/whole-wave-register-copy.ll | 4 +-
.../AMDGPU/whole-wave-register-spill.ll | 6 +-
55 files changed, 3402 insertions(+), 3599 deletions(-)
delete mode 100644 llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/cc-update-stress.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/gfx-callable-return-types-stress.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-sgpr-spill.ll
rename llvm/test/CodeGen/AMDGPU/{attr-amdgpu-num-vgpr.ll => hsa-metadata-kernel-code-props-vgpr-spill.ll} (78%)
create mode 100644 llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-sgpr-stress.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-vgpr-stress.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/nsa-reassign-stress.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr1.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr3.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr4.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr5.mir
delete mode 100644 llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers-stress.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr10.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr12.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr32.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr6.ll
delete mode 100644 llvm/test/CodeGen/AMDGPU/spill-agpr.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr16.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr18.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-vgpr-stress10.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/spill-vgpr-stress11.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress11.ll
create mode 100644 llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress6.ll
diff --git a/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp b/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
index 53617e89af757..8befd1c614b1e 100644
--- a/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNRegPressure.cpp
@@ -373,9 +373,8 @@ static LaneBitmask findUseBetween(unsigned Reg, LaneBitmask LastUseMask,
GCNRPTarget::GCNRPTarget(const MachineFunction &MF, const GCNRegPressure &RP)
: GCNRPTarget(RP, MF) {
- const Function &F = MF.getFunction();
const GCNSubtarget &ST = MF.getSubtarget<GCNSubtarget>();
- setTarget(ST.getMaxNumSGPRs(F), ST.getMaxNumVGPRs(F));
+ setTarget(ST.getMaxNumSGPRs(MF), ST.getMaxNumVGPRs(MF));
}
GCNRPTarget::GCNRPTarget(unsigned NumSGPRs, unsigned NumVGPRs,
diff --git a/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp b/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp
index 0ac656a3e02a7..256454b793b34 100644
--- a/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNSchedStrategy.cpp
@@ -1389,7 +1389,8 @@ bool RewriteMFMAFormStage::initGCNSchedStage() {
RegionsWithExcessArchVGPR.reset();
for (unsigned Region = 0; Region < DAG.Regions.size(); Region++) {
GCNRegPressure PressureBefore = DAG.Pressure[Region];
- if (PressureBefore.getArchVGPRNum() > ST.getAddressableNumArchVGPRs())
+ if (PressureBefore.getArchVGPRNum() >
+ DAG.RegClassInfo->getNumAllocatableRegs(&AMDGPU::VGPR_32RegClass))
RegionsWithExcessArchVGPR[Region] = true;
}
@@ -2959,11 +2960,9 @@ unsigned PreRARematStage::getStageTargetOccupancy() const {
}
bool PreRARematStage::setObjective() {
- const Function &F = MF.getFunction();
-
// Set up "spilling targets" for all regions.
- unsigned MaxSGPRs = ST.getMaxNumSGPRs(F);
- unsigned MaxVGPRs = ST.getMaxNumVGPRs(F);
+ unsigned MaxSGPRs = ST.getMaxNumSGPRs(MF);
+ unsigned MaxVGPRs = ST.getMaxNumVGPRs(MF);
bool HasVectorRegisterExcess = false;
for (unsigned I = 0, E = DAG.Regions.size(); I != E; ++I) {
const GCNRegPressure &RP = DAG.Pressure[I];
diff --git a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
index 1c0e718bd8d97..c47565ef9b1b7 100644
--- a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
@@ -53,6 +53,18 @@ static cl::opt<unsigned>
cl::desc("Number of addresses from which to enable MIMG NSA."),
cl::init(2), cl::Hidden);
+static cl::opt<unsigned> StressVGPRLimit(
+ "amdgpu-stress-vgpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit VGPRs to N arch registers"));
+
+static cl::opt<unsigned> StressAGPRLimit(
+ "amdgpu-stress-agpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit AGPRs to N registers"));
+
+static cl::opt<unsigned> StressSGPRLimit(
+ "amdgpu-stress-sgpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit SGPRs to N registers"));
+
GCNSubtarget::~GCNSubtarget() = default;
static AMDGPUSubtarget::Generation computeDefaultGeneration(const Triple &TT) {
@@ -538,38 +550,9 @@ unsigned GCNSubtarget::getBaseMaxNumSGPRs(
unsigned MaxNumSGPRs = getMaxNumSGPRs(WavesPerEU.first, false);
unsigned MaxAddressableNumSGPRs = getMaxNumSGPRs(WavesPerEU.first, true);
- // Check if maximum number of SGPRs was explicitly requested using
- // "amdgpu-num-sgpr" attribute.
- unsigned Requested =
- F.getFnAttributeAsParsedInteger("amdgpu-num-sgpr", MaxNumSGPRs);
-
- if (Requested != MaxNumSGPRs) {
- // Make sure requested value does not violate subtarget's specifications.
- if (Requested && (Requested <= ReservedNumSGPRs))
- Requested = 0;
-
- // If more SGPRs are required to support the input user/system SGPRs,
- // increase to accommodate them.
- //
- // FIXME: This really ends up using the requested number of SGPRs + number
- // of reserved special registers in total. Theoretically you could re-use
- // the last input registers for these special registers, but this would
- // require a lot of complexity to deal with the weird aliasing.
- unsigned InputNumSGPRs = PreloadedSGPRs;
- if (Requested && Requested < InputNumSGPRs)
- Requested = InputNumSGPRs;
-
- // Make sure requested value is compatible with values implied by
- // default/requested minimum/maximum number of waves per execution unit.
- if (Requested && Requested > getMaxNumSGPRs(WavesPerEU.first, false))
- Requested = 0;
- if (WavesPerEU.second && Requested &&
- Requested < getMinNumSGPRs(WavesPerEU.second))
- Requested = 0;
-
- if (Requested)
- MaxNumSGPRs = Requested;
- }
+ // Stress test: override SGPR limit.
+ if (StressSGPRLimit.getNumOccurrences())
+ MaxNumSGPRs = StressSGPRLimit;
if (hasSGPRInitBug())
MaxNumSGPRs = AMDGPU::IsaInfo::FIXED_NUM_SGPRS_FOR_INIT_BUG;
@@ -609,27 +592,19 @@ unsigned GCNSubtarget::getMaxNumPreloadedSGPRs() const {
return MaxUserSGPRs + MaxSystemSGPRs + SyntheticSGPRs;
}
-unsigned GCNSubtarget::getMaxNumSGPRs(const Function &F) const {
- return getBaseMaxNumSGPRs(F, getWavesPerEU(F), getMaxNumPreloadedSGPRs(),
- getReservedNumSGPRs(F));
+unsigned GCNSubtarget::getMaxNumVGPRs(const Function &F) const {
+ auto [VGPRs, AGPRs] = getMaxNumVectorRegs(F);
+ // On gfx90a+ VGPRs and AGPRs share a unified register file.
+ // On gfx908 they are independent, so only VGPRs count toward the budget.
+ return hasGFX90AInsts() ? VGPRs + AGPRs : VGPRs;
}
-unsigned GCNSubtarget::getBaseMaxNumVGPRs(
- const Function &F, std::pair<unsigned, unsigned> NumVGPRBounds) const {
- const auto [Min, Max] = NumVGPRBounds;
-
- // Check if maximum number of VGPRs was explicitly requested using
- // "amdgpu-num-vgpr" attribute.
-
- unsigned Requested = F.getFnAttributeAsParsedInteger("amdgpu-num-vgpr", Max);
- if (Requested != Max && hasGFX90AInsts())
- Requested *= 2;
-
- // Make sure requested value is inside the range of possible VGPR usage.
- return std::clamp(Requested, Min, Max);
+unsigned GCNSubtarget::getMaxNumVGPRs(const MachineFunction &MF) const {
+ return getMaxNumVGPRs(MF.getFunction());
}
-unsigned GCNSubtarget::getMaxNumVGPRs(const Function &F) const {
+std::pair<unsigned, unsigned>
+GCNSubtarget::getMaxNumVectorRegs(const Function &F) const {
// Temporarily check both the attribute and the subtarget feature, until the
// latter is removed.
unsigned DynamicVGPRBlockSize = AMDGPU::getDynamicVGPRBlockSize(F);
@@ -637,18 +612,8 @@ unsigned GCNSubtarget::getMaxNumVGPRs(const Function &F) const {
DynamicVGPRBlockSize = getDynamicVGPRBlockSize();
std::pair<unsigned, unsigned> Waves = getWavesPerEU(F);
- return getBaseMaxNumVGPRs(
- F, {getMinNumVGPRs(Waves.second, DynamicVGPRBlockSize),
- getMaxNumVGPRs(Waves.first, DynamicVGPRBlockSize)});
-}
-
-unsigned GCNSubtarget::getMaxNumVGPRs(const MachineFunction &MF) const {
- return getMaxNumVGPRs(MF.getFunction());
-}
-
-std::pair<unsigned, unsigned>
-GCNSubtarget::getMaxNumVectorRegs(const Function &F) const {
- const unsigned MaxVectorRegs = getMaxNumVGPRs(F);
+ const unsigned MaxVectorRegs = getMaxNumVGPRs(Waves.first,
+ DynamicVGPRBlockSize);
unsigned MaxNumVGPRs = MaxVectorRegs;
unsigned MaxNumAGPRs = 0;
@@ -700,6 +665,12 @@ GCNSubtarget::getMaxNumVectorRegs(const Function &F) const {
MaxNumAGPRs = MaxNumVGPRs = MaxVectorRegs;
}
+ // Stress test: override VGPR/AGPR limits.
+ if (StressVGPRLimit.getNumOccurrences())
+ MaxNumVGPRs = StressVGPRLimit;
+ if (StressAGPRLimit.getNumOccurrences())
+ MaxNumAGPRs = StressAGPRLimit;
+
return std::pair(MaxNumVGPRs, MaxNumAGPRs);
}
diff --git a/llvm/lib/Target/AMDGPU/GCNSubtarget.h b/llvm/lib/Target/AMDGPU/GCNSubtarget.h
index c42ca8e19ef9c..8545961017f54 100644
--- a/llvm/lib/Target/AMDGPU/GCNSubtarget.h
+++ b/llvm/lib/Target/AMDGPU/GCNSubtarget.h
@@ -811,25 +811,9 @@ class GCNSubtarget final : public AMDGPUGenSubtargetInfo,
unsigned ReservedNumSGPRs) const;
/// \returns Maximum number of SGPRs that meets number of waves per execution
- /// unit requirement for function \p MF, or number of SGPRs explicitly
- /// requested using "amdgpu-num-sgpr" attribute attached to function \p MF.
- ///
- /// \returns Value that meets number of waves per execution unit requirement
- /// if explicitly requested value cannot be converted to integer, violates
- /// subtarget's specifications, or does not meet number of waves per execution
- /// unit requirement.
+ /// unit requirement for function \p MF.
unsigned getMaxNumSGPRs(const MachineFunction &MF) const;
- /// \returns Maximum number of SGPRs that meets number of waves per execution
- /// unit requirement for function \p F, or number of SGPRs explicitly
- /// requested using "amdgpu-num-sgpr" attribute attached to function \p F.
- ///
- /// \returns Value that meets number of waves per execution unit requirement
- /// if explicitly requested value cannot be converted to integer, violates
- /// subtarget's specifications, or does not meet number of waves per execution
- /// unit requirement.
- unsigned getMaxNumSGPRs(const Function &F) const;
-
/// \returns VGPR allocation granularity supported by the subtarget.
unsigned getVGPRAllocGranule(unsigned DynamicVGPRBlockSize) const {
return AMDGPU::IsaInfo::getVGPRAllocGranule(*this, DynamicVGPRBlockSize);
@@ -872,36 +856,14 @@ class GCNSubtarget final : public AMDGPUGenSubtargetInfo,
DynamicVGPRBlockSize);
}
- /// \returns max num VGPRs. This is the common utility function
- /// called by MachineFunction and Function variants of getMaxNumVGPRs.
- unsigned
- getBaseMaxNumVGPRs(const Function &F,
- std::pair<unsigned, unsigned> NumVGPRBounds) const;
-
- /// \returns Maximum number of VGPRs that meets number of waves per execution
- /// unit requirement for function \p F, or number of VGPRs explicitly
- /// requested using "amdgpu-num-vgpr" attribute attached to function \p F.
- ///
- /// \returns Value that meets number of waves per execution unit requirement
- /// if explicitly requested value cannot be converted to integer, violates
- /// subtarget's specifications, or does not meet number of waves per execution
- /// unit requirement.
- unsigned getMaxNumVGPRs(const Function &F) const;
-
- unsigned getMaxNumAGPRs(const Function &F) const { return getMaxNumVGPRs(F); }
-
/// Return a pair of maximum numbers of VGPRs and AGPRs that meet the number
- /// of waves per execution unit required for the function \p MF.
+ /// of waves per execution unit required for the function \p F.
std::pair<unsigned, unsigned> getMaxNumVectorRegs(const Function &F) const;
- /// \returns Maximum number of VGPRs that meets number of waves per execution
- /// unit requirement for function \p MF, or number of VGPRs explicitly
- /// requested using "amdgpu-num-vgpr" attribute attached to function \p MF.
- ///
- /// \returns Value that meets number of waves per execution unit requirement
- /// if explicitly requested value cannot be converted to integer, violates
- /// subtarget's specifications, or does not meet number of waves per execution
- /// unit requirement.
+ /// \returns Total vector register budget (VGPRs + AGPRs) for function \p F.
+ unsigned getMaxNumVGPRs(const Function &F) const;
+
+ /// \returns Total vector register budget (VGPRs + AGPRs) for function \p MF.
unsigned getMaxNumVGPRs(const MachineFunction &MF) const;
bool isWave32() const { return getWavefrontSize() == 32; }
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index 01220a701a453..3bc24f2c363b3 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -40,18 +40,6 @@ static cl::opt<bool> EnableSpillCFISavedRegs(
cl::desc("Enable spilling the registers required for CFI emission"),
cl::ReallyHidden, cl::init(false), cl::ZeroOrMore);
-static cl::opt<unsigned> StressVGPRLimit(
- "amdgpu-stress-vgpr", cl::Hidden, cl::init(0),
- cl::desc("Limit VGPRs to N registers by reserving the rest"));
-
-static cl::opt<unsigned> StressAGPRLimit(
- "amdgpu-stress-agpr", cl::Hidden, cl::init(0),
- cl::desc("Limit AGPRs to N registers by reserving the rest"));
-
-static cl::opt<unsigned> StressSGPRLimit(
- "amdgpu-stress-sgpr", cl::Hidden, cl::init(0),
- cl::desc("Limit SGPRs to N registers by reserving the rest"));
-
std::array<std::vector<int16_t>, 32> SIRegisterInfo::RegSplitParts;
std::array<std::array<uint16_t, 32>, 9> SIRegisterInfo::SubRegFromChannelTable;
@@ -656,8 +644,6 @@ BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const {
// Reserve SGPRs.
//
unsigned MaxNumSGPRs = ST.getMaxNumSGPRs(MF);
- if (StressSGPRLimit.getNumOccurrences() && StressSGPRLimit < MaxNumSGPRs)
- MaxNumSGPRs = StressSGPRLimit;
unsigned TotalNumSGPRs = AMDGPU::SGPR_32RegClass.getNumRegs();
for (const TargetRegisterClass &RC : regclasses()) {
if (RC.isBaseClass() && isSGPRClass(&RC)) {
@@ -715,12 +701,6 @@ BitVector SIRegisterInfo::getReservedRegs(const MachineFunction &MF) const {
//
auto [MaxNumVGPRs, MaxNumAGPRs] = ST.getMaxNumVectorRegs(MF.getFunction());
- // Stress test: override VGPR/AGPR limits.
- if (StressVGPRLimit.getNumOccurrences() && StressVGPRLimit < MaxNumVGPRs)
- MaxNumVGPRs = StressVGPRLimit;
- if (StressAGPRLimit.getNumOccurrences() && StressAGPRLimit < MaxNumAGPRs)
- MaxNumAGPRs = StressAGPRLimit;
-
for (const TargetRegisterClass &RC : regclasses()) {
if (RC.isBaseClass() && isVGPRClass(&RC)) {
unsigned NumRegs = divideCeil(getRegSizeInBits(RC), 32);
diff --git a/llvm/test/CodeGen/AMDGPU/agpr-remat.ll b/llvm/test/CodeGen/AMDGPU/agpr-remat.ll
index ede7ce390e159..79956b7ac4629 100644
--- a/llvm/test/CodeGen/AMDGPU/agpr-remat.ll
+++ b/llvm/test/CodeGen/AMDGPU/agpr-remat.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgpu9.08 < %s | FileCheck -enable-var-scope -check-prefixes=GFX908 %s
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=8 -amdgpu-stress-agpr=8 < %s | FileCheck -enable-var-scope -check-prefixes=GFX908 %s
; Make sure there are no v_accvgpr_read_b32 copying back and forth
; between AGPR and VGPR.
@@ -48,4 +48,4 @@ define void @remat_regcopy_avoids_spill(i32 %v0, i32 %v1, i32 %v2, i32 %v3, i32
ret void
}
-attributes #1 = { nounwind "amdgpu-num-vgpr"="8" }
+attributes #1 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll b/llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll
deleted file mode 100644
index d3e56054a7128..0000000000000
--- a/llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-sgpr.ll
+++ /dev/null
@@ -1,131 +0,0 @@
-; RUN: llc -mtriple=amdgpu8.03--amdhsa < %s | FileCheck -check-prefix=ALL %s
-
-; FIXME: Vectorization can increase required SGPR count beyond limit.
-
-; ALL-LABEL: {{^}}max_10_sgprs:
-
-; ALL: SGPRBlocks: 2
-; ALL: NumSGPRsForWavesPerEU: 24
-define amdgpu_kernel void @max_10_sgprs() #0 {
- %one = load volatile i32, ptr addrspace(4) poison
- %two = load volatile i32, ptr addrspace(4) poison
- %three = load volatile i32, ptr addrspace(4) poison
- %four = load volatile i32, ptr addrspace(4) poison
- %five = load volatile i32, ptr addrspace(4) poison
- %six = load volatile i32, ptr addrspace(4) poison
- %seven = load volatile i32, ptr addrspace(4) poison
- %eight = load volatile i32, ptr addrspace(4) poison
- %nine = load volatile i32, ptr addrspace(4) poison
- %ten = load volatile i32, ptr addrspace(4) poison
- %eleven = load volatile i32, ptr addrspace(4) poison
- call void asm sideeffect "", "s,s,s,s,s,s,s,s,s,s"(i32 %one, i32 %two, i32 %three, i32 %four, i32 %five, i32 %six, i32 %seven, i32 %eight, i32 %nine, i32 %ten)
- store volatile i32 %one, ptr addrspace(1) poison
- store volatile i32 %two, ptr addrspace(1) poison
- store volatile i32 %three, ptr addrspace(1) poison
- store volatile i32 %four, ptr addrspace(1) poison
- store volatile i32 %five, ptr addrspace(1) poison
- store volatile i32 %six, ptr addrspace(1) poison
- store volatile i32 %seven, ptr addrspace(1) poison
- store volatile i32 %eight, ptr addrspace(1) poison
- store volatile i32 %nine, ptr addrspace(1) poison
- store volatile i32 %ten, ptr addrspace(1) poison
- store volatile i32 %eleven, ptr addrspace(1) poison
- ret void
-}
-
-; private resource: 4
-; scratch wave offset: 1
-; workgroup ids: 3
-; dispatch id: 2
-; queue ptr: 2
-; flat scratch init: 2
-; ---------------------
-; total: 14
-
-; + reserved vcc = 16
-
-; Because we can't handle re-using the last few input registers as the
-; special vcc etc. registers (as well as decide to not use the unused
-; features when the number of registers is frozen), this ends up using
-; more than expected.
-
-; XALL-LABEL: {{^}}max_12_sgprs_14_input_sgprs:
-; XTOSGPR: SGPRBlocks: 1
-; XTOSGPR: NumSGPRsForWavesPerEU: 16
-
-; This test case is disabled: When calculating the spillslot addresses AMDGPU
-; creates an extra vreg to save/restore m0 which in a point of maximum register
-; pressure would trigger an endless loop; the compiler aborts earlier with
-; "Incomplete scavenging after 2nd pass" in practice.
-;define amdgpu_kernel void @max_12_sgprs_14_input_sgprs(ptr addrspace(1) %out1,
-; ptr addrspace(1) %out2,
-; ptr addrspace(1) %out3,
-; ptr addrspace(1) %out4,
-; i32 %one, i32 %two, i32 %three, i32 %four) #2 {
-; %x.0 = call i32 @llvm.amdgcn.workgroup.id.x()
-; %x.1 = call i32 @llvm.amdgcn.workgroup.id.y()
-; %x.2 = call i32 @llvm.amdgcn.workgroup.id.z()
-; %x.3 = call i64 @llvm.amdgcn.dispatch.id()
-; %x.4 = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
-; %x.5 = call ptr addrspace(4) @llvm.amdgcn.queue.ptr()
-; store volatile i32 0, ptr poison
-; br label %stores
-;
-;stores:
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; store volatile i64 %x.3, ptr addrspace(1) poison
-; store volatile ptr addrspace(4) %x.4, ptr addrspace(1) poison
-; store volatile ptr addrspace(4) %x.5, ptr addrspace(1) poison
-;
-; store i32 %one, ptr addrspace(1) %out1
-; store i32 %two, ptr addrspace(1) %out2
-; store i32 %three, ptr addrspace(1) %out3
-; store i32 %four, ptr addrspace(1) %out4
-; ret void
-;}
-
-; The following test is commented out for now; http://llvm.org/PR31230
-; XALL-LABEL: max_12_sgprs_12_input_sgprs{{$}}
-; ; Make sure copies for input buffer are not clobbered. This requires
-; ; swapping the order the registers are copied from what normally
-; ; happens.
-
-; XALL: SGPRBlocks: 2
-; XALL: NumSGPRsForWavesPerEU: 18
-;define amdgpu_kernel void @max_12_sgprs_12_input_sgprs(ptr addrspace(1) %out1,
-; ptr addrspace(1) %out2,
-; ptr addrspace(1) %out3,
-; ptr addrspace(1) %out4,
-; i32 %one, i32 %two, i32 %three, i32 %four) #2 {
-; store volatile i32 0, ptr poison
-; %x.0 = call i32 @llvm.amdgcn.workgroup.id.x()
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; %x.1 = call i32 @llvm.amdgcn.workgroup.id.y()
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; %x.2 = call i32 @llvm.amdgcn.workgroup.id.z()
-; store volatile i32 %x.0, ptr addrspace(1) poison
-; %x.3 = call i64 @llvm.amdgcn.dispatch.id()
-; store volatile i64 %x.3, ptr addrspace(1) poison
-; %x.4 = call ptr addrspace(4) @llvm.amdgcn.dispatch.ptr()
-; store volatile ptr addrspace(4) %x.4, ptr addrspace(1) poison
-;
-; store i32 %one, ptr addrspace(1) %out1
-; store i32 %two, ptr addrspace(1) %out2
-; store i32 %three, ptr addrspace(1) %out3
-; store i32 %four, ptr addrspace(1) %out4
-; ret void
-;}
-
-declare i32 @llvm.amdgcn.workgroup.id.x() #1
-declare i32 @llvm.amdgcn.workgroup.id.y() #1
-declare i32 @llvm.amdgcn.workgroup.id.z() #1
-declare i64 @llvm.amdgcn.dispatch.id() #1
-declare ptr addrspace(4) @llvm.amdgcn.dispatch.ptr() #1
-declare ptr addrspace(4) @llvm.amdgcn.queue.ptr() #1
-
-attributes #0 = { nounwind "amdgpu-num-sgpr"="14" }
-attributes #1 = { nounwind readnone }
-attributes #2 = { nounwind "amdgpu-num-sgpr"="12" }
-attributes #3 = { nounwind "amdgpu-num-sgpr"="11" }
diff --git a/llvm/test/CodeGen/AMDGPU/attr-unparseable.ll b/llvm/test/CodeGen/AMDGPU/attr-unparseable.ll
index 4732fe5e0ab5f..c740d2cc53e0e 100644
--- a/llvm/test/CodeGen/AMDGPU/attr-unparseable.ll
+++ b/llvm/test/CodeGen/AMDGPU/attr-unparseable.ll
@@ -1,26 +1,5 @@
; RUN: not llc -mtriple=amdgpu8.03--amdhsa < %s 2>&1 | FileCheck %s
-; CHECK: cannot parse integer attribute amdgpu-num-sgpr
-define amdgpu_kernel void @unparseable_single_0() #0 {
-entry:
- ret void
-}
-attributes #0 = {"amdgpu-num-sgpr"}
-
-; CHECK: cannot parse integer attribute amdgpu-num-sgpr
-define amdgpu_kernel void @unparseable_single_1() #1 {
-entry:
- ret void
-}
-attributes #1 = {"amdgpu-num-sgpr"="k"}
-
-; CHECK: cannot parse integer attribute amdgpu-num-sgpr
-define amdgpu_kernel void @unparseable_single_2() #2 {
-entry:
- ret void
-}
-attributes #2 = {"amdgpu-num-sgpr"="1,2"}
-
; CHECK: can't parse first integer attribute amdgpu-flat-work-group-size
define amdgpu_kernel void @unparseable_pair_0() #3 {
entry:
diff --git a/llvm/test/CodeGen/AMDGPU/cc-update-stress.ll b/llvm/test/CodeGen/AMDGPU/cc-update-stress.ll
new file mode 100644
index 0000000000000..5f4d2741b79aa
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/cc-update-stress.ll
@@ -0,0 +1,89 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=amdgpu8.03-amd-amdhsa -amdgpu-stress-vgpr=8 < %s | FileCheck --check-prefix=GFX803 %s
+; RUN: llc -mtriple=amdgpu9.00-amd-amdhsa -amdgpu-stress-vgpr=8 < %s | FileCheck --check-prefix=GFX900 %s
+; RUN: llc -mtriple=amdgpu10.10-amd-amdhsa -amdgpu-stress-vgpr=8 < %s | FileCheck --check-prefix=GFX1010 %s
+; RUN: llc -mtriple=amdgpu11.00-amd-amdhsa -amdgpu-enable-delay-alu=0 -amdgpu-stress-vgpr=8 < %s | FileCheck --check-prefix=GFX1100 %s
+
+define amdgpu_kernel void @test_sgpr_offset_kernel() #1 {
+; GFX803-LABEL: test_sgpr_offset_kernel:
+; GFX803: ; %bb.0: ; %entry
+; GFX803-NEXT: s_add_u32 s0, s0, s17
+; GFX803-NEXT: s_addc_u32 s1, s1, 0
+; GFX803-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
+; GFX803-NEXT: s_waitcnt vmcnt(0)
+; GFX803-NEXT: s_mov_b32 s4, 0x40000
+; GFX803-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
+; GFX803-NEXT: ;;#ASMSTART
+; GFX803-NEXT: ;;#ASMEND
+; GFX803-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
+; GFX803-NEXT: s_waitcnt vmcnt(0)
+; GFX803-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
+; GFX803-NEXT: s_waitcnt vmcnt(0)
+; GFX803-NEXT: s_endpgm
+;
+; GFX900-LABEL: test_sgpr_offset_kernel:
+; GFX900: ; %bb.0: ; %entry
+; GFX900-NEXT: s_add_u32 s0, s0, s17
+; GFX900-NEXT: s_addc_u32 s1, s1, 0
+; GFX900-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
+; GFX900-NEXT: s_waitcnt vmcnt(0)
+; GFX900-NEXT: s_mov_b32 s4, 0x40000
+; GFX900-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
+; GFX900-NEXT: ;;#ASMSTART
+; GFX900-NEXT: ;;#ASMEND
+; GFX900-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
+; GFX900-NEXT: s_waitcnt vmcnt(0)
+; GFX900-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
+; GFX900-NEXT: s_waitcnt vmcnt(0)
+; GFX900-NEXT: s_endpgm
+;
+; GFX1010-LABEL: test_sgpr_offset_kernel:
+; GFX1010: ; %bb.0: ; %entry
+; GFX1010-NEXT: s_add_u32 s0, s0, s17
+; GFX1010-NEXT: s_addc_u32 s1, s1, 0
+; GFX1010-NEXT: s_mov_b32 s4, 0x20000
+; GFX1010-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc dlc
+; GFX1010-NEXT: s_waitcnt vmcnt(0)
+; GFX1010-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
+; GFX1010-NEXT: ;;#ASMSTART
+; GFX1010-NEXT: ;;#ASMEND
+; GFX1010-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
+; GFX1010-NEXT: s_waitcnt vmcnt(0)
+; GFX1010-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
+; GFX1010-NEXT: s_waitcnt_vscnt null, 0x0
+; GFX1010-NEXT: s_endpgm
+;
+; GFX1100-LABEL: test_sgpr_offset_kernel:
+; GFX1100: ; %bb.0: ; %entry
+; GFX1100-NEXT: scratch_load_b32 v0, off, off offset:8 glc dlc
+; GFX1100-NEXT: s_waitcnt vmcnt(0)
+; GFX1100-NEXT: s_movk_i32 s0, 0x1000
+; GFX1100-NEXT: scratch_store_b32 off, v0, s0 ; 4-byte Folded Spill
+; GFX1100-NEXT: ;;#ASMSTART
+; GFX1100-NEXT: ;;#ASMEND
+; GFX1100-NEXT: scratch_load_b32 v0, off, s0 ; 4-byte Folded Reload
+; GFX1100-NEXT: s_waitcnt vmcnt(0)
+; GFX1100-NEXT: scratch_store_b32 off, v0, off offset:8 dlc
+; GFX1100-NEXT: s_waitcnt_vscnt null, 0x0
+; GFX1100-NEXT: s_endpgm
+entry:
+ ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
+ ; fit in the instruction, and has to live in the SGPR offset.
+ %alloca = alloca i8, i32 4092, align 4, addrspace(5)
+
+ %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
+ ; 0x40000 / 64 = 4096 (for wave64)
+ ; CHECK: s_add_u32 s6, s7, 0x40000
+ ; CHECK: buffer_store_dword v{{[0-9]+}}, off, s[{{[0-9]+:[0-9]+}}], s6 ; 4-byte Folded Spill
+ %a = load volatile i32, ptr addrspace(5) %aptr
+
+ ; Force %a to spill
+ call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}" ()
+
+ %outptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
+ store volatile i32 %a, ptr addrspace(5) %outptr
+
+ ret void
+}
+
+attributes #1 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/cc-update.ll b/llvm/test/CodeGen/AMDGPU/cc-update.ll
index 1bc5423f50e84..f86792876e55c 100644
--- a/llvm/test/CodeGen/AMDGPU/cc-update.ll
+++ b/llvm/test/CodeGen/AMDGPU/cc-update.ll
@@ -522,91 +522,8 @@ entry:
ret void
}
-define amdgpu_kernel void @test_sgpr_offset_kernel() #1 {
-; GFX803-LABEL: test_sgpr_offset_kernel:
-; GFX803: ; %bb.0: ; %entry
-; GFX803-NEXT: s_add_u32 s0, s0, s17
-; GFX803-NEXT: s_addc_u32 s1, s1, 0
-; GFX803-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
-; GFX803-NEXT: s_waitcnt vmcnt(0)
-; GFX803-NEXT: s_mov_b32 s4, 0x40000
-; GFX803-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
-; GFX803-NEXT: ;;#ASMSTART
-; GFX803-NEXT: ;;#ASMEND
-; GFX803-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
-; GFX803-NEXT: s_waitcnt vmcnt(0)
-; GFX803-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
-; GFX803-NEXT: s_waitcnt vmcnt(0)
-; GFX803-NEXT: s_endpgm
-;
-; GFX900-LABEL: test_sgpr_offset_kernel:
-; GFX900: ; %bb.0: ; %entry
-; GFX900-NEXT: s_add_u32 s0, s0, s17
-; GFX900-NEXT: s_addc_u32 s1, s1, 0
-; GFX900-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
-; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: s_mov_b32 s4, 0x40000
-; GFX900-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
-; GFX900-NEXT: ;;#ASMSTART
-; GFX900-NEXT: ;;#ASMEND
-; GFX900-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
-; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
-; GFX900-NEXT: s_waitcnt vmcnt(0)
-; GFX900-NEXT: s_endpgm
-;
-; GFX1010-LABEL: test_sgpr_offset_kernel:
-; GFX1010: ; %bb.0: ; %entry
-; GFX1010-NEXT: s_add_u32 s0, s0, s17
-; GFX1010-NEXT: s_addc_u32 s1, s1, 0
-; GFX1010-NEXT: s_mov_b32 s4, 0x20000
-; GFX1010-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc dlc
-; GFX1010-NEXT: s_waitcnt vmcnt(0)
-; GFX1010-NEXT: buffer_store_dword v0, off, s[0:3], s4 ; 4-byte Folded Spill
-; GFX1010-NEXT: ;;#ASMSTART
-; GFX1010-NEXT: ;;#ASMEND
-; GFX1010-NEXT: buffer_load_dword v0, off, s[0:3], s4 ; 4-byte Folded Reload
-; GFX1010-NEXT: s_waitcnt vmcnt(0)
-; GFX1010-NEXT: buffer_store_dword v0, off, s[0:3], 0 offset:8
-; GFX1010-NEXT: s_waitcnt_vscnt null, 0x0
-; GFX1010-NEXT: s_endpgm
-;
-; GFX1100-LABEL: test_sgpr_offset_kernel:
-; GFX1100: ; %bb.0: ; %entry
-; GFX1100-NEXT: scratch_load_b32 v0, off, off offset:8 glc dlc
-; GFX1100-NEXT: s_waitcnt vmcnt(0)
-; GFX1100-NEXT: s_movk_i32 s0, 0x1000
-; GFX1100-NEXT: scratch_store_b32 off, v0, s0 ; 4-byte Folded Spill
-; GFX1100-NEXT: ;;#ASMSTART
-; GFX1100-NEXT: ;;#ASMEND
-; GFX1100-NEXT: scratch_load_b32 v0, off, s0 ; 4-byte Folded Reload
-; GFX1100-NEXT: s_waitcnt vmcnt(0)
-; GFX1100-NEXT: scratch_store_b32 off, v0, off offset:8 dlc
-; GFX1100-NEXT: s_waitcnt_vscnt null, 0x0
-; GFX1100-NEXT: s_endpgm
-entry:
- ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
- ; fit in the instruction, and has to live in the SGPR offset.
- %alloca = alloca i8, i32 4092, align 4, addrspace(5)
-
- %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
- ; 0x40000 / 64 = 4096 (for wave64)
- ; CHECK: s_add_u32 s6, s7, 0x40000
- ; CHECK: buffer_store_dword v{{[0-9]+}}, off, s[{{[0-9]+:[0-9]+}}], s6 ; 4-byte Folded Spill
- %a = load volatile i32, ptr addrspace(5) %aptr
-
- ; Force %a to spill
- call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}" ()
-
- %outptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
- store volatile i32 %a, ptr addrspace(5) %outptr
-
- ret void
-}
-
declare hidden void @ex() local_unnamed_addr #0
attributes #0 = { nounwind }
-attributes #1 = { nounwind "amdgpu-num-vgpr"="8" }
attributes #2 = { nounwind "frame-pointer"="all" }
diff --git a/llvm/test/CodeGen/AMDGPU/frame-lowering-entry-all-sgpr-used.mir b/llvm/test/CodeGen/AMDGPU/frame-lowering-entry-all-sgpr-used.mir
index 0cd7b6d37fe2f..d40b387b752fe 100644
--- a/llvm/test/CodeGen/AMDGPU/frame-lowering-entry-all-sgpr-used.mir
+++ b/llvm/test/CodeGen/AMDGPU/frame-lowering-entry-all-sgpr-used.mir
@@ -1,4 +1,4 @@
-# RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -verify-machineinstrs -run-pass=prolog-epilog %s -o - | FileCheck %s
+# RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -amdgpu-stress-sgpr=8 -verify-machineinstrs -run-pass=prolog-epilog %s -o - | FileCheck %s
# CHECK-LABEL: all_sgpr_used
# CHECK: V_CMP_LT_U32_e64
@@ -6,7 +6,7 @@
define amdgpu_kernel void @all_sgpr_used() #0 {
ret void
}
- attributes #0 = { "amdgpu-num-sgpr"="8" "frame-pointer"="all"}
+ attributes #0 = { "frame-pointer"="all"}
...
---
name: all_sgpr_used
diff --git a/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types-stress.ll b/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types-stress.ll
new file mode 100644
index 0000000000000..a5743d930c8ae
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types-stress.ll
@@ -0,0 +1,1169 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 2
+; RUN: llc -mtriple=amdgpu9.00--amdpal -amdgpu-stress-vgpr=64 < %s | FileCheck --check-prefix=GFX9 %s
+; RUN: llc -mtriple=amdgpu10.10--amdpal -amdgpu-stress-vgpr=64 < %s | FileCheck --check-prefix=GFX10 %s
+; RUN: llc -mtriple=amdgpu11.00--amdpal -amdgpu-stress-vgpr=64 < %s | FileCheck --check-prefix=GFX11 %s
+
+; Check that return values larger than VGPR limit are handled correctly
+
+define amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val) #1 {
+; GFX9-LABEL: return_72xi32:
+; GFX9: ; %bb.0:
+; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:284
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:156
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:280
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:152
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:276
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:148
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:272
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:144
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:268
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:140
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:264
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:136
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:260
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:132
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:256
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:128
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:252
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:124
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:248
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:120
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:244
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:240
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:236
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:108
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:232
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:104
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:228
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:100
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:224
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:96
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:220
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:92
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:216
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:88
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:212
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:84
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:208
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:204
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:200
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:196
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:68
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:192
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:64
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:188
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:60
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:184
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:56
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:180
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:52
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:176
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:48
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:172
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:44
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:168
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:40
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:164
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:36
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:160
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:32
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:156
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:28
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:152
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:24
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:148
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:20
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:144
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:16
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:140
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:12
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:136
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:8
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:132
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:4
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:128
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:124
+; GFX9-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:120
+; GFX9-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:116
+; GFX9-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:112
+; GFX9-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:108
+; GFX9-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:104
+; GFX9-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:100
+; GFX9-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:96
+; GFX9-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:92
+; GFX9-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:88
+; GFX9-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:84
+; GFX9-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:80
+; GFX9-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:76
+; GFX9-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:72
+; GFX9-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:68
+; GFX9-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:64
+; GFX9-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:60
+; GFX9-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:56
+; GFX9-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:52
+; GFX9-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:48
+; GFX9-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:44
+; GFX9-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:40
+; GFX9-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:36
+; GFX9-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:32
+; GFX9-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:28
+; GFX9-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:24
+; GFX9-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:20
+; GFX9-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:16
+; GFX9-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:12
+; GFX9-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:8
+; GFX9-NEXT: buffer_store_dword v2, v0, s[0:3], 0 offen offset:4
+; GFX9-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: return_72xi32:
+; GFX10: ; %bb.0:
+; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT: s_clause 0x7
+; GFX10-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:64
+; GFX10-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
+; GFX10-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:72
+; GFX10-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
+; GFX10-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:80
+; GFX10-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:84
+; GFX10-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:88
+; GFX10-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:92
+; GFX10-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:120
+; GFX10-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:116
+; GFX10-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:112
+; GFX10-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:108
+; GFX10-NEXT: s_clause 0x7
+; GFX10-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128
+; GFX10-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:132
+; GFX10-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:136
+; GFX10-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:140
+; GFX10-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:144
+; GFX10-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:148
+; GFX10-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:152
+; GFX10-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:156
+; GFX10-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:104
+; GFX10-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:100
+; GFX10-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:96
+; GFX10-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:92
+; GFX10-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:88
+; GFX10-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:84
+; GFX10-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:80
+; GFX10-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:76
+; GFX10-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:72
+; GFX10-NEXT: s_clause 0x7
+; GFX10-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:96
+; GFX10-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:100
+; GFX10-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:104
+; GFX10-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:108
+; GFX10-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:112
+; GFX10-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:116
+; GFX10-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:120
+; GFX10-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:124
+; GFX10-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:68
+; GFX10-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:64
+; GFX10-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:60
+; GFX10-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:56
+; GFX10-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:52
+; GFX10-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:48
+; GFX10-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:44
+; GFX10-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:40
+; GFX10-NEXT: s_clause 0x7
+; GFX10-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:32
+; GFX10-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:36
+; GFX10-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:40
+; GFX10-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:44
+; GFX10-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:48
+; GFX10-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:52
+; GFX10-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:56
+; GFX10-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:60
+; GFX10-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:36
+; GFX10-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:32
+; GFX10-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:28
+; GFX10-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:24
+; GFX10-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:20
+; GFX10-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:16
+; GFX10-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:12
+; GFX10-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:8
+; GFX10-NEXT: s_clause 0x8
+; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28
+; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:24
+; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:20
+; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
+; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:12
+; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:8
+; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:4
+; GFX10-NEXT: buffer_load_dword v10, off, s[0:3], s32
+; GFX10-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:160
+; GFX10-NEXT: buffer_store_dword v2, v0, s[0:3], 0 offen offset:4
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:284
+; GFX10-NEXT: buffer_store_dword v51, v0, s[0:3], 0 offen offset:280
+; GFX10-NEXT: buffer_store_dword v50, v0, s[0:3], 0 offen offset:276
+; GFX10-NEXT: buffer_store_dword v49, v0, s[0:3], 0 offen offset:272
+; GFX10-NEXT: buffer_store_dword v48, v0, s[0:3], 0 offen offset:268
+; GFX10-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:264
+; GFX10-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:260
+; GFX10-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:256
+; GFX10-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:252
+; GFX10-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:248
+; GFX10-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:244
+; GFX10-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:240
+; GFX10-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:236
+; GFX10-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:232
+; GFX10-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:228
+; GFX10-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:224
+; GFX10-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:220
+; GFX10-NEXT: buffer_store_dword v39, v0, s[0:3], 0 offen offset:216
+; GFX10-NEXT: buffer_store_dword v38, v0, s[0:3], 0 offen offset:212
+; GFX10-NEXT: buffer_store_dword v37, v0, s[0:3], 0 offen offset:208
+; GFX10-NEXT: buffer_store_dword v36, v0, s[0:3], 0 offen offset:204
+; GFX10-NEXT: buffer_store_dword v35, v0, s[0:3], 0 offen offset:200
+; GFX10-NEXT: buffer_store_dword v34, v0, s[0:3], 0 offen offset:196
+; GFX10-NEXT: buffer_store_dword v33, v0, s[0:3], 0 offen offset:192
+; GFX10-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:188
+; GFX10-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:184
+; GFX10-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:180
+; GFX10-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:176
+; GFX10-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:172
+; GFX10-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:168
+; GFX10-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:164
+; GFX10-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:160
+; GFX10-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:156
+; GFX10-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:152
+; GFX10-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:148
+; GFX10-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:144
+; GFX10-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:140
+; GFX10-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:136
+; GFX10-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:132
+; GFX10-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:128
+; GFX10-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:124
+; GFX10-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen
+; GFX10-NEXT: s_setpc_b64 s[30:31]
+;
+; GFX11-LABEL: return_72xi32:
+; GFX11: ; %bb.0:
+; GFX11-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX11-NEXT: s_clause 0xc ; 52-byte Folded Spill
+; GFX11-NEXT: scratch_store_b32 off, v40, s32 offset:212
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v41, s32 offset:208
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v42, s32 offset:204
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v43, s32 offset:200
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v44, s32 offset:196
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v56, s32 offset:192
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v57, s32 offset:188
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v58, s32 offset:184
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v59, s32 offset:180
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v60, s32 offset:176
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v61, s32 offset:172
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v62, s32 offset:168
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v63, s32 offset:164
+; GFX11-NEXT: s_clause 0x11
+; GFX11-NEXT: scratch_load_b32 v36, off, s32 offset:16
+; GFX11-NEXT: scratch_load_b32 v35, off, s32 offset:12
+; GFX11-NEXT: scratch_load_b32 v34, off, s32 offset:8
+; GFX11-NEXT: scratch_load_b32 v51, off, s32 offset:32
+; GFX11-NEXT: scratch_load_b32 v50, off, s32 offset:28
+; GFX11-NEXT: scratch_load_b32 v49, off, s32 offset:24
+; GFX11-NEXT: scratch_load_b32 v55, off, s32 offset:48
+; GFX11-NEXT: scratch_load_b32 v54, off, s32 offset:44
+; GFX11-NEXT: scratch_load_b32 v53, off, s32 offset:40
+; GFX11-NEXT: scratch_load_b32 v40, off, s32 offset:64
+; GFX11-NEXT: scratch_load_b32 v39, off, s32 offset:60
+; GFX11-NEXT: scratch_load_b32 v38, off, s32 offset:56
+; GFX11-NEXT: scratch_load_b32 v44, off, s32 offset:80
+; GFX11-NEXT: scratch_load_b32 v43, off, s32 offset:76
+; GFX11-NEXT: scratch_load_b32 v42, off, s32 offset:72
+; GFX11-NEXT: scratch_load_b32 v59, off, s32 offset:96
+; GFX11-NEXT: scratch_load_b32 v58, off, s32 offset:92
+; GFX11-NEXT: scratch_load_b32 v57, off, s32 offset:88
+; GFX11-NEXT: s_clause 0x1
+; GFX11-NEXT: scratch_store_b128 v0, v[21:24], off offset:80
+; GFX11-NEXT: scratch_store_b128 v0, v[17:20], off offset:64
+; GFX11-NEXT: s_clause 0x5
+; GFX11-NEXT: scratch_load_b32 v23, off, s32 offset:112
+; GFX11-NEXT: scratch_load_b32 v22, off, s32 offset:108
+; GFX11-NEXT: scratch_load_b32 v21, off, s32 offset:104
+; GFX11-NEXT: scratch_load_b32 v19, off, s32 offset:128
+; GFX11-NEXT: scratch_load_b32 v18, off, s32 offset:124
+; GFX11-NEXT: scratch_load_b32 v17, off, s32 offset:120
+; GFX11-NEXT: s_clause 0x1
+; GFX11-NEXT: scratch_store_b128 v0, v[13:16], off offset:48
+; GFX11-NEXT: scratch_store_b128 v0, v[9:12], off offset:32
+; GFX11-NEXT: s_clause 0x10
+; GFX11-NEXT: scratch_load_b32 v15, off, s32 offset:144
+; GFX11-NEXT: scratch_load_b32 v14, off, s32 offset:140
+; GFX11-NEXT: scratch_load_b32 v13, off, s32 offset:136
+; GFX11-NEXT: scratch_load_b32 v63, off, s32 offset:160
+; GFX11-NEXT: scratch_load_b32 v62, off, s32 offset:156
+; GFX11-NEXT: scratch_load_b32 v61, off, s32 offset:152
+; GFX11-NEXT: scratch_load_b32 v60, off, s32 offset:148
+; GFX11-NEXT: scratch_load_b32 v12, off, s32 offset:132
+; GFX11-NEXT: scratch_load_b32 v16, off, s32 offset:116
+; GFX11-NEXT: scratch_load_b32 v20, off, s32 offset:100
+; GFX11-NEXT: scratch_load_b32 v56, off, s32 offset:84
+; GFX11-NEXT: scratch_load_b32 v41, off, s32 offset:68
+; GFX11-NEXT: scratch_load_b32 v37, off, s32 offset:52
+; GFX11-NEXT: scratch_load_b32 v52, off, s32 offset:36
+; GFX11-NEXT: scratch_load_b32 v48, off, s32 offset:20
+; GFX11-NEXT: scratch_load_b32 v33, off, s32 offset:4
+; GFX11-NEXT: scratch_load_b32 v32, off, s32
+; GFX11-NEXT: s_waitcnt vmcnt(10)
+; GFX11-NEXT: scratch_store_b128 v0, v[60:63], off offset:272
+; GFX11-NEXT: s_waitcnt vmcnt(9)
+; GFX11-NEXT: scratch_store_b128 v0, v[12:15], off offset:256
+; GFX11-NEXT: s_waitcnt vmcnt(8)
+; GFX11-NEXT: scratch_store_b128 v0, v[16:19], off offset:240
+; GFX11-NEXT: s_waitcnt vmcnt(7)
+; GFX11-NEXT: scratch_store_b128 v0, v[20:23], off offset:224
+; GFX11-NEXT: s_waitcnt vmcnt(6)
+; GFX11-NEXT: scratch_store_b128 v0, v[56:59], off offset:208
+; GFX11-NEXT: s_waitcnt vmcnt(5)
+; GFX11-NEXT: scratch_store_b128 v0, v[41:44], off offset:192
+; GFX11-NEXT: s_waitcnt vmcnt(4)
+; GFX11-NEXT: scratch_store_b128 v0, v[37:40], off offset:176
+; GFX11-NEXT: s_waitcnt vmcnt(3)
+; GFX11-NEXT: scratch_store_b128 v0, v[52:55], off offset:160
+; GFX11-NEXT: s_waitcnt vmcnt(2)
+; GFX11-NEXT: scratch_store_b128 v0, v[48:51], off offset:144
+; GFX11-NEXT: s_waitcnt vmcnt(1)
+; GFX11-NEXT: scratch_store_b128 v0, v[33:36], off offset:128
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: s_clause 0x3
+; GFX11-NEXT: scratch_store_b128 v0, v[29:32], off offset:112
+; GFX11-NEXT: scratch_store_b128 v0, v[25:28], off offset:96
+; GFX11-NEXT: scratch_store_b128 v0, v[5:8], off offset:16
+; GFX11-NEXT: scratch_store_b128 v0, v[1:4], off
+; GFX11-NEXT: s_clause 0xc ; 52-byte Folded Reload
+; GFX11-NEXT: scratch_load_b32 v63, off, s32 offset:164
+; GFX11-NEXT: scratch_load_b32 v62, off, s32 offset:168
+; GFX11-NEXT: scratch_load_b32 v61, off, s32 offset:172
+; GFX11-NEXT: scratch_load_b32 v60, off, s32 offset:176
+; GFX11-NEXT: scratch_load_b32 v59, off, s32 offset:180
+; GFX11-NEXT: scratch_load_b32 v58, off, s32 offset:184
+; GFX11-NEXT: scratch_load_b32 v57, off, s32 offset:188
+; GFX11-NEXT: scratch_load_b32 v56, off, s32 offset:192
+; GFX11-NEXT: scratch_load_b32 v44, off, s32 offset:196
+; GFX11-NEXT: scratch_load_b32 v43, off, s32 offset:200
+; GFX11-NEXT: scratch_load_b32 v42, off, s32 offset:204
+; GFX11-NEXT: scratch_load_b32 v41, off, s32 offset:208
+; GFX11-NEXT: scratch_load_b32 v40, off, s32 offset:212
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: s_setpc_b64 s[30:31]
+ ret <72 x i32> %val
+}
+
+define amdgpu_gfx void @call_72xi32() #1 {
+; GFX9-LABEL: call_72xi32:
+; GFX9: ; %bb.0: ; %entry
+; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX9-NEXT: s_mov_b32 s35, s33
+; GFX9-NEXT: s_add_i32 s33, s32, 0x7fc0
+; GFX9-NEXT: s_and_b32 s33, s33, 0xffff8000
+; GFX9-NEXT: s_or_saveexec_b64 s[36:37], -1
+; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s33 offset:1564 ; 4-byte Folded Spill
+; GFX9-NEXT: s_mov_b64 exec, s[36:37]
+; GFX9-NEXT: s_mov_b32 s38, s34
+; GFX9-NEXT: s_mov_b32 s34, s32
+; GFX9-NEXT: s_add_i32 s32, s32, 0x28000
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s33 ; 4-byte Folded Spill
+; GFX9-NEXT: v_writelane_b32 v63, s30, 0
+; GFX9-NEXT: v_writelane_b32 v63, s31, 1
+; GFX9-NEXT: v_mov_b32_e32 v0, 0
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:16
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:20
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:32
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:36
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:40
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:160
+; GFX9-NEXT: v_lshrrev_b32_e64 v0, 6, s33
+; GFX9-NEXT: s_mov_b32 s37, return_72xi32 at abs32@hi
+; GFX9-NEXT: s_mov_b32 s36, return_72xi32 at abs32@lo
+; GFX9-NEXT: v_add_u32_e32 v0, 0x200, v0
+; GFX9-NEXT: v_mov_b32_e32 v1, 0
+; GFX9-NEXT: v_mov_b32_e32 v2, 0
+; GFX9-NEXT: v_mov_b32_e32 v3, 0
+; GFX9-NEXT: v_mov_b32_e32 v4, 0
+; GFX9-NEXT: v_mov_b32_e32 v5, 0
+; GFX9-NEXT: v_mov_b32_e32 v6, 0
+; GFX9-NEXT: v_mov_b32_e32 v7, 0
+; GFX9-NEXT: v_mov_b32_e32 v8, 0
+; GFX9-NEXT: v_mov_b32_e32 v9, 0
+; GFX9-NEXT: v_mov_b32_e32 v10, 0
+; GFX9-NEXT: v_mov_b32_e32 v11, 0
+; GFX9-NEXT: v_mov_b32_e32 v12, 0
+; GFX9-NEXT: v_mov_b32_e32 v13, 0
+; GFX9-NEXT: v_mov_b32_e32 v14, 0
+; GFX9-NEXT: v_mov_b32_e32 v15, 0
+; GFX9-NEXT: v_mov_b32_e32 v16, 0
+; GFX9-NEXT: v_mov_b32_e32 v17, 0
+; GFX9-NEXT: v_mov_b32_e32 v18, 0
+; GFX9-NEXT: v_mov_b32_e32 v19, 0
+; GFX9-NEXT: v_mov_b32_e32 v20, 0
+; GFX9-NEXT: v_mov_b32_e32 v21, 0
+; GFX9-NEXT: v_mov_b32_e32 v22, 0
+; GFX9-NEXT: v_mov_b32_e32 v23, 0
+; GFX9-NEXT: v_mov_b32_e32 v24, 0
+; GFX9-NEXT: v_mov_b32_e32 v25, 0
+; GFX9-NEXT: v_mov_b32_e32 v26, 0
+; GFX9-NEXT: v_mov_b32_e32 v27, 0
+; GFX9-NEXT: v_mov_b32_e32 v28, 0
+; GFX9-NEXT: v_mov_b32_e32 v29, 0
+; GFX9-NEXT: v_mov_b32_e32 v30, 0
+; GFX9-NEXT: v_mov_b32_e32 v31, 0
+; GFX9-NEXT: s_swappc_b64 s[30:31], s[36:37]
+; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:636
+; GFX9-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:640
+; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s33 offset:644
+; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s33 offset:648
+; GFX9-NEXT: buffer_load_dword v34, off, s[0:3], s33 offset:652
+; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s33 offset:656
+; GFX9-NEXT: buffer_load_dword v36, off, s[0:3], s33 offset:660
+; GFX9-NEXT: buffer_load_dword v37, off, s[0:3], s33 offset:664
+; GFX9-NEXT: buffer_load_dword v38, off, s[0:3], s33 offset:668
+; GFX9-NEXT: buffer_load_dword v39, off, s[0:3], s33 offset:672
+; GFX9-NEXT: buffer_load_dword v48, off, s[0:3], s33 offset:676
+; GFX9-NEXT: buffer_load_dword v49, off, s[0:3], s33 offset:680
+; GFX9-NEXT: buffer_load_dword v50, off, s[0:3], s33 offset:684
+; GFX9-NEXT: buffer_load_dword v51, off, s[0:3], s33 offset:688
+; GFX9-NEXT: buffer_load_dword v52, off, s[0:3], s33 offset:692
+; GFX9-NEXT: buffer_load_dword v53, off, s[0:3], s33 offset:696
+; GFX9-NEXT: buffer_load_dword v54, off, s[0:3], s33 offset:700
+; GFX9-NEXT: buffer_load_dword v55, off, s[0:3], s33 offset:704
+; GFX9-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:708
+; GFX9-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:712
+; GFX9-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:716
+; GFX9-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:720
+; GFX9-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:724
+; GFX9-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:728
+; GFX9-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:732
+; GFX9-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:736
+; GFX9-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:740
+; GFX9-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:748
+; GFX9-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:752
+; GFX9-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:756
+; GFX9-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:760
+; GFX9-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:764
+; GFX9-NEXT: buffer_load_dword v62, off, s[0:3], s33 offset:768
+; GFX9-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:772
+; GFX9-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:776
+; GFX9-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:780
+; GFX9-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:784
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:788
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:792
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:796
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:516
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1536 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:520
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1540 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:524
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1544 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:528
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1548 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:532
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1552 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:536
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1556 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:540
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1560 ; 4-byte Folded Spill
+; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:544
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_load_dword v10, off, s[0:3], s33 offset:548
+; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s33 offset:552
+; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s33 offset:556
+; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s33 offset:560
+; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s33 offset:564
+; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s33 offset:568
+; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s33 offset:572
+; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s33 offset:576
+; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s33 offset:580
+; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s33 offset:584
+; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s33 offset:588
+; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s33 offset:592
+; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s33 offset:596
+; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s33 offset:600
+; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s33 offset:604
+; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s33 offset:608
+; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s33 offset:612
+; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s33 offset:616
+; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s33 offset:620
+; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s33 offset:624
+; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s33 offset:628
+; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:632
+; GFX9-NEXT: s_nop 0
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32
+; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:4
+; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:8
+; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:12
+; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:16
+; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:20
+; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:24
+; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:28
+; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:32
+; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:36
+; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:40
+; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:44
+; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:48
+; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:52
+; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:56
+; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:60
+; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:64
+; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:68
+; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72
+; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:76
+; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:80
+; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:84
+; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:88
+; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:92
+; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:96
+; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:100
+; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:104
+; GFX9-NEXT: v_mov_b32_e32 v0, 24
+; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
+; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:112
+; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:116
+; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:120
+; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:124
+; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:128
+; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:132
+; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136
+; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:140
+; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:144
+; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:148
+; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:152
+; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:156
+; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:160
+; GFX9-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:1536 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:1540 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:1544 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:1548 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:1552 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:1556 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:1560 ; 4-byte Folded Reload
+; GFX9-NEXT: v_lshrrev_b32_e64 v0, 6, s33
+; GFX9-NEXT: v_add_u32_e32 v0, 0x400, v0
+; GFX9-NEXT: v_mov_b32_e32 v1, 42
+; GFX9-NEXT: s_swappc_b64 s[30:31], s[36:37]
+; GFX9-NEXT: buffer_load_dword v62, off, s[0:3], s33 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Reload
+; GFX9-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Reload
+; GFX9-NEXT: v_readlane_b32 s30, v63, 0
+; GFX9-NEXT: v_readlane_b32 s31, v63, 1
+; GFX9-NEXT: s_mov_b32 s32, s34
+; GFX9-NEXT: s_mov_b32 s34, s38
+; GFX9-NEXT: s_or_saveexec_b64 s[36:37], -1
+; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s33 offset:1564 ; 4-byte Folded Reload
+; GFX9-NEXT: s_mov_b64 exec, s[36:37]
+; GFX9-NEXT: s_mov_b32 s33, s35
+; GFX9-NEXT: s_waitcnt vmcnt(0)
+; GFX9-NEXT: s_setpc_b64 s[30:31]
+;
+; GFX10-LABEL: call_72xi32:
+; GFX10: ; %bb.0: ; %entry
+; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX10-NEXT: s_mov_b32 s35, s33
+; GFX10-NEXT: s_add_i32 s33, s32, 0x3fe0
+; GFX10-NEXT: s_and_b32 s33, s33, 0xffffc000
+; GFX10-NEXT: s_or_saveexec_b32 s36, -1
+; GFX10-NEXT: buffer_store_dword v63, off, s[0:3], s33 offset:1568 ; 4-byte Folded Spill
+; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
+; GFX10-NEXT: s_mov_b32 exec_lo, s36
+; GFX10-NEXT: s_mov_b32 s38, s34
+; GFX10-NEXT: s_mov_b32 s34, s32
+; GFX10-NEXT: s_add_i32 s32, s32, 0x14000
+; GFX10-NEXT: buffer_store_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_store_dword v62, off, s[0:3], s33 ; 4-byte Folded Spill
+; GFX10-NEXT: v_writelane_b32 v63, s30, 0
+; GFX10-NEXT: v_writelane_b32 v63, s31, 1
+; GFX10-NEXT: v_mov_b32_e32 v0, 0
+; GFX10-NEXT: v_mov_b32_e32 v1, 0
+; GFX10-NEXT: v_mov_b32_e32 v2, 0
+; GFX10-NEXT: v_mov_b32_e32 v3, 0
+; GFX10-NEXT: v_mov_b32_e32 v4, 0
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:4
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:8
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:12
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:16
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:20
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:28
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:32
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:36
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:40
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:44
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:48
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:52
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:144
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:160
+; GFX10-NEXT: v_lshrrev_b32_e64 v0, 5, s33
+; GFX10-NEXT: v_mov_b32_e32 v5, 0
+; GFX10-NEXT: v_mov_b32_e32 v6, 0
+; GFX10-NEXT: v_mov_b32_e32 v7, 0
+; GFX10-NEXT: v_mov_b32_e32 v8, 0
+; GFX10-NEXT: v_add_nc_u32_e32 v0, 0x200, v0
+; GFX10-NEXT: v_mov_b32_e32 v9, 0
+; GFX10-NEXT: v_mov_b32_e32 v10, 0
+; GFX10-NEXT: v_mov_b32_e32 v11, 0
+; GFX10-NEXT: v_mov_b32_e32 v12, 0
+; GFX10-NEXT: v_mov_b32_e32 v13, 0
+; GFX10-NEXT: v_mov_b32_e32 v14, 0
+; GFX10-NEXT: v_mov_b32_e32 v15, 0
+; GFX10-NEXT: v_mov_b32_e32 v16, 0
+; GFX10-NEXT: v_mov_b32_e32 v17, 0
+; GFX10-NEXT: v_mov_b32_e32 v18, 0
+; GFX10-NEXT: v_mov_b32_e32 v19, 0
+; GFX10-NEXT: v_mov_b32_e32 v20, 0
+; GFX10-NEXT: v_mov_b32_e32 v21, 0
+; GFX10-NEXT: v_mov_b32_e32 v22, 0
+; GFX10-NEXT: v_mov_b32_e32 v23, 0
+; GFX10-NEXT: v_mov_b32_e32 v24, 0
+; GFX10-NEXT: v_mov_b32_e32 v25, 0
+; GFX10-NEXT: v_mov_b32_e32 v26, 0
+; GFX10-NEXT: v_mov_b32_e32 v27, 0
+; GFX10-NEXT: v_mov_b32_e32 v28, 0
+; GFX10-NEXT: v_mov_b32_e32 v29, 0
+; GFX10-NEXT: v_mov_b32_e32 v30, 0
+; GFX10-NEXT: v_mov_b32_e32 v31, 0
+; GFX10-NEXT: s_mov_b32 s37, return_72xi32 at abs32@hi
+; GFX10-NEXT: s_mov_b32 s36, return_72xi32 at abs32@lo
+; GFX10-NEXT: s_swappc_b64 s[30:31], s[36:37]
+; GFX10-NEXT: s_clause 0x28
+; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:636
+; GFX10-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:640
+; GFX10-NEXT: buffer_load_dword v32, off, s[0:3], s33 offset:644
+; GFX10-NEXT: buffer_load_dword v33, off, s[0:3], s33 offset:648
+; GFX10-NEXT: buffer_load_dword v34, off, s[0:3], s33 offset:652
+; GFX10-NEXT: buffer_load_dword v35, off, s[0:3], s33 offset:656
+; GFX10-NEXT: buffer_load_dword v36, off, s[0:3], s33 offset:660
+; GFX10-NEXT: buffer_load_dword v37, off, s[0:3], s33 offset:664
+; GFX10-NEXT: buffer_load_dword v38, off, s[0:3], s33 offset:668
+; GFX10-NEXT: buffer_load_dword v39, off, s[0:3], s33 offset:672
+; GFX10-NEXT: buffer_load_dword v48, off, s[0:3], s33 offset:676
+; GFX10-NEXT: buffer_load_dword v49, off, s[0:3], s33 offset:680
+; GFX10-NEXT: buffer_load_dword v50, off, s[0:3], s33 offset:684
+; GFX10-NEXT: buffer_load_dword v51, off, s[0:3], s33 offset:688
+; GFX10-NEXT: buffer_load_dword v52, off, s[0:3], s33 offset:692
+; GFX10-NEXT: buffer_load_dword v53, off, s[0:3], s33 offset:696
+; GFX10-NEXT: buffer_load_dword v54, off, s[0:3], s33 offset:700
+; GFX10-NEXT: buffer_load_dword v55, off, s[0:3], s33 offset:704
+; GFX10-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:708
+; GFX10-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:712
+; GFX10-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:716
+; GFX10-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:720
+; GFX10-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:724
+; GFX10-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:728
+; GFX10-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:732
+; GFX10-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:736
+; GFX10-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:740
+; GFX10-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:748
+; GFX10-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:752
+; GFX10-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:756
+; GFX10-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:760
+; GFX10-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:764
+; GFX10-NEXT: buffer_load_dword v62, off, s[0:3], s33 offset:768
+; GFX10-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:772
+; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:776
+; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:780
+; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:784
+; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:788
+; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:792
+; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:796
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:516
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1536 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:520
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1540 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:524
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1544 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:528
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1548 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:532
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1552 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:536
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1556 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:540
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1560 ; 4-byte Folded Spill
+; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:544
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1564 ; 4-byte Folded Spill
+; GFX10-NEXT: s_clause 0x15
+; GFX10-NEXT: buffer_load_dword v10, off, s[0:3], s33 offset:548
+; GFX10-NEXT: buffer_load_dword v11, off, s[0:3], s33 offset:552
+; GFX10-NEXT: buffer_load_dword v12, off, s[0:3], s33 offset:556
+; GFX10-NEXT: buffer_load_dword v13, off, s[0:3], s33 offset:560
+; GFX10-NEXT: buffer_load_dword v14, off, s[0:3], s33 offset:564
+; GFX10-NEXT: buffer_load_dword v15, off, s[0:3], s33 offset:568
+; GFX10-NEXT: buffer_load_dword v16, off, s[0:3], s33 offset:572
+; GFX10-NEXT: buffer_load_dword v17, off, s[0:3], s33 offset:576
+; GFX10-NEXT: buffer_load_dword v18, off, s[0:3], s33 offset:580
+; GFX10-NEXT: buffer_load_dword v19, off, s[0:3], s33 offset:584
+; GFX10-NEXT: buffer_load_dword v20, off, s[0:3], s33 offset:588
+; GFX10-NEXT: buffer_load_dword v21, off, s[0:3], s33 offset:592
+; GFX10-NEXT: buffer_load_dword v22, off, s[0:3], s33 offset:596
+; GFX10-NEXT: buffer_load_dword v23, off, s[0:3], s33 offset:600
+; GFX10-NEXT: buffer_load_dword v24, off, s[0:3], s33 offset:604
+; GFX10-NEXT: buffer_load_dword v25, off, s[0:3], s33 offset:608
+; GFX10-NEXT: buffer_load_dword v26, off, s[0:3], s33 offset:612
+; GFX10-NEXT: buffer_load_dword v27, off, s[0:3], s33 offset:616
+; GFX10-NEXT: buffer_load_dword v28, off, s[0:3], s33 offset:620
+; GFX10-NEXT: buffer_load_dword v29, off, s[0:3], s33 offset:624
+; GFX10-NEXT: buffer_load_dword v30, off, s[0:3], s33 offset:628
+; GFX10-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:632
+; GFX10-NEXT: v_mov_b32_e32 v0, 24
+; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
+; GFX10-NEXT: buffer_store_dword v9, off, s[0:3], s32
+; GFX10-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:4
+; GFX10-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:8
+; GFX10-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:12
+; GFX10-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:16
+; GFX10-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:20
+; GFX10-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:24
+; GFX10-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:28
+; GFX10-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:32
+; GFX10-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:36
+; GFX10-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:40
+; GFX10-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:44
+; GFX10-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:48
+; GFX10-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:52
+; GFX10-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:56
+; GFX10-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:60
+; GFX10-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:64
+; GFX10-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:68
+; GFX10-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72
+; GFX10-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:76
+; GFX10-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:80
+; GFX10-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:84
+; GFX10-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:88
+; GFX10-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:92
+; GFX10-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:96
+; GFX10-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:100
+; GFX10-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:104
+; GFX10-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:112
+; GFX10-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:116
+; GFX10-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:120
+; GFX10-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:124
+; GFX10-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:128
+; GFX10-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:132
+; GFX10-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136
+; GFX10-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:140
+; GFX10-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:144
+; GFX10-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:148
+; GFX10-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:152
+; GFX10-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:156
+; GFX10-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:160
+; GFX10-NEXT: s_clause 0x7 ; 32-byte Folded Reload
+; GFX10-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:1536
+; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:1540
+; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:1544
+; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:1548
+; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:1552
+; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:1556
+; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:1560
+; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:1564
+; GFX10-NEXT: v_lshrrev_b32_e64 v0, 5, s33
+; GFX10-NEXT: v_mov_b32_e32 v1, 42
+; GFX10-NEXT: v_add_nc_u32_e32 v0, 0x400, v0
+; GFX10-NEXT: s_swappc_b64 s[30:31], s[36:37]
+; GFX10-NEXT: s_clause 0xe ; 60-byte Folded Reload
+; GFX10-NEXT: buffer_load_dword v62, off, s[0:3], s33
+; GFX10-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:4
+; GFX10-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:8
+; GFX10-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:12
+; GFX10-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:16
+; GFX10-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:20
+; GFX10-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:24
+; GFX10-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:28
+; GFX10-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:32
+; GFX10-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:36
+; GFX10-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:40
+; GFX10-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:44
+; GFX10-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:48
+; GFX10-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:52
+; GFX10-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:56
+; GFX10-NEXT: v_readlane_b32 s30, v63, 0
+; GFX10-NEXT: v_readlane_b32 s31, v63, 1
+; GFX10-NEXT: s_mov_b32 s32, s34
+; GFX10-NEXT: s_mov_b32 s34, s38
+; GFX10-NEXT: s_or_saveexec_b32 s36, -1
+; GFX10-NEXT: buffer_load_dword v63, off, s[0:3], s33 offset:1568 ; 4-byte Folded Reload
+; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
+; GFX10-NEXT: s_mov_b32 exec_lo, s36
+; GFX10-NEXT: s_mov_b32 s33, s35
+; GFX10-NEXT: s_waitcnt vmcnt(0)
+; GFX10-NEXT: s_setpc_b64 s[30:31]
+;
+; GFX11-LABEL: call_72xi32:
+; GFX11: ; %bb.0: ; %entry
+; GFX11-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; GFX11-NEXT: s_mov_b32 s38, s33
+; GFX11-NEXT: s_add_i32 s33, s32, 0x1ff
+; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
+; GFX11-NEXT: s_and_b32 s33, s33, 0xfffffe00
+; GFX11-NEXT: s_or_saveexec_b32 s0, -1
+; GFX11-NEXT: scratch_store_b32 off, v62, s33 offset:1600 ; 4-byte Folded Spill
+; GFX11-NEXT: s_mov_b32 exec_lo, s0
+; GFX11-NEXT: s_mov_b32 s39, s34
+; GFX11-NEXT: s_mov_b32 s34, s32
+; GFX11-NEXT: s_addk_i32 s32, 0xa00
+; GFX11-NEXT: s_clause 0xd ; 56-byte Folded Spill
+; GFX11-NEXT: scratch_store_b32 off, v40, s33 offset:52
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v41, s33 offset:48
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v42, s33 offset:44
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v43, s33 offset:40
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v44, s33 offset:36
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v45, s33 offset:32
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v46, s33 offset:28
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v47, s33 offset:24
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v56, s33 offset:20
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v57, s33 offset:16
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v58, s33 offset:12
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v59, s33 offset:8
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v60, s33 offset:4
+; GFX11-NEXT: ; meta instruction
+; GFX11-NEXT: scratch_store_b32 off, v61, s33
+; GFX11-NEXT: v_writelane_b32 v62, s30, 0
+; GFX11-NEXT: v_writelane_b32 v62, s31, 1
+; GFX11-NEXT: s_mov_b32 s0, 0
+; GFX11-NEXT: v_mov_b32_e32 v4, 0
+; GFX11-NEXT: s_mov_b32 s1, s0
+; GFX11-NEXT: s_mov_b32 s2, s0
+; GFX11-NEXT: s_mov_b32 s3, s0
+; GFX11-NEXT: v_dual_mov_b32 v0, s0 :: v_dual_mov_b32 v1, s1
+; GFX11-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3
+; GFX11-NEXT: s_add_i32 s0, s32, 0xa0
+; GFX11-NEXT: s_add_i32 s1, s32, 0x90
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s32
+; GFX11-NEXT: scratch_store_b32 off, v4, s0
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
+; GFX11-NEXT: s_add_i32 s0, s32, 0x80
+; GFX11-NEXT: s_add_i32 s1, s32, 0x70
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
+; GFX11-NEXT: s_add_i32 s0, s32, 0x60
+; GFX11-NEXT: s_add_i32 s1, s32, 0x50
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
+; GFX11-NEXT: s_add_i32 s0, s32, 64
+; GFX11-NEXT: s_add_i32 s1, s32, 48
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
+; GFX11-NEXT: s_add_i32 s0, s32, 32
+; GFX11-NEXT: s_add_i32 s1, s32, 16
+; GFX11-NEXT: s_add_i32 s2, s33, 0x200
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
+; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
+; GFX11-NEXT: v_dual_mov_b32 v0, s2 :: v_dual_mov_b32 v3, 0
+; GFX11-NEXT: v_dual_mov_b32 v1, 0 :: v_dual_mov_b32 v2, 0
+; GFX11-NEXT: v_dual_mov_b32 v5, 0 :: v_dual_mov_b32 v4, 0
+; GFX11-NEXT: v_dual_mov_b32 v7, 0 :: v_dual_mov_b32 v6, 0
+; GFX11-NEXT: v_dual_mov_b32 v9, 0 :: v_dual_mov_b32 v8, 0
+; GFX11-NEXT: v_dual_mov_b32 v11, 0 :: v_dual_mov_b32 v10, 0
+; GFX11-NEXT: v_dual_mov_b32 v13, 0 :: v_dual_mov_b32 v12, 0
+; GFX11-NEXT: v_dual_mov_b32 v15, 0 :: v_dual_mov_b32 v14, 0
+; GFX11-NEXT: v_dual_mov_b32 v17, 0 :: v_dual_mov_b32 v16, 0
+; GFX11-NEXT: v_dual_mov_b32 v19, 0 :: v_dual_mov_b32 v18, 0
+; GFX11-NEXT: v_dual_mov_b32 v21, 0 :: v_dual_mov_b32 v20, 0
+; GFX11-NEXT: v_dual_mov_b32 v23, 0 :: v_dual_mov_b32 v22, 0
+; GFX11-NEXT: v_dual_mov_b32 v25, 0 :: v_dual_mov_b32 v24, 0
+; GFX11-NEXT: v_dual_mov_b32 v27, 0 :: v_dual_mov_b32 v26, 0
+; GFX11-NEXT: v_dual_mov_b32 v29, 0 :: v_dual_mov_b32 v28, 0
+; GFX11-NEXT: v_dual_mov_b32 v31, 0 :: v_dual_mov_b32 v30, 0
+; GFX11-NEXT: s_mov_b32 s1, return_72xi32 at abs32@hi
+; GFX11-NEXT: s_mov_b32 s0, return_72xi32 at abs32@lo
+; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
+; GFX11-NEXT: s_swappc_b64 s[30:31], s[0:1]
+; GFX11-NEXT: s_clause 0xb
+; GFX11-NEXT: scratch_load_b128 v[43:46], off, s33 offset:624
+; GFX11-NEXT: scratch_load_b128 v[47:50], off, s33 offset:640
+; GFX11-NEXT: scratch_load_b128 v[16:19], off, s33 offset:784
+; GFX11-NEXT: scratch_load_b128 v[12:15], off, s33 offset:768
+; GFX11-NEXT: scratch_load_b128 v[8:11], off, s33 offset:752
+; GFX11-NEXT: scratch_load_b128 v[4:7], off, s33 offset:736
+; GFX11-NEXT: scratch_load_b128 v[0:3], off, s33 offset:720
+; GFX11-NEXT: scratch_load_b128 v[24:27], off, s33 offset:656
+; GFX11-NEXT: scratch_load_b128 v[36:39], off, s33 offset:704
+; GFX11-NEXT: scratch_load_b128 v[32:35], off, s33 offset:688
+; GFX11-NEXT: scratch_load_b128 v[28:31], off, s33 offset:672
+; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:512
+; GFX11-NEXT: s_waitcnt vmcnt(6)
+; GFX11-NEXT: v_mov_b32_e32 v6, 24
+; GFX11-NEXT: s_add_i32 s2, s32, 0xa0
+; GFX11-NEXT: s_add_i32 s3, s32, 0x90
+; GFX11-NEXT: s_add_i32 s35, s32, 0x80
+; GFX11-NEXT: s_add_i32 s36, s32, 0x70
+; GFX11-NEXT: s_add_i32 s37, s32, 0x6c
+; GFX11-NEXT: s_waitcnt vmcnt(4)
+; GFX11-NEXT: v_dual_mov_b32 v40, v0 :: v_dual_mov_b32 v53, v26
+; GFX11-NEXT: v_dual_mov_b32 v41, v1 :: v_dual_mov_b32 v42, v2
+; GFX11-NEXT: v_mov_b32_e32 v51, v24
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1584 ; 16-byte Folded Spill
+; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:528
+; GFX11-NEXT: v_mov_b32_e32 v52, v25
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1536 ; 16-byte Folded Spill
+; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:544
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1568 ; 16-byte Folded Spill
+; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:560
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1552 ; 16-byte Folded Spill
+; GFX11-NEXT: s_clause 0x2
+; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:576
+; GFX11-NEXT: scratch_load_b128 v[58:61], off, s33 offset:592
+; GFX11-NEXT: scratch_load_b128 v[54:57], off, s33 offset:608
+; GFX11-NEXT: scratch_store_b128 off, v[46:49], s32
+; GFX11-NEXT: scratch_store_b32 off, v19, s2
+; GFX11-NEXT: scratch_store_b128 off, v[15:18], s3
+; GFX11-NEXT: scratch_store_b128 off, v[11:14], s35
+; GFX11-NEXT: scratch_store_b128 off, v[7:10], s36
+; GFX11-NEXT: s_add_i32 s2, s32, 0x60
+; GFX11-NEXT: scratch_store_b32 off, v6, s37
+; GFX11-NEXT: scratch_store_b96 off, v[3:5], s2
+; GFX11-NEXT: s_add_i32 s2, s32, 0x50
+; GFX11-NEXT: s_add_i32 s3, s32, 64
+; GFX11-NEXT: s_add_i32 s35, s32, 48
+; GFX11-NEXT: s_add_i32 s36, s32, 32
+; GFX11-NEXT: scratch_store_b128 off, v[39:42], s2
+; GFX11-NEXT: scratch_store_b128 off, v[35:38], s3
+; GFX11-NEXT: scratch_store_b128 off, v[31:34], s35
+; GFX11-NEXT: scratch_store_b128 off, v[27:30], s36
+; GFX11-NEXT: s_add_i32 s2, s32, 16
+; GFX11-NEXT: v_mov_b32_e32 v29, v43
+; GFX11-NEXT: scratch_store_b128 off, v[50:53], s2
+; GFX11-NEXT: s_clause 0x3 ; 64-byte Folded Reload
+; GFX11-NEXT: scratch_load_b128 v[1:4], off, s33 offset:1584
+; GFX11-NEXT: scratch_load_b128 v[5:8], off, s33 offset:1536
+; GFX11-NEXT: scratch_load_b128 v[9:12], off, s33 offset:1568
+; GFX11-NEXT: scratch_load_b128 v[13:16], off, s33 offset:1552
+; GFX11-NEXT: s_add_i32 s2, s33, 0x400
+; GFX11-NEXT: v_dual_mov_b32 v30, v44 :: v_dual_mov_b32 v31, v45
+; GFX11-NEXT: s_waitcnt vmcnt(3)
+; GFX11-NEXT: v_dual_mov_b32 v0, s2 :: v_dual_mov_b32 v1, 42
+; GFX11-NEXT: v_dual_mov_b32 v17, v20 :: v_dual_mov_b32 v18, v21
+; GFX11-NEXT: v_dual_mov_b32 v19, v22 :: v_dual_mov_b32 v20, v23
+; GFX11-NEXT: v_dual_mov_b32 v21, v58 :: v_dual_mov_b32 v22, v59
+; GFX11-NEXT: v_dual_mov_b32 v23, v60 :: v_dual_mov_b32 v24, v61
+; GFX11-NEXT: v_dual_mov_b32 v25, v54 :: v_dual_mov_b32 v26, v55
+; GFX11-NEXT: v_dual_mov_b32 v27, v56 :: v_dual_mov_b32 v28, v57
+; GFX11-NEXT: s_swappc_b64 s[30:31], s[0:1]
+; GFX11-NEXT: s_clause 0xd ; 56-byte Folded Reload
+; GFX11-NEXT: scratch_load_b32 v61, off, s33
+; GFX11-NEXT: scratch_load_b32 v60, off, s33 offset:4
+; GFX11-NEXT: scratch_load_b32 v59, off, s33 offset:8
+; GFX11-NEXT: scratch_load_b32 v58, off, s33 offset:12
+; GFX11-NEXT: scratch_load_b32 v57, off, s33 offset:16
+; GFX11-NEXT: scratch_load_b32 v56, off, s33 offset:20
+; GFX11-NEXT: scratch_load_b32 v47, off, s33 offset:24
+; GFX11-NEXT: scratch_load_b32 v46, off, s33 offset:28
+; GFX11-NEXT: scratch_load_b32 v45, off, s33 offset:32
+; GFX11-NEXT: scratch_load_b32 v44, off, s33 offset:36
+; GFX11-NEXT: scratch_load_b32 v43, off, s33 offset:40
+; GFX11-NEXT: scratch_load_b32 v42, off, s33 offset:44
+; GFX11-NEXT: scratch_load_b32 v41, off, s33 offset:48
+; GFX11-NEXT: scratch_load_b32 v40, off, s33 offset:52
+; GFX11-NEXT: v_readlane_b32 s30, v62, 0
+; GFX11-NEXT: v_readlane_b32 s31, v62, 1
+; GFX11-NEXT: s_mov_b32 s32, s34
+; GFX11-NEXT: s_mov_b32 s34, s39
+; GFX11-NEXT: s_or_saveexec_b32 s0, -1
+; GFX11-NEXT: scratch_load_b32 v62, off, s33 offset:1600 ; 4-byte Folded Reload
+; GFX11-NEXT: s_mov_b32 exec_lo, s0
+; GFX11-NEXT: s_mov_b32 s33, s38
+; GFX11-NEXT: s_waitcnt vmcnt(0)
+; GFX11-NEXT: s_setpc_b64 s[30:31]
+entry:
+ %ret.0 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> zeroinitializer)
+ %val.0 = insertelement <72 x i32> %ret.0, i32 42, i32 0
+ %val.1 = insertelement <72 x i32> %val.0, i32 24, i32 58
+ %ret.1 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val.1)
+ ret void
+}
+
+attributes #1 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll b/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll
index f805b11d35db5..4245ec7374d0e 100644
--- a/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll
+++ b/llvm/test/CodeGen/AMDGPU/gfx-callable-return-types.ll
@@ -2252,1171 +2252,5 @@ entry:
ret void
}
-; Check that return values larger than VGPR limit are handled correctly
-
-define amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val) #1 {
-; GFX9-LABEL: return_72xi32:
-; GFX9: ; %bb.0:
-; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:284
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:156
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:280
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:152
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:276
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:148
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:272
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:144
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:268
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:140
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:264
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:136
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:260
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:132
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:256
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:128
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:252
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:124
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:248
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:120
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:244
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:116
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:240
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:112
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:236
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:108
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:232
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:104
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:228
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:100
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:224
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:96
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:220
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:92
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:216
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:88
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:212
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:84
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:208
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:80
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:204
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:76
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:200
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:196
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:68
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:192
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:64
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:188
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:60
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:184
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:56
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:180
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:52
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:176
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:48
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:172
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:44
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:168
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:40
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:164
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:36
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:160
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:32
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:156
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:28
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:152
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:24
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:148
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:20
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:144
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:16
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:140
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:12
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:136
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:8
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:132
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:4
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:128
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s32
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:124
-; GFX9-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:120
-; GFX9-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:116
-; GFX9-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:112
-; GFX9-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:108
-; GFX9-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:104
-; GFX9-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:100
-; GFX9-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:96
-; GFX9-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:92
-; GFX9-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:88
-; GFX9-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:84
-; GFX9-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:80
-; GFX9-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:76
-; GFX9-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:72
-; GFX9-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:68
-; GFX9-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:64
-; GFX9-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:60
-; GFX9-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:56
-; GFX9-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:52
-; GFX9-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:48
-; GFX9-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:44
-; GFX9-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:40
-; GFX9-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:36
-; GFX9-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:32
-; GFX9-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:28
-; GFX9-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:24
-; GFX9-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:20
-; GFX9-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:16
-; GFX9-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:12
-; GFX9-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:8
-; GFX9-NEXT: buffer_store_dword v2, v0, s[0:3], 0 offen offset:4
-; GFX9-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: s_setpc_b64 s[30:31]
-;
-; GFX10-LABEL: return_72xi32:
-; GFX10: ; %bb.0:
-; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX10-NEXT: s_clause 0x7
-; GFX10-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:64
-; GFX10-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
-; GFX10-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:72
-; GFX10-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:76
-; GFX10-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:80
-; GFX10-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:84
-; GFX10-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:88
-; GFX10-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:92
-; GFX10-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:120
-; GFX10-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:116
-; GFX10-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:112
-; GFX10-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:108
-; GFX10-NEXT: s_clause 0x7
-; GFX10-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:128
-; GFX10-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:132
-; GFX10-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:136
-; GFX10-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:140
-; GFX10-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:144
-; GFX10-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:148
-; GFX10-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:152
-; GFX10-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:156
-; GFX10-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:104
-; GFX10-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:100
-; GFX10-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:96
-; GFX10-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:92
-; GFX10-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:88
-; GFX10-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:84
-; GFX10-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:80
-; GFX10-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:76
-; GFX10-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:72
-; GFX10-NEXT: s_clause 0x7
-; GFX10-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:96
-; GFX10-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:100
-; GFX10-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:104
-; GFX10-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:108
-; GFX10-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:112
-; GFX10-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:116
-; GFX10-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:120
-; GFX10-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:124
-; GFX10-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:68
-; GFX10-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:64
-; GFX10-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:60
-; GFX10-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:56
-; GFX10-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:52
-; GFX10-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:48
-; GFX10-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:44
-; GFX10-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:40
-; GFX10-NEXT: s_clause 0x7
-; GFX10-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:32
-; GFX10-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:36
-; GFX10-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:40
-; GFX10-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:44
-; GFX10-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:48
-; GFX10-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:52
-; GFX10-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:56
-; GFX10-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:60
-; GFX10-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:36
-; GFX10-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:32
-; GFX10-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:28
-; GFX10-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:24
-; GFX10-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:20
-; GFX10-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:16
-; GFX10-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:12
-; GFX10-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:8
-; GFX10-NEXT: s_clause 0x8
-; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28
-; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:24
-; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:20
-; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:16
-; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:12
-; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:8
-; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:4
-; GFX10-NEXT: buffer_load_dword v10, off, s[0:3], s32
-; GFX10-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:160
-; GFX10-NEXT: buffer_store_dword v2, v0, s[0:3], 0 offen offset:4
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v27, v0, s[0:3], 0 offen offset:284
-; GFX10-NEXT: buffer_store_dword v51, v0, s[0:3], 0 offen offset:280
-; GFX10-NEXT: buffer_store_dword v50, v0, s[0:3], 0 offen offset:276
-; GFX10-NEXT: buffer_store_dword v49, v0, s[0:3], 0 offen offset:272
-; GFX10-NEXT: buffer_store_dword v48, v0, s[0:3], 0 offen offset:268
-; GFX10-NEXT: buffer_store_dword v31, v0, s[0:3], 0 offen offset:264
-; GFX10-NEXT: buffer_store_dword v30, v0, s[0:3], 0 offen offset:260
-; GFX10-NEXT: buffer_store_dword v29, v0, s[0:3], 0 offen offset:256
-; GFX10-NEXT: buffer_store_dword v28, v0, s[0:3], 0 offen offset:252
-; GFX10-NEXT: buffer_store_dword v26, v0, s[0:3], 0 offen offset:248
-; GFX10-NEXT: buffer_store_dword v25, v0, s[0:3], 0 offen offset:244
-; GFX10-NEXT: buffer_store_dword v24, v0, s[0:3], 0 offen offset:240
-; GFX10-NEXT: buffer_store_dword v23, v0, s[0:3], 0 offen offset:236
-; GFX10-NEXT: buffer_store_dword v22, v0, s[0:3], 0 offen offset:232
-; GFX10-NEXT: buffer_store_dword v21, v0, s[0:3], 0 offen offset:228
-; GFX10-NEXT: buffer_store_dword v20, v0, s[0:3], 0 offen offset:224
-; GFX10-NEXT: buffer_store_dword v19, v0, s[0:3], 0 offen offset:220
-; GFX10-NEXT: buffer_store_dword v39, v0, s[0:3], 0 offen offset:216
-; GFX10-NEXT: buffer_store_dword v38, v0, s[0:3], 0 offen offset:212
-; GFX10-NEXT: buffer_store_dword v37, v0, s[0:3], 0 offen offset:208
-; GFX10-NEXT: buffer_store_dword v36, v0, s[0:3], 0 offen offset:204
-; GFX10-NEXT: buffer_store_dword v35, v0, s[0:3], 0 offen offset:200
-; GFX10-NEXT: buffer_store_dword v34, v0, s[0:3], 0 offen offset:196
-; GFX10-NEXT: buffer_store_dword v33, v0, s[0:3], 0 offen offset:192
-; GFX10-NEXT: buffer_store_dword v32, v0, s[0:3], 0 offen offset:188
-; GFX10-NEXT: buffer_store_dword v18, v0, s[0:3], 0 offen offset:184
-; GFX10-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen offset:180
-; GFX10-NEXT: buffer_store_dword v16, v0, s[0:3], 0 offen offset:176
-; GFX10-NEXT: buffer_store_dword v15, v0, s[0:3], 0 offen offset:172
-; GFX10-NEXT: buffer_store_dword v14, v0, s[0:3], 0 offen offset:168
-; GFX10-NEXT: buffer_store_dword v13, v0, s[0:3], 0 offen offset:164
-; GFX10-NEXT: buffer_store_dword v12, v0, s[0:3], 0 offen offset:160
-; GFX10-NEXT: buffer_store_dword v11, v0, s[0:3], 0 offen offset:156
-; GFX10-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:152
-; GFX10-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:148
-; GFX10-NEXT: buffer_store_dword v5, v0, s[0:3], 0 offen offset:144
-; GFX10-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:140
-; GFX10-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:136
-; GFX10-NEXT: buffer_store_dword v8, v0, s[0:3], 0 offen offset:132
-; GFX10-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:128
-; GFX10-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:124
-; GFX10-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen
-; GFX10-NEXT: s_setpc_b64 s[30:31]
-;
-; GFX11-LABEL: return_72xi32:
-; GFX11: ; %bb.0:
-; GFX11-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX11-NEXT: s_clause 0xc ; 52-byte Folded Spill
-; GFX11-NEXT: scratch_store_b32 off, v40, s32 offset:212
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v41, s32 offset:208
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v42, s32 offset:204
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v43, s32 offset:200
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v44, s32 offset:196
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v56, s32 offset:192
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v57, s32 offset:188
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v58, s32 offset:184
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v59, s32 offset:180
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v60, s32 offset:176
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v61, s32 offset:172
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v62, s32 offset:168
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v63, s32 offset:164
-; GFX11-NEXT: s_clause 0x11
-; GFX11-NEXT: scratch_load_b32 v36, off, s32 offset:16
-; GFX11-NEXT: scratch_load_b32 v35, off, s32 offset:12
-; GFX11-NEXT: scratch_load_b32 v34, off, s32 offset:8
-; GFX11-NEXT: scratch_load_b32 v51, off, s32 offset:32
-; GFX11-NEXT: scratch_load_b32 v50, off, s32 offset:28
-; GFX11-NEXT: scratch_load_b32 v49, off, s32 offset:24
-; GFX11-NEXT: scratch_load_b32 v55, off, s32 offset:48
-; GFX11-NEXT: scratch_load_b32 v54, off, s32 offset:44
-; GFX11-NEXT: scratch_load_b32 v53, off, s32 offset:40
-; GFX11-NEXT: scratch_load_b32 v40, off, s32 offset:64
-; GFX11-NEXT: scratch_load_b32 v39, off, s32 offset:60
-; GFX11-NEXT: scratch_load_b32 v38, off, s32 offset:56
-; GFX11-NEXT: scratch_load_b32 v44, off, s32 offset:80
-; GFX11-NEXT: scratch_load_b32 v43, off, s32 offset:76
-; GFX11-NEXT: scratch_load_b32 v42, off, s32 offset:72
-; GFX11-NEXT: scratch_load_b32 v59, off, s32 offset:96
-; GFX11-NEXT: scratch_load_b32 v58, off, s32 offset:92
-; GFX11-NEXT: scratch_load_b32 v57, off, s32 offset:88
-; GFX11-NEXT: s_clause 0x1
-; GFX11-NEXT: scratch_store_b128 v0, v[21:24], off offset:80
-; GFX11-NEXT: scratch_store_b128 v0, v[17:20], off offset:64
-; GFX11-NEXT: s_clause 0x5
-; GFX11-NEXT: scratch_load_b32 v23, off, s32 offset:112
-; GFX11-NEXT: scratch_load_b32 v22, off, s32 offset:108
-; GFX11-NEXT: scratch_load_b32 v21, off, s32 offset:104
-; GFX11-NEXT: scratch_load_b32 v19, off, s32 offset:128
-; GFX11-NEXT: scratch_load_b32 v18, off, s32 offset:124
-; GFX11-NEXT: scratch_load_b32 v17, off, s32 offset:120
-; GFX11-NEXT: s_clause 0x1
-; GFX11-NEXT: scratch_store_b128 v0, v[13:16], off offset:48
-; GFX11-NEXT: scratch_store_b128 v0, v[9:12], off offset:32
-; GFX11-NEXT: s_clause 0x10
-; GFX11-NEXT: scratch_load_b32 v15, off, s32 offset:144
-; GFX11-NEXT: scratch_load_b32 v14, off, s32 offset:140
-; GFX11-NEXT: scratch_load_b32 v13, off, s32 offset:136
-; GFX11-NEXT: scratch_load_b32 v63, off, s32 offset:160
-; GFX11-NEXT: scratch_load_b32 v62, off, s32 offset:156
-; GFX11-NEXT: scratch_load_b32 v61, off, s32 offset:152
-; GFX11-NEXT: scratch_load_b32 v60, off, s32 offset:148
-; GFX11-NEXT: scratch_load_b32 v12, off, s32 offset:132
-; GFX11-NEXT: scratch_load_b32 v16, off, s32 offset:116
-; GFX11-NEXT: scratch_load_b32 v20, off, s32 offset:100
-; GFX11-NEXT: scratch_load_b32 v56, off, s32 offset:84
-; GFX11-NEXT: scratch_load_b32 v41, off, s32 offset:68
-; GFX11-NEXT: scratch_load_b32 v37, off, s32 offset:52
-; GFX11-NEXT: scratch_load_b32 v52, off, s32 offset:36
-; GFX11-NEXT: scratch_load_b32 v48, off, s32 offset:20
-; GFX11-NEXT: scratch_load_b32 v33, off, s32 offset:4
-; GFX11-NEXT: scratch_load_b32 v32, off, s32
-; GFX11-NEXT: s_waitcnt vmcnt(10)
-; GFX11-NEXT: scratch_store_b128 v0, v[60:63], off offset:272
-; GFX11-NEXT: s_waitcnt vmcnt(9)
-; GFX11-NEXT: scratch_store_b128 v0, v[12:15], off offset:256
-; GFX11-NEXT: s_waitcnt vmcnt(8)
-; GFX11-NEXT: scratch_store_b128 v0, v[16:19], off offset:240
-; GFX11-NEXT: s_waitcnt vmcnt(7)
-; GFX11-NEXT: scratch_store_b128 v0, v[20:23], off offset:224
-; GFX11-NEXT: s_waitcnt vmcnt(6)
-; GFX11-NEXT: scratch_store_b128 v0, v[56:59], off offset:208
-; GFX11-NEXT: s_waitcnt vmcnt(5)
-; GFX11-NEXT: scratch_store_b128 v0, v[41:44], off offset:192
-; GFX11-NEXT: s_waitcnt vmcnt(4)
-; GFX11-NEXT: scratch_store_b128 v0, v[37:40], off offset:176
-; GFX11-NEXT: s_waitcnt vmcnt(3)
-; GFX11-NEXT: scratch_store_b128 v0, v[52:55], off offset:160
-; GFX11-NEXT: s_waitcnt vmcnt(2)
-; GFX11-NEXT: scratch_store_b128 v0, v[48:51], off offset:144
-; GFX11-NEXT: s_waitcnt vmcnt(1)
-; GFX11-NEXT: scratch_store_b128 v0, v[33:36], off offset:128
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: s_clause 0x3
-; GFX11-NEXT: scratch_store_b128 v0, v[29:32], off offset:112
-; GFX11-NEXT: scratch_store_b128 v0, v[25:28], off offset:96
-; GFX11-NEXT: scratch_store_b128 v0, v[5:8], off offset:16
-; GFX11-NEXT: scratch_store_b128 v0, v[1:4], off
-; GFX11-NEXT: s_clause 0xc ; 52-byte Folded Reload
-; GFX11-NEXT: scratch_load_b32 v63, off, s32 offset:164
-; GFX11-NEXT: scratch_load_b32 v62, off, s32 offset:168
-; GFX11-NEXT: scratch_load_b32 v61, off, s32 offset:172
-; GFX11-NEXT: scratch_load_b32 v60, off, s32 offset:176
-; GFX11-NEXT: scratch_load_b32 v59, off, s32 offset:180
-; GFX11-NEXT: scratch_load_b32 v58, off, s32 offset:184
-; GFX11-NEXT: scratch_load_b32 v57, off, s32 offset:188
-; GFX11-NEXT: scratch_load_b32 v56, off, s32 offset:192
-; GFX11-NEXT: scratch_load_b32 v44, off, s32 offset:196
-; GFX11-NEXT: scratch_load_b32 v43, off, s32 offset:200
-; GFX11-NEXT: scratch_load_b32 v42, off, s32 offset:204
-; GFX11-NEXT: scratch_load_b32 v41, off, s32 offset:208
-; GFX11-NEXT: scratch_load_b32 v40, off, s32 offset:212
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: s_setpc_b64 s[30:31]
- ret <72 x i32> %val
-}
-
-define amdgpu_gfx void @call_72xi32() #1 {
-; GFX9-LABEL: call_72xi32:
-; GFX9: ; %bb.0: ; %entry
-; GFX9-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX9-NEXT: s_mov_b32 s35, s33
-; GFX9-NEXT: s_add_i32 s33, s32, 0x7fc0
-; GFX9-NEXT: s_and_b32 s33, s33, 0xffff8000
-; GFX9-NEXT: s_or_saveexec_b64 s[36:37], -1
-; GFX9-NEXT: buffer_store_dword v63, off, s[0:3], s33 offset:1564 ; 4-byte Folded Spill
-; GFX9-NEXT: s_mov_b64 exec, s[36:37]
-; GFX9-NEXT: s_mov_b32 s38, s34
-; GFX9-NEXT: s_mov_b32 s34, s32
-; GFX9-NEXT: s_add_i32 s32, s32, 0x28000
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s33 ; 4-byte Folded Spill
-; GFX9-NEXT: v_writelane_b32 v63, s30, 0
-; GFX9-NEXT: v_writelane_b32 v63, s31, 1
-; GFX9-NEXT: v_mov_b32_e32 v0, 0
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:20
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:36
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:40
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:44
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:160
-; GFX9-NEXT: v_lshrrev_b32_e64 v0, 6, s33
-; GFX9-NEXT: s_mov_b32 s37, return_72xi32 at abs32@hi
-; GFX9-NEXT: s_mov_b32 s36, return_72xi32 at abs32@lo
-; GFX9-NEXT: v_add_u32_e32 v0, 0x200, v0
-; GFX9-NEXT: v_mov_b32_e32 v1, 0
-; GFX9-NEXT: v_mov_b32_e32 v2, 0
-; GFX9-NEXT: v_mov_b32_e32 v3, 0
-; GFX9-NEXT: v_mov_b32_e32 v4, 0
-; GFX9-NEXT: v_mov_b32_e32 v5, 0
-; GFX9-NEXT: v_mov_b32_e32 v6, 0
-; GFX9-NEXT: v_mov_b32_e32 v7, 0
-; GFX9-NEXT: v_mov_b32_e32 v8, 0
-; GFX9-NEXT: v_mov_b32_e32 v9, 0
-; GFX9-NEXT: v_mov_b32_e32 v10, 0
-; GFX9-NEXT: v_mov_b32_e32 v11, 0
-; GFX9-NEXT: v_mov_b32_e32 v12, 0
-; GFX9-NEXT: v_mov_b32_e32 v13, 0
-; GFX9-NEXT: v_mov_b32_e32 v14, 0
-; GFX9-NEXT: v_mov_b32_e32 v15, 0
-; GFX9-NEXT: v_mov_b32_e32 v16, 0
-; GFX9-NEXT: v_mov_b32_e32 v17, 0
-; GFX9-NEXT: v_mov_b32_e32 v18, 0
-; GFX9-NEXT: v_mov_b32_e32 v19, 0
-; GFX9-NEXT: v_mov_b32_e32 v20, 0
-; GFX9-NEXT: v_mov_b32_e32 v21, 0
-; GFX9-NEXT: v_mov_b32_e32 v22, 0
-; GFX9-NEXT: v_mov_b32_e32 v23, 0
-; GFX9-NEXT: v_mov_b32_e32 v24, 0
-; GFX9-NEXT: v_mov_b32_e32 v25, 0
-; GFX9-NEXT: v_mov_b32_e32 v26, 0
-; GFX9-NEXT: v_mov_b32_e32 v27, 0
-; GFX9-NEXT: v_mov_b32_e32 v28, 0
-; GFX9-NEXT: v_mov_b32_e32 v29, 0
-; GFX9-NEXT: v_mov_b32_e32 v30, 0
-; GFX9-NEXT: v_mov_b32_e32 v31, 0
-; GFX9-NEXT: s_swappc_b64 s[30:31], s[36:37]
-; GFX9-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:636
-; GFX9-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:640
-; GFX9-NEXT: buffer_load_dword v32, off, s[0:3], s33 offset:644
-; GFX9-NEXT: buffer_load_dword v33, off, s[0:3], s33 offset:648
-; GFX9-NEXT: buffer_load_dword v34, off, s[0:3], s33 offset:652
-; GFX9-NEXT: buffer_load_dword v35, off, s[0:3], s33 offset:656
-; GFX9-NEXT: buffer_load_dword v36, off, s[0:3], s33 offset:660
-; GFX9-NEXT: buffer_load_dword v37, off, s[0:3], s33 offset:664
-; GFX9-NEXT: buffer_load_dword v38, off, s[0:3], s33 offset:668
-; GFX9-NEXT: buffer_load_dword v39, off, s[0:3], s33 offset:672
-; GFX9-NEXT: buffer_load_dword v48, off, s[0:3], s33 offset:676
-; GFX9-NEXT: buffer_load_dword v49, off, s[0:3], s33 offset:680
-; GFX9-NEXT: buffer_load_dword v50, off, s[0:3], s33 offset:684
-; GFX9-NEXT: buffer_load_dword v51, off, s[0:3], s33 offset:688
-; GFX9-NEXT: buffer_load_dword v52, off, s[0:3], s33 offset:692
-; GFX9-NEXT: buffer_load_dword v53, off, s[0:3], s33 offset:696
-; GFX9-NEXT: buffer_load_dword v54, off, s[0:3], s33 offset:700
-; GFX9-NEXT: buffer_load_dword v55, off, s[0:3], s33 offset:704
-; GFX9-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:708
-; GFX9-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:712
-; GFX9-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:716
-; GFX9-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:720
-; GFX9-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:724
-; GFX9-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:728
-; GFX9-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:732
-; GFX9-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:736
-; GFX9-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:740
-; GFX9-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:748
-; GFX9-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:752
-; GFX9-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:756
-; GFX9-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:760
-; GFX9-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:764
-; GFX9-NEXT: buffer_load_dword v62, off, s[0:3], s33 offset:768
-; GFX9-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:772
-; GFX9-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:776
-; GFX9-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:780
-; GFX9-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:784
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:788
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:792
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:796
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:516
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1536 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:520
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1540 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:524
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1544 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:528
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1548 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:532
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1552 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:536
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1556 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:540
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: buffer_store_dword v9, off, s[0:3], s33 offset:1560 ; 4-byte Folded Spill
-; GFX9-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:544
-; GFX9-NEXT: s_nop 0
-; GFX9-NEXT: buffer_load_dword v10, off, s[0:3], s33 offset:548
-; GFX9-NEXT: buffer_load_dword v11, off, s[0:3], s33 offset:552
-; GFX9-NEXT: buffer_load_dword v12, off, s[0:3], s33 offset:556
-; GFX9-NEXT: buffer_load_dword v13, off, s[0:3], s33 offset:560
-; GFX9-NEXT: buffer_load_dword v14, off, s[0:3], s33 offset:564
-; GFX9-NEXT: buffer_load_dword v15, off, s[0:3], s33 offset:568
-; GFX9-NEXT: buffer_load_dword v16, off, s[0:3], s33 offset:572
-; GFX9-NEXT: buffer_load_dword v17, off, s[0:3], s33 offset:576
-; GFX9-NEXT: buffer_load_dword v18, off, s[0:3], s33 offset:580
-; GFX9-NEXT: buffer_load_dword v19, off, s[0:3], s33 offset:584
-; GFX9-NEXT: buffer_load_dword v20, off, s[0:3], s33 offset:588
-; GFX9-NEXT: buffer_load_dword v21, off, s[0:3], s33 offset:592
-; GFX9-NEXT: buffer_load_dword v22, off, s[0:3], s33 offset:596
-; GFX9-NEXT: buffer_load_dword v23, off, s[0:3], s33 offset:600
-; GFX9-NEXT: buffer_load_dword v24, off, s[0:3], s33 offset:604
-; GFX9-NEXT: buffer_load_dword v25, off, s[0:3], s33 offset:608
-; GFX9-NEXT: buffer_load_dword v26, off, s[0:3], s33 offset:612
-; GFX9-NEXT: buffer_load_dword v27, off, s[0:3], s33 offset:616
-; GFX9-NEXT: buffer_load_dword v28, off, s[0:3], s33 offset:620
-; GFX9-NEXT: buffer_load_dword v29, off, s[0:3], s33 offset:624
-; GFX9-NEXT: buffer_load_dword v30, off, s[0:3], s33 offset:628
-; GFX9-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:632
-; GFX9-NEXT: s_nop 0
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32
-; GFX9-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:4
-; GFX9-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:8
-; GFX9-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:12
-; GFX9-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:16
-; GFX9-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:20
-; GFX9-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:24
-; GFX9-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:28
-; GFX9-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:32
-; GFX9-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:36
-; GFX9-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:40
-; GFX9-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:44
-; GFX9-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:48
-; GFX9-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:52
-; GFX9-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:56
-; GFX9-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:60
-; GFX9-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:64
-; GFX9-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:68
-; GFX9-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72
-; GFX9-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:76
-; GFX9-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:80
-; GFX9-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:84
-; GFX9-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:88
-; GFX9-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:92
-; GFX9-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:96
-; GFX9-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:100
-; GFX9-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:104
-; GFX9-NEXT: v_mov_b32_e32 v0, 24
-; GFX9-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
-; GFX9-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:112
-; GFX9-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:116
-; GFX9-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:120
-; GFX9-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:124
-; GFX9-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:128
-; GFX9-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:132
-; GFX9-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136
-; GFX9-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:140
-; GFX9-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:144
-; GFX9-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:148
-; GFX9-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:152
-; GFX9-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:156
-; GFX9-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:160
-; GFX9-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:1536 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:1540 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:1544 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:1548 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:1552 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:1556 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:1560 ; 4-byte Folded Reload
-; GFX9-NEXT: v_lshrrev_b32_e64 v0, 6, s33
-; GFX9-NEXT: v_add_u32_e32 v0, 0x400, v0
-; GFX9-NEXT: v_mov_b32_e32 v1, 42
-; GFX9-NEXT: s_swappc_b64 s[30:31], s[36:37]
-; GFX9-NEXT: buffer_load_dword v62, off, s[0:3], s33 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Reload
-; GFX9-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Reload
-; GFX9-NEXT: v_readlane_b32 s30, v63, 0
-; GFX9-NEXT: v_readlane_b32 s31, v63, 1
-; GFX9-NEXT: s_mov_b32 s32, s34
-; GFX9-NEXT: s_mov_b32 s34, s38
-; GFX9-NEXT: s_or_saveexec_b64 s[36:37], -1
-; GFX9-NEXT: buffer_load_dword v63, off, s[0:3], s33 offset:1564 ; 4-byte Folded Reload
-; GFX9-NEXT: s_mov_b64 exec, s[36:37]
-; GFX9-NEXT: s_mov_b32 s33, s35
-; GFX9-NEXT: s_waitcnt vmcnt(0)
-; GFX9-NEXT: s_setpc_b64 s[30:31]
-;
-; GFX10-LABEL: call_72xi32:
-; GFX10: ; %bb.0: ; %entry
-; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX10-NEXT: s_mov_b32 s35, s33
-; GFX10-NEXT: s_add_i32 s33, s32, 0x3fe0
-; GFX10-NEXT: s_and_b32 s33, s33, 0xffffc000
-; GFX10-NEXT: s_or_saveexec_b32 s36, -1
-; GFX10-NEXT: buffer_store_dword v63, off, s[0:3], s33 offset:1568 ; 4-byte Folded Spill
-; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
-; GFX10-NEXT: s_mov_b32 exec_lo, s36
-; GFX10-NEXT: s_mov_b32 s38, s34
-; GFX10-NEXT: s_mov_b32 s34, s32
-; GFX10-NEXT: s_add_i32 s32, s32, 0x14000
-; GFX10-NEXT: buffer_store_dword v40, off, s[0:3], s33 offset:56 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v41, off, s[0:3], s33 offset:52 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v42, off, s[0:3], s33 offset:48 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v43, off, s[0:3], s33 offset:44 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v44, off, s[0:3], s33 offset:40 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v45, off, s[0:3], s33 offset:36 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v46, off, s[0:3], s33 offset:32 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v47, off, s[0:3], s33 offset:28 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v56, off, s[0:3], s33 offset:24 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v57, off, s[0:3], s33 offset:20 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v58, off, s[0:3], s33 offset:16 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v59, off, s[0:3], s33 offset:12 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v60, off, s[0:3], s33 offset:8 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v61, off, s[0:3], s33 offset:4 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_store_dword v62, off, s[0:3], s33 ; 4-byte Folded Spill
-; GFX10-NEXT: v_writelane_b32 v63, s30, 0
-; GFX10-NEXT: v_writelane_b32 v63, s31, 1
-; GFX10-NEXT: v_mov_b32_e32 v0, 0
-; GFX10-NEXT: v_mov_b32_e32 v1, 0
-; GFX10-NEXT: v_mov_b32_e32 v2, 0
-; GFX10-NEXT: v_mov_b32_e32 v3, 0
-; GFX10-NEXT: v_mov_b32_e32 v4, 0
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:4
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:8
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:12
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:16
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:20
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:28
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:32
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:36
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:40
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:44
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:48
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:52
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:84
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:88
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:92
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:100
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:104
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:112
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:116
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:120
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:124
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:128
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:132
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:136
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:140
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:144
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:148
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:152
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:156
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:160
-; GFX10-NEXT: v_lshrrev_b32_e64 v0, 5, s33
-; GFX10-NEXT: v_mov_b32_e32 v5, 0
-; GFX10-NEXT: v_mov_b32_e32 v6, 0
-; GFX10-NEXT: v_mov_b32_e32 v7, 0
-; GFX10-NEXT: v_mov_b32_e32 v8, 0
-; GFX10-NEXT: v_add_nc_u32_e32 v0, 0x200, v0
-; GFX10-NEXT: v_mov_b32_e32 v9, 0
-; GFX10-NEXT: v_mov_b32_e32 v10, 0
-; GFX10-NEXT: v_mov_b32_e32 v11, 0
-; GFX10-NEXT: v_mov_b32_e32 v12, 0
-; GFX10-NEXT: v_mov_b32_e32 v13, 0
-; GFX10-NEXT: v_mov_b32_e32 v14, 0
-; GFX10-NEXT: v_mov_b32_e32 v15, 0
-; GFX10-NEXT: v_mov_b32_e32 v16, 0
-; GFX10-NEXT: v_mov_b32_e32 v17, 0
-; GFX10-NEXT: v_mov_b32_e32 v18, 0
-; GFX10-NEXT: v_mov_b32_e32 v19, 0
-; GFX10-NEXT: v_mov_b32_e32 v20, 0
-; GFX10-NEXT: v_mov_b32_e32 v21, 0
-; GFX10-NEXT: v_mov_b32_e32 v22, 0
-; GFX10-NEXT: v_mov_b32_e32 v23, 0
-; GFX10-NEXT: v_mov_b32_e32 v24, 0
-; GFX10-NEXT: v_mov_b32_e32 v25, 0
-; GFX10-NEXT: v_mov_b32_e32 v26, 0
-; GFX10-NEXT: v_mov_b32_e32 v27, 0
-; GFX10-NEXT: v_mov_b32_e32 v28, 0
-; GFX10-NEXT: v_mov_b32_e32 v29, 0
-; GFX10-NEXT: v_mov_b32_e32 v30, 0
-; GFX10-NEXT: v_mov_b32_e32 v31, 0
-; GFX10-NEXT: s_mov_b32 s37, return_72xi32 at abs32@hi
-; GFX10-NEXT: s_mov_b32 s36, return_72xi32 at abs32@lo
-; GFX10-NEXT: s_swappc_b64 s[30:31], s[36:37]
-; GFX10-NEXT: s_clause 0x28
-; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:636
-; GFX10-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:640
-; GFX10-NEXT: buffer_load_dword v32, off, s[0:3], s33 offset:644
-; GFX10-NEXT: buffer_load_dword v33, off, s[0:3], s33 offset:648
-; GFX10-NEXT: buffer_load_dword v34, off, s[0:3], s33 offset:652
-; GFX10-NEXT: buffer_load_dword v35, off, s[0:3], s33 offset:656
-; GFX10-NEXT: buffer_load_dword v36, off, s[0:3], s33 offset:660
-; GFX10-NEXT: buffer_load_dword v37, off, s[0:3], s33 offset:664
-; GFX10-NEXT: buffer_load_dword v38, off, s[0:3], s33 offset:668
-; GFX10-NEXT: buffer_load_dword v39, off, s[0:3], s33 offset:672
-; GFX10-NEXT: buffer_load_dword v48, off, s[0:3], s33 offset:676
-; GFX10-NEXT: buffer_load_dword v49, off, s[0:3], s33 offset:680
-; GFX10-NEXT: buffer_load_dword v50, off, s[0:3], s33 offset:684
-; GFX10-NEXT: buffer_load_dword v51, off, s[0:3], s33 offset:688
-; GFX10-NEXT: buffer_load_dword v52, off, s[0:3], s33 offset:692
-; GFX10-NEXT: buffer_load_dword v53, off, s[0:3], s33 offset:696
-; GFX10-NEXT: buffer_load_dword v54, off, s[0:3], s33 offset:700
-; GFX10-NEXT: buffer_load_dword v55, off, s[0:3], s33 offset:704
-; GFX10-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:708
-; GFX10-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:712
-; GFX10-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:716
-; GFX10-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:720
-; GFX10-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:724
-; GFX10-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:728
-; GFX10-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:732
-; GFX10-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:736
-; GFX10-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:740
-; GFX10-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:748
-; GFX10-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:752
-; GFX10-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:756
-; GFX10-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:760
-; GFX10-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:764
-; GFX10-NEXT: buffer_load_dword v62, off, s[0:3], s33 offset:768
-; GFX10-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:772
-; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:776
-; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:780
-; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:784
-; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:788
-; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:792
-; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:796
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:516
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1536 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:520
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1540 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:524
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1544 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:528
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1548 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:532
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1552 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:536
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1556 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:540
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1560 ; 4-byte Folded Spill
-; GFX10-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:544
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s33 offset:1564 ; 4-byte Folded Spill
-; GFX10-NEXT: s_clause 0x15
-; GFX10-NEXT: buffer_load_dword v10, off, s[0:3], s33 offset:548
-; GFX10-NEXT: buffer_load_dword v11, off, s[0:3], s33 offset:552
-; GFX10-NEXT: buffer_load_dword v12, off, s[0:3], s33 offset:556
-; GFX10-NEXT: buffer_load_dword v13, off, s[0:3], s33 offset:560
-; GFX10-NEXT: buffer_load_dword v14, off, s[0:3], s33 offset:564
-; GFX10-NEXT: buffer_load_dword v15, off, s[0:3], s33 offset:568
-; GFX10-NEXT: buffer_load_dword v16, off, s[0:3], s33 offset:572
-; GFX10-NEXT: buffer_load_dword v17, off, s[0:3], s33 offset:576
-; GFX10-NEXT: buffer_load_dword v18, off, s[0:3], s33 offset:580
-; GFX10-NEXT: buffer_load_dword v19, off, s[0:3], s33 offset:584
-; GFX10-NEXT: buffer_load_dword v20, off, s[0:3], s33 offset:588
-; GFX10-NEXT: buffer_load_dword v21, off, s[0:3], s33 offset:592
-; GFX10-NEXT: buffer_load_dword v22, off, s[0:3], s33 offset:596
-; GFX10-NEXT: buffer_load_dword v23, off, s[0:3], s33 offset:600
-; GFX10-NEXT: buffer_load_dword v24, off, s[0:3], s33 offset:604
-; GFX10-NEXT: buffer_load_dword v25, off, s[0:3], s33 offset:608
-; GFX10-NEXT: buffer_load_dword v26, off, s[0:3], s33 offset:612
-; GFX10-NEXT: buffer_load_dword v27, off, s[0:3], s33 offset:616
-; GFX10-NEXT: buffer_load_dword v28, off, s[0:3], s33 offset:620
-; GFX10-NEXT: buffer_load_dword v29, off, s[0:3], s33 offset:624
-; GFX10-NEXT: buffer_load_dword v30, off, s[0:3], s33 offset:628
-; GFX10-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:632
-; GFX10-NEXT: v_mov_b32_e32 v0, 24
-; GFX10-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:108
-; GFX10-NEXT: buffer_store_dword v9, off, s[0:3], s32
-; GFX10-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:4
-; GFX10-NEXT: buffer_store_dword v32, off, s[0:3], s32 offset:8
-; GFX10-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:12
-; GFX10-NEXT: buffer_store_dword v34, off, s[0:3], s32 offset:16
-; GFX10-NEXT: buffer_store_dword v35, off, s[0:3], s32 offset:20
-; GFX10-NEXT: buffer_store_dword v36, off, s[0:3], s32 offset:24
-; GFX10-NEXT: buffer_store_dword v37, off, s[0:3], s32 offset:28
-; GFX10-NEXT: buffer_store_dword v38, off, s[0:3], s32 offset:32
-; GFX10-NEXT: buffer_store_dword v39, off, s[0:3], s32 offset:36
-; GFX10-NEXT: buffer_store_dword v48, off, s[0:3], s32 offset:40
-; GFX10-NEXT: buffer_store_dword v49, off, s[0:3], s32 offset:44
-; GFX10-NEXT: buffer_store_dword v50, off, s[0:3], s32 offset:48
-; GFX10-NEXT: buffer_store_dword v51, off, s[0:3], s32 offset:52
-; GFX10-NEXT: buffer_store_dword v52, off, s[0:3], s32 offset:56
-; GFX10-NEXT: buffer_store_dword v53, off, s[0:3], s32 offset:60
-; GFX10-NEXT: buffer_store_dword v54, off, s[0:3], s32 offset:64
-; GFX10-NEXT: buffer_store_dword v55, off, s[0:3], s32 offset:68
-; GFX10-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72
-; GFX10-NEXT: buffer_store_dword v41, off, s[0:3], s32 offset:76
-; GFX10-NEXT: buffer_store_dword v42, off, s[0:3], s32 offset:80
-; GFX10-NEXT: buffer_store_dword v43, off, s[0:3], s32 offset:84
-; GFX10-NEXT: buffer_store_dword v44, off, s[0:3], s32 offset:88
-; GFX10-NEXT: buffer_store_dword v45, off, s[0:3], s32 offset:92
-; GFX10-NEXT: buffer_store_dword v46, off, s[0:3], s32 offset:96
-; GFX10-NEXT: buffer_store_dword v47, off, s[0:3], s32 offset:100
-; GFX10-NEXT: buffer_store_dword v56, off, s[0:3], s32 offset:104
-; GFX10-NEXT: buffer_store_dword v57, off, s[0:3], s32 offset:112
-; GFX10-NEXT: buffer_store_dword v58, off, s[0:3], s32 offset:116
-; GFX10-NEXT: buffer_store_dword v59, off, s[0:3], s32 offset:120
-; GFX10-NEXT: buffer_store_dword v60, off, s[0:3], s32 offset:124
-; GFX10-NEXT: buffer_store_dword v61, off, s[0:3], s32 offset:128
-; GFX10-NEXT: buffer_store_dword v62, off, s[0:3], s32 offset:132
-; GFX10-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:136
-; GFX10-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:140
-; GFX10-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:144
-; GFX10-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:148
-; GFX10-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:152
-; GFX10-NEXT: buffer_store_dword v7, off, s[0:3], s32 offset:156
-; GFX10-NEXT: buffer_store_dword v8, off, s[0:3], s32 offset:160
-; GFX10-NEXT: s_clause 0x7 ; 32-byte Folded Reload
-; GFX10-NEXT: buffer_load_dword v2, off, s[0:3], s33 offset:1536
-; GFX10-NEXT: buffer_load_dword v3, off, s[0:3], s33 offset:1540
-; GFX10-NEXT: buffer_load_dword v4, off, s[0:3], s33 offset:1544
-; GFX10-NEXT: buffer_load_dword v5, off, s[0:3], s33 offset:1548
-; GFX10-NEXT: buffer_load_dword v6, off, s[0:3], s33 offset:1552
-; GFX10-NEXT: buffer_load_dword v7, off, s[0:3], s33 offset:1556
-; GFX10-NEXT: buffer_load_dword v8, off, s[0:3], s33 offset:1560
-; GFX10-NEXT: buffer_load_dword v9, off, s[0:3], s33 offset:1564
-; GFX10-NEXT: v_lshrrev_b32_e64 v0, 5, s33
-; GFX10-NEXT: v_mov_b32_e32 v1, 42
-; GFX10-NEXT: v_add_nc_u32_e32 v0, 0x400, v0
-; GFX10-NEXT: s_swappc_b64 s[30:31], s[36:37]
-; GFX10-NEXT: s_clause 0xe ; 60-byte Folded Reload
-; GFX10-NEXT: buffer_load_dword v62, off, s[0:3], s33
-; GFX10-NEXT: buffer_load_dword v61, off, s[0:3], s33 offset:4
-; GFX10-NEXT: buffer_load_dword v60, off, s[0:3], s33 offset:8
-; GFX10-NEXT: buffer_load_dword v59, off, s[0:3], s33 offset:12
-; GFX10-NEXT: buffer_load_dword v58, off, s[0:3], s33 offset:16
-; GFX10-NEXT: buffer_load_dword v57, off, s[0:3], s33 offset:20
-; GFX10-NEXT: buffer_load_dword v56, off, s[0:3], s33 offset:24
-; GFX10-NEXT: buffer_load_dword v47, off, s[0:3], s33 offset:28
-; GFX10-NEXT: buffer_load_dword v46, off, s[0:3], s33 offset:32
-; GFX10-NEXT: buffer_load_dword v45, off, s[0:3], s33 offset:36
-; GFX10-NEXT: buffer_load_dword v44, off, s[0:3], s33 offset:40
-; GFX10-NEXT: buffer_load_dword v43, off, s[0:3], s33 offset:44
-; GFX10-NEXT: buffer_load_dword v42, off, s[0:3], s33 offset:48
-; GFX10-NEXT: buffer_load_dword v41, off, s[0:3], s33 offset:52
-; GFX10-NEXT: buffer_load_dword v40, off, s[0:3], s33 offset:56
-; GFX10-NEXT: v_readlane_b32 s30, v63, 0
-; GFX10-NEXT: v_readlane_b32 s31, v63, 1
-; GFX10-NEXT: s_mov_b32 s32, s34
-; GFX10-NEXT: s_mov_b32 s34, s38
-; GFX10-NEXT: s_or_saveexec_b32 s36, -1
-; GFX10-NEXT: buffer_load_dword v63, off, s[0:3], s33 offset:1568 ; 4-byte Folded Reload
-; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
-; GFX10-NEXT: s_mov_b32 exec_lo, s36
-; GFX10-NEXT: s_mov_b32 s33, s35
-; GFX10-NEXT: s_waitcnt vmcnt(0)
-; GFX10-NEXT: s_setpc_b64 s[30:31]
-;
-; GFX11-LABEL: call_72xi32:
-; GFX11: ; %bb.0: ; %entry
-; GFX11-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; GFX11-NEXT: s_mov_b32 s38, s33
-; GFX11-NEXT: s_add_i32 s33, s32, 0x1ff
-; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
-; GFX11-NEXT: s_and_b32 s33, s33, 0xfffffe00
-; GFX11-NEXT: s_or_saveexec_b32 s0, -1
-; GFX11-NEXT: scratch_store_b32 off, v62, s33 offset:1600 ; 4-byte Folded Spill
-; GFX11-NEXT: s_mov_b32 exec_lo, s0
-; GFX11-NEXT: s_mov_b32 s39, s34
-; GFX11-NEXT: s_mov_b32 s34, s32
-; GFX11-NEXT: s_addk_i32 s32, 0xa00
-; GFX11-NEXT: s_clause 0xd ; 56-byte Folded Spill
-; GFX11-NEXT: scratch_store_b32 off, v40, s33 offset:52
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v41, s33 offset:48
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v42, s33 offset:44
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v43, s33 offset:40
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v44, s33 offset:36
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v45, s33 offset:32
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v46, s33 offset:28
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v47, s33 offset:24
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v56, s33 offset:20
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v57, s33 offset:16
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v58, s33 offset:12
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v59, s33 offset:8
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v60, s33 offset:4
-; GFX11-NEXT: ; meta instruction
-; GFX11-NEXT: scratch_store_b32 off, v61, s33
-; GFX11-NEXT: v_writelane_b32 v62, s30, 0
-; GFX11-NEXT: v_writelane_b32 v62, s31, 1
-; GFX11-NEXT: s_mov_b32 s0, 0
-; GFX11-NEXT: v_mov_b32_e32 v4, 0
-; GFX11-NEXT: s_mov_b32 s1, s0
-; GFX11-NEXT: s_mov_b32 s2, s0
-; GFX11-NEXT: s_mov_b32 s3, s0
-; GFX11-NEXT: v_dual_mov_b32 v0, s0 :: v_dual_mov_b32 v1, s1
-; GFX11-NEXT: v_dual_mov_b32 v2, s2 :: v_dual_mov_b32 v3, s3
-; GFX11-NEXT: s_add_i32 s0, s32, 0xa0
-; GFX11-NEXT: s_add_i32 s1, s32, 0x90
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s32
-; GFX11-NEXT: scratch_store_b32 off, v4, s0
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
-; GFX11-NEXT: s_add_i32 s0, s32, 0x80
-; GFX11-NEXT: s_add_i32 s1, s32, 0x70
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
-; GFX11-NEXT: s_add_i32 s0, s32, 0x60
-; GFX11-NEXT: s_add_i32 s1, s32, 0x50
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
-; GFX11-NEXT: s_add_i32 s0, s32, 64
-; GFX11-NEXT: s_add_i32 s1, s32, 48
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
-; GFX11-NEXT: s_add_i32 s0, s32, 32
-; GFX11-NEXT: s_add_i32 s1, s32, 16
-; GFX11-NEXT: s_add_i32 s2, s33, 0x200
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s0
-; GFX11-NEXT: scratch_store_b128 off, v[0:3], s1
-; GFX11-NEXT: v_dual_mov_b32 v0, s2 :: v_dual_mov_b32 v3, 0
-; GFX11-NEXT: v_dual_mov_b32 v1, 0 :: v_dual_mov_b32 v2, 0
-; GFX11-NEXT: v_dual_mov_b32 v5, 0 :: v_dual_mov_b32 v4, 0
-; GFX11-NEXT: v_dual_mov_b32 v7, 0 :: v_dual_mov_b32 v6, 0
-; GFX11-NEXT: v_dual_mov_b32 v9, 0 :: v_dual_mov_b32 v8, 0
-; GFX11-NEXT: v_dual_mov_b32 v11, 0 :: v_dual_mov_b32 v10, 0
-; GFX11-NEXT: v_dual_mov_b32 v13, 0 :: v_dual_mov_b32 v12, 0
-; GFX11-NEXT: v_dual_mov_b32 v15, 0 :: v_dual_mov_b32 v14, 0
-; GFX11-NEXT: v_dual_mov_b32 v17, 0 :: v_dual_mov_b32 v16, 0
-; GFX11-NEXT: v_dual_mov_b32 v19, 0 :: v_dual_mov_b32 v18, 0
-; GFX11-NEXT: v_dual_mov_b32 v21, 0 :: v_dual_mov_b32 v20, 0
-; GFX11-NEXT: v_dual_mov_b32 v23, 0 :: v_dual_mov_b32 v22, 0
-; GFX11-NEXT: v_dual_mov_b32 v25, 0 :: v_dual_mov_b32 v24, 0
-; GFX11-NEXT: v_dual_mov_b32 v27, 0 :: v_dual_mov_b32 v26, 0
-; GFX11-NEXT: v_dual_mov_b32 v29, 0 :: v_dual_mov_b32 v28, 0
-; GFX11-NEXT: v_dual_mov_b32 v31, 0 :: v_dual_mov_b32 v30, 0
-; GFX11-NEXT: s_mov_b32 s1, return_72xi32 at abs32@hi
-; GFX11-NEXT: s_mov_b32 s0, return_72xi32 at abs32@lo
-; GFX11-NEXT: s_delay_alu instid0(SALU_CYCLE_1)
-; GFX11-NEXT: s_swappc_b64 s[30:31], s[0:1]
-; GFX11-NEXT: s_clause 0xb
-; GFX11-NEXT: scratch_load_b128 v[43:46], off, s33 offset:624
-; GFX11-NEXT: scratch_load_b128 v[47:50], off, s33 offset:640
-; GFX11-NEXT: scratch_load_b128 v[16:19], off, s33 offset:784
-; GFX11-NEXT: scratch_load_b128 v[12:15], off, s33 offset:768
-; GFX11-NEXT: scratch_load_b128 v[8:11], off, s33 offset:752
-; GFX11-NEXT: scratch_load_b128 v[4:7], off, s33 offset:736
-; GFX11-NEXT: scratch_load_b128 v[0:3], off, s33 offset:720
-; GFX11-NEXT: scratch_load_b128 v[24:27], off, s33 offset:656
-; GFX11-NEXT: scratch_load_b128 v[36:39], off, s33 offset:704
-; GFX11-NEXT: scratch_load_b128 v[32:35], off, s33 offset:688
-; GFX11-NEXT: scratch_load_b128 v[28:31], off, s33 offset:672
-; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:512
-; GFX11-NEXT: s_waitcnt vmcnt(6)
-; GFX11-NEXT: v_mov_b32_e32 v6, 24
-; GFX11-NEXT: s_add_i32 s2, s32, 0xa0
-; GFX11-NEXT: s_add_i32 s3, s32, 0x90
-; GFX11-NEXT: s_add_i32 s35, s32, 0x80
-; GFX11-NEXT: s_add_i32 s36, s32, 0x70
-; GFX11-NEXT: s_add_i32 s37, s32, 0x6c
-; GFX11-NEXT: s_waitcnt vmcnt(4)
-; GFX11-NEXT: v_dual_mov_b32 v40, v0 :: v_dual_mov_b32 v53, v26
-; GFX11-NEXT: v_dual_mov_b32 v41, v1 :: v_dual_mov_b32 v42, v2
-; GFX11-NEXT: v_mov_b32_e32 v51, v24
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1584 ; 16-byte Folded Spill
-; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:528
-; GFX11-NEXT: v_mov_b32_e32 v52, v25
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1536 ; 16-byte Folded Spill
-; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:544
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1568 ; 16-byte Folded Spill
-; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:560
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: scratch_store_b128 off, v[20:23], s33 offset:1552 ; 16-byte Folded Spill
-; GFX11-NEXT: s_clause 0x2
-; GFX11-NEXT: scratch_load_b128 v[20:23], off, s33 offset:576
-; GFX11-NEXT: scratch_load_b128 v[58:61], off, s33 offset:592
-; GFX11-NEXT: scratch_load_b128 v[54:57], off, s33 offset:608
-; GFX11-NEXT: scratch_store_b128 off, v[46:49], s32
-; GFX11-NEXT: scratch_store_b32 off, v19, s2
-; GFX11-NEXT: scratch_store_b128 off, v[15:18], s3
-; GFX11-NEXT: scratch_store_b128 off, v[11:14], s35
-; GFX11-NEXT: scratch_store_b128 off, v[7:10], s36
-; GFX11-NEXT: s_add_i32 s2, s32, 0x60
-; GFX11-NEXT: scratch_store_b32 off, v6, s37
-; GFX11-NEXT: scratch_store_b96 off, v[3:5], s2
-; GFX11-NEXT: s_add_i32 s2, s32, 0x50
-; GFX11-NEXT: s_add_i32 s3, s32, 64
-; GFX11-NEXT: s_add_i32 s35, s32, 48
-; GFX11-NEXT: s_add_i32 s36, s32, 32
-; GFX11-NEXT: scratch_store_b128 off, v[39:42], s2
-; GFX11-NEXT: scratch_store_b128 off, v[35:38], s3
-; GFX11-NEXT: scratch_store_b128 off, v[31:34], s35
-; GFX11-NEXT: scratch_store_b128 off, v[27:30], s36
-; GFX11-NEXT: s_add_i32 s2, s32, 16
-; GFX11-NEXT: v_mov_b32_e32 v29, v43
-; GFX11-NEXT: scratch_store_b128 off, v[50:53], s2
-; GFX11-NEXT: s_clause 0x3 ; 64-byte Folded Reload
-; GFX11-NEXT: scratch_load_b128 v[1:4], off, s33 offset:1584
-; GFX11-NEXT: scratch_load_b128 v[5:8], off, s33 offset:1536
-; GFX11-NEXT: scratch_load_b128 v[9:12], off, s33 offset:1568
-; GFX11-NEXT: scratch_load_b128 v[13:16], off, s33 offset:1552
-; GFX11-NEXT: s_add_i32 s2, s33, 0x400
-; GFX11-NEXT: v_dual_mov_b32 v30, v44 :: v_dual_mov_b32 v31, v45
-; GFX11-NEXT: s_waitcnt vmcnt(3)
-; GFX11-NEXT: v_dual_mov_b32 v0, s2 :: v_dual_mov_b32 v1, 42
-; GFX11-NEXT: v_dual_mov_b32 v17, v20 :: v_dual_mov_b32 v18, v21
-; GFX11-NEXT: v_dual_mov_b32 v19, v22 :: v_dual_mov_b32 v20, v23
-; GFX11-NEXT: v_dual_mov_b32 v21, v58 :: v_dual_mov_b32 v22, v59
-; GFX11-NEXT: v_dual_mov_b32 v23, v60 :: v_dual_mov_b32 v24, v61
-; GFX11-NEXT: v_dual_mov_b32 v25, v54 :: v_dual_mov_b32 v26, v55
-; GFX11-NEXT: v_dual_mov_b32 v27, v56 :: v_dual_mov_b32 v28, v57
-; GFX11-NEXT: s_swappc_b64 s[30:31], s[0:1]
-; GFX11-NEXT: s_clause 0xd ; 56-byte Folded Reload
-; GFX11-NEXT: scratch_load_b32 v61, off, s33
-; GFX11-NEXT: scratch_load_b32 v60, off, s33 offset:4
-; GFX11-NEXT: scratch_load_b32 v59, off, s33 offset:8
-; GFX11-NEXT: scratch_load_b32 v58, off, s33 offset:12
-; GFX11-NEXT: scratch_load_b32 v57, off, s33 offset:16
-; GFX11-NEXT: scratch_load_b32 v56, off, s33 offset:20
-; GFX11-NEXT: scratch_load_b32 v47, off, s33 offset:24
-; GFX11-NEXT: scratch_load_b32 v46, off, s33 offset:28
-; GFX11-NEXT: scratch_load_b32 v45, off, s33 offset:32
-; GFX11-NEXT: scratch_load_b32 v44, off, s33 offset:36
-; GFX11-NEXT: scratch_load_b32 v43, off, s33 offset:40
-; GFX11-NEXT: scratch_load_b32 v42, off, s33 offset:44
-; GFX11-NEXT: scratch_load_b32 v41, off, s33 offset:48
-; GFX11-NEXT: scratch_load_b32 v40, off, s33 offset:52
-; GFX11-NEXT: v_readlane_b32 s30, v62, 0
-; GFX11-NEXT: v_readlane_b32 s31, v62, 1
-; GFX11-NEXT: s_mov_b32 s32, s34
-; GFX11-NEXT: s_mov_b32 s34, s39
-; GFX11-NEXT: s_or_saveexec_b32 s0, -1
-; GFX11-NEXT: scratch_load_b32 v62, off, s33 offset:1600 ; 4-byte Folded Reload
-; GFX11-NEXT: s_mov_b32 exec_lo, s0
-; GFX11-NEXT: s_mov_b32 s33, s38
-; GFX11-NEXT: s_waitcnt vmcnt(0)
-; GFX11-NEXT: s_setpc_b64 s[30:31]
-entry:
- %ret.0 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> zeroinitializer)
- %val.0 = insertelement <72 x i32> %ret.0, i32 42, i32 0
- %val.1 = insertelement <72 x i32> %val.0, i32 24, i32 58
- %ret.1 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val.1)
- ret void
-}
-
; Ensure all VGPRs are available
attributes #0 = { nounwind "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="1,1" }
-
-; Limit to 64 VGPRs
-attributes #1 = { nounwind "amdgpu-num-vgpr"="64" }
diff --git a/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-sgpr-spill.ll b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-sgpr-spill.ll
new file mode 100644
index 0000000000000..02124470b6780
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-sgpr-spill.ll
@@ -0,0 +1,56 @@
+; RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -enable-misched=0 -amdgpu-stress-sgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX700 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu8.03-amd-amdhsa -enable-misched=0 -amdgpu-stress-sgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX803 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -amdgpu-stress-sgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX900 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu10.10-amd-amdhsa -enable-misched=0 -amdgpu-stress-sgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX1010 %s
+
+; CHECK: .name: num_spilled_sgprs
+; GFX700: .sgpr_spill_count: 10
+; GFX803: .sgpr_spill_count: 10
+; GFX900: .sgpr_spill_count: 62
+; GFX1010: .sgpr_spill_count: 60
+; CHECK: .symbol: num_spilled_sgprs.kd
+define amdgpu_kernel void @num_spilled_sgprs(
+ ptr addrspace(1) %out0, ptr addrspace(1) %out1, [8 x i32],
+ ptr addrspace(1) %out2, ptr addrspace(1) %out3, [8 x i32],
+ ptr addrspace(1) %out4, ptr addrspace(1) %out5, [8 x i32],
+ ptr addrspace(1) %out6, ptr addrspace(1) %out7, [8 x i32],
+ ptr addrspace(1) %out8, ptr addrspace(1) %out9, [8 x i32],
+ ptr addrspace(1) %outa, ptr addrspace(1) %outb, [8 x i32],
+ ptr addrspace(1) %outc, ptr addrspace(1) %outd, [8 x i32],
+ ptr addrspace(1) %oute, ptr addrspace(1) %outf, [8 x i32],
+ ptr addrspace(1) %outg, ptr addrspace(1) %outh, [8 x i32],
+ ptr addrspace(1) %outi, ptr addrspace(1) %outj, [8 x i32],
+ ptr addrspace(1) %outk, ptr addrspace(1) %outl, [8 x i32],
+ ptr addrspace(1) %outm, ptr addrspace(1) %outn, [8 x i32],
+ i32 %in0, i32 %in1, i32 %in2, i32 %in3, [8 x i32],
+ i32 %in4, i32 %in5, i32 %in6, i32 %in7, [8 x i32],
+ i32 %in8, i32 %in9, i32 %ina, i32 %inb, [8 x i32],
+ i32 %inc, i32 %ind, i32 %ine, i32 %inf, i32 %ing, i32 %inh,
+ i32 %ini, i32 %inj, i32 %ink) "amdgpu-no-flat-scratch-init" {
+entry:
+ store volatile i32 %in0, ptr addrspace(1) %out0
+ store volatile i32 %in1, ptr addrspace(1) %out1
+ store volatile i32 %in2, ptr addrspace(1) %out2
+ store volatile i32 %in3, ptr addrspace(1) %out3
+ store volatile i32 %in4, ptr addrspace(1) %out4
+ store volatile i32 %in5, ptr addrspace(1) %out5
+ store volatile i32 %in6, ptr addrspace(1) %out6
+ store volatile i32 %in7, ptr addrspace(1) %out7
+ store volatile i32 %in8, ptr addrspace(1) %out8
+ store volatile i32 %in9, ptr addrspace(1) %out9
+ store volatile i32 %ina, ptr addrspace(1) %outa
+ store volatile i32 %inb, ptr addrspace(1) %outb
+ store volatile i32 %inc, ptr addrspace(1) %outc
+ store volatile i32 %ind, ptr addrspace(1) %outd
+ store volatile i32 %ine, ptr addrspace(1) %oute
+ store volatile i32 %inf, ptr addrspace(1) %outf
+ store volatile i32 %ing, ptr addrspace(1) %outg
+ store volatile i32 %inh, ptr addrspace(1) %outh
+ store volatile i32 %ini, ptr addrspace(1) %outi
+ store volatile i32 %inj, ptr addrspace(1) %outj
+ store volatile i32 %ink, ptr addrspace(1) %outk
+ ret void
+}
+
+!llvm.module.flags = !{!0}
+!0 = !{i32 1, !"amdhsa_code_object_version", i32 400}
diff --git a/llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-vgpr.ll b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-vgpr-spill.ll
similarity index 78%
rename from llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-vgpr.ll
rename to llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-vgpr-spill.ll
index e202b29597770..8e4da56f1c76d 100644
--- a/llvm/test/CodeGen/AMDGPU/attr-amdgpu-num-vgpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props-vgpr-spill.ll
@@ -1,11 +1,14 @@
-; RUN: llc -mtriple=amdgpu8.03--amdhsa < %s | FileCheck %s
+; RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -enable-misched=0 -amdgpu-stress-vgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu8.03-amd-amdhsa -enable-misched=0 -amdgpu-stress-vgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -amdgpu-stress-vgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu10.10-amd-amdhsa -enable-misched=0 -amdgpu-stress-vgpr=20 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck %s
@var = addrspace(1) global float 0.0
-; CHECK-LABEL: {{^}}max_20_vgprs:
-; CHECK: VGPRBlocks: 4
-; CHECK: NumVGPRsForWavesPerEU: 20
-define amdgpu_kernel void @max_20_vgprs() #1 {
+; CHECK: .name: num_spilled_vgprs
+; CHECK: .symbol: num_spilled_vgprs.kd
+; CHECK: .vgpr_spill_count: {{13|14}}
+define amdgpu_kernel void @num_spilled_vgprs() nounwind {
%val0 = load volatile float, ptr addrspace(1) @var
%val1 = load volatile float, ptr addrspace(1) @var
%val2 = load volatile float, ptr addrspace(1) @var
@@ -72,4 +75,6 @@ define amdgpu_kernel void @max_20_vgprs() #1 {
ret void
}
-attributes #1 = {"amdgpu-num-vgpr"="20"}
+
+!llvm.module.flags = !{!0}
+!0 = !{i32 1, !"amdhsa_code_object_version", i32 400}
diff --git a/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props.ll b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props.ll
index 06ecf2cc49402..161b23069bc5c 100644
--- a/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props.ll
+++ b/llvm/test/CodeGen/AMDGPU/hsa-metadata-kernel-code-props.ll
@@ -1,7 +1,7 @@
-; RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX700,WAVE64 %s
-; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu8.03-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX803,WAVE64 %s
-; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX900,WAVE64 %s
-; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu10.10-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,GFX1010,WAVE32 %s
+; RUN: llc -mtriple=amdgpu7.00-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,WAVE64 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu8.03-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,WAVE64 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,WAVE64 %s
+; RUN: llc --amdgpu-xnack=false -mtriple=amdgpu10.10-amd-amdhsa -enable-misched=0 -filetype=obj -o - < %s | llvm-readelf --notes - | FileCheck --check-prefixes=CHECK,WAVE32 %s
@var = addrspace(1) global float 0.0
@@ -46,132 +46,10 @@ entry:
ret void
}
-; CHECK: .name: num_spilled_sgprs
-; GFX700: .sgpr_spill_count: 10
-; GFX803: .sgpr_spill_count: 10
-; GFX900: .sgpr_spill_count: 62
-; GFX1010: .sgpr_spill_count: 60
-; CHECK: .symbol: num_spilled_sgprs.kd
-define amdgpu_kernel void @num_spilled_sgprs(
- ptr addrspace(1) %out0, ptr addrspace(1) %out1, [8 x i32],
- ptr addrspace(1) %out2, ptr addrspace(1) %out3, [8 x i32],
- ptr addrspace(1) %out4, ptr addrspace(1) %out5, [8 x i32],
- ptr addrspace(1) %out6, ptr addrspace(1) %out7, [8 x i32],
- ptr addrspace(1) %out8, ptr addrspace(1) %out9, [8 x i32],
- ptr addrspace(1) %outa, ptr addrspace(1) %outb, [8 x i32],
- ptr addrspace(1) %outc, ptr addrspace(1) %outd, [8 x i32],
- ptr addrspace(1) %oute, ptr addrspace(1) %outf, [8 x i32],
- ptr addrspace(1) %outg, ptr addrspace(1) %outh, [8 x i32],
- ptr addrspace(1) %outi, ptr addrspace(1) %outj, [8 x i32],
- ptr addrspace(1) %outk, ptr addrspace(1) %outl, [8 x i32],
- ptr addrspace(1) %outm, ptr addrspace(1) %outn, [8 x i32],
- i32 %in0, i32 %in1, i32 %in2, i32 %in3, [8 x i32],
- i32 %in4, i32 %in5, i32 %in6, i32 %in7, [8 x i32],
- i32 %in8, i32 %in9, i32 %ina, i32 %inb, [8 x i32],
- i32 %inc, i32 %ind, i32 %ine, i32 %inf, i32 %ing, i32 %inh,
- i32 %ini, i32 %inj, i32 %ink) #0 {
-entry:
- store volatile i32 %in0, ptr addrspace(1) %out0
- store volatile i32 %in1, ptr addrspace(1) %out1
- store volatile i32 %in2, ptr addrspace(1) %out2
- store volatile i32 %in3, ptr addrspace(1) %out3
- store volatile i32 %in4, ptr addrspace(1) %out4
- store volatile i32 %in5, ptr addrspace(1) %out5
- store volatile i32 %in6, ptr addrspace(1) %out6
- store volatile i32 %in7, ptr addrspace(1) %out7
- store volatile i32 %in8, ptr addrspace(1) %out8
- store volatile i32 %in9, ptr addrspace(1) %out9
- store volatile i32 %ina, ptr addrspace(1) %outa
- store volatile i32 %inb, ptr addrspace(1) %outb
- store volatile i32 %inc, ptr addrspace(1) %outc
- store volatile i32 %ind, ptr addrspace(1) %outd
- store volatile i32 %ine, ptr addrspace(1) %oute
- store volatile i32 %inf, ptr addrspace(1) %outf
- store volatile i32 %ing, ptr addrspace(1) %outg
- store volatile i32 %inh, ptr addrspace(1) %outh
- store volatile i32 %ini, ptr addrspace(1) %outi
- store volatile i32 %inj, ptr addrspace(1) %outj
- store volatile i32 %ink, ptr addrspace(1) %outk
- ret void
-}
-
-; CHECK: .name: num_spilled_vgprs
-; CHECK: .symbol: num_spilled_vgprs.kd
-; CHECK: .vgpr_spill_count: {{13|14}}
-define amdgpu_kernel void @num_spilled_vgprs() #1 {
- %val0 = load volatile float, ptr addrspace(1) @var
- %val1 = load volatile float, ptr addrspace(1) @var
- %val2 = load volatile float, ptr addrspace(1) @var
- %val3 = load volatile float, ptr addrspace(1) @var
- %val4 = load volatile float, ptr addrspace(1) @var
- %val5 = load volatile float, ptr addrspace(1) @var
- %val6 = load volatile float, ptr addrspace(1) @var
- %val7 = load volatile float, ptr addrspace(1) @var
- %val8 = load volatile float, ptr addrspace(1) @var
- %val9 = load volatile float, ptr addrspace(1) @var
- %val10 = load volatile float, ptr addrspace(1) @var
- %val11 = load volatile float, ptr addrspace(1) @var
- %val12 = load volatile float, ptr addrspace(1) @var
- %val13 = load volatile float, ptr addrspace(1) @var
- %val14 = load volatile float, ptr addrspace(1) @var
- %val15 = load volatile float, ptr addrspace(1) @var
- %val16 = load volatile float, ptr addrspace(1) @var
- %val17 = load volatile float, ptr addrspace(1) @var
- %val18 = load volatile float, ptr addrspace(1) @var
- %val19 = load volatile float, ptr addrspace(1) @var
- %val20 = load volatile float, ptr addrspace(1) @var
- %val21 = load volatile float, ptr addrspace(1) @var
- %val22 = load volatile float, ptr addrspace(1) @var
- %val23 = load volatile float, ptr addrspace(1) @var
- %val24 = load volatile float, ptr addrspace(1) @var
- %val25 = load volatile float, ptr addrspace(1) @var
- %val26 = load volatile float, ptr addrspace(1) @var
- %val27 = load volatile float, ptr addrspace(1) @var
- %val28 = load volatile float, ptr addrspace(1) @var
- %val29 = load volatile float, ptr addrspace(1) @var
- %val30 = load volatile float, ptr addrspace(1) @var
-
- store volatile float %val0, ptr addrspace(1) @var
- store volatile float %val1, ptr addrspace(1) @var
- store volatile float %val2, ptr addrspace(1) @var
- store volatile float %val3, ptr addrspace(1) @var
- store volatile float %val4, ptr addrspace(1) @var
- store volatile float %val5, ptr addrspace(1) @var
- store volatile float %val6, ptr addrspace(1) @var
- store volatile float %val7, ptr addrspace(1) @var
- store volatile float %val8, ptr addrspace(1) @var
- store volatile float %val9, ptr addrspace(1) @var
- store volatile float %val10, ptr addrspace(1) @var
- store volatile float %val11, ptr addrspace(1) @var
- store volatile float %val12, ptr addrspace(1) @var
- store volatile float %val13, ptr addrspace(1) @var
- store volatile float %val14, ptr addrspace(1) @var
- store volatile float %val15, ptr addrspace(1) @var
- store volatile float %val16, ptr addrspace(1) @var
- store volatile float %val17, ptr addrspace(1) @var
- store volatile float %val18, ptr addrspace(1) @var
- store volatile float %val19, ptr addrspace(1) @var
- store volatile float %val20, ptr addrspace(1) @var
- store volatile float %val21, ptr addrspace(1) @var
- store volatile float %val22, ptr addrspace(1) @var
- store volatile float %val23, ptr addrspace(1) @var
- store volatile float %val24, ptr addrspace(1) @var
- store volatile float %val25, ptr addrspace(1) @var
- store volatile float %val26, ptr addrspace(1) @var
- store volatile float %val27, ptr addrspace(1) @var
- store volatile float %val28, ptr addrspace(1) @var
- store volatile float %val29, ptr addrspace(1) @var
- store volatile float %val30, ptr addrspace(1) @var
-
- ret void
-}
-
; CHECK: amdhsa.version:
; CHECK-NEXT: - 1
; CHECK-NEXT: - 1
-attributes #0 = { "amdgpu-num-sgpr"="20" "amdgpu-no-flat-scratch-init" }
-attributes #1 = { "amdgpu-num-vgpr"="20" }
attributes #2 = { "amdgpu-flat-work-group-size"="1,256" }
!llvm.module.flags = !{!0}
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-attr.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-attr.mir
index 8b928bc9defa5..6d18c75d164df 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-attr.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-attr.mir
@@ -3,12 +3,6 @@
# RUN: llc -mtriple=amdgpu9.0a -run-pass=machine-scheduler -amdgpu-disable-unclustered-high-rp-reschedule -verify-machineinstrs %s -o - | FileCheck -check-prefix=GFX90A %s
--- |
- define void @small_num_sgprs_as_spill() "amdgpu-num-sgpr"="85" {
- ret void
- }
- define void @small_num_vgprs_as_spill() "amdgpu-num-vgpr"="14" {
- ret void
- }
define void @dont_remat_waves_per_eu() "amdgpu-flat-work-group-size"="1024,1024" "amdgpu-waves-per-eu"="7,7" {
ret void
}
@@ -24,420 +18,6 @@
define void @reduce_spill_agpr_above_addressable_limit() "amdgpu-flat-work-group-size"="1,64" "amdgpu-waves-per-eu"="1,2" {
ret void
}
----
-# User-requested maximum number of SGPRs need to be taken into account by
-# the scheduler's rematerialization stage. Register usage above that number
-# is considered like spill.
-name: small_num_sgprs_as_spill
-tracksRegLiveness: true
-machineFunctionInfo:
- isEntryFunction: true
-body: |
- ; GFX908-LABEL: name: small_num_sgprs_as_spill
- ; GFX908: bb.0:
- ; GFX908-NEXT: successors: %bb.1(0x80000000)
- ; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[S_MOV_B32_:%[0-9]+]]:sgpr_32 = S_MOV_B32 6
- ; GFX908-NEXT: [[S_MOV_B32_1:%[0-9]+]]:sgpr_32 = S_MOV_B32 7
- ; GFX908-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sgpr_32 = S_MOV_B32 8
- ; GFX908-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sgpr_32 = S_MOV_B32 9
- ; GFX908-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sgpr_32 = S_MOV_B32 10
- ; GFX908-NEXT: [[S_MOV_B32_5:%[0-9]+]]:sgpr_32 = S_MOV_B32 11
- ; GFX908-NEXT: [[S_MOV_B32_6:%[0-9]+]]:sgpr_32 = S_MOV_B32 12
- ; GFX908-NEXT: [[S_MOV_B32_7:%[0-9]+]]:sgpr_32 = S_MOV_B32 13
- ; GFX908-NEXT: [[S_MOV_B32_8:%[0-9]+]]:sgpr_32 = S_MOV_B32 14
- ; GFX908-NEXT: [[S_MOV_B32_9:%[0-9]+]]:sgpr_32 = S_MOV_B32 15
- ; GFX908-NEXT: [[S_MOV_B32_10:%[0-9]+]]:sgpr_32 = S_MOV_B32 16
- ; GFX908-NEXT: [[S_MOV_B32_11:%[0-9]+]]:sgpr_32 = S_MOV_B32 17
- ; GFX908-NEXT: [[S_MOV_B32_12:%[0-9]+]]:sgpr_32 = S_MOV_B32 18
- ; GFX908-NEXT: [[S_MOV_B32_13:%[0-9]+]]:sgpr_32 = S_MOV_B32 19
- ; GFX908-NEXT: [[S_MOV_B32_14:%[0-9]+]]:sgpr_32 = S_MOV_B32 20
- ; GFX908-NEXT: [[S_MOV_B32_15:%[0-9]+]]:sgpr_32 = S_MOV_B32 21
- ; GFX908-NEXT: [[S_MOV_B32_16:%[0-9]+]]:sgpr_32 = S_MOV_B32 22
- ; GFX908-NEXT: [[S_MOV_B32_17:%[0-9]+]]:sgpr_32 = S_MOV_B32 23
- ; GFX908-NEXT: [[S_MOV_B32_18:%[0-9]+]]:sgpr_32 = S_MOV_B32 24
- ; GFX908-NEXT: [[S_MOV_B32_19:%[0-9]+]]:sgpr_32 = S_MOV_B32 25
- ; GFX908-NEXT: [[S_MOV_B32_20:%[0-9]+]]:sgpr_32 = S_MOV_B32 26
- ; GFX908-NEXT: [[S_MOV_B32_21:%[0-9]+]]:sgpr_32 = S_MOV_B32 27
- ; GFX908-NEXT: [[S_MOV_B32_22:%[0-9]+]]:sgpr_32 = S_MOV_B32 28
- ; GFX908-NEXT: [[S_MOV_B32_23:%[0-9]+]]:sgpr_32 = S_MOV_B32 29
- ; GFX908-NEXT: [[S_MOV_B32_24:%[0-9]+]]:sgpr_32 = S_MOV_B32 30
- ; GFX908-NEXT: [[S_MOV_B32_25:%[0-9]+]]:sgpr_32 = S_MOV_B32 31
- ; GFX908-NEXT: [[S_MOV_B32_26:%[0-9]+]]:sgpr_32 = S_MOV_B32 32
- ; GFX908-NEXT: [[S_MOV_B32_27:%[0-9]+]]:sgpr_32 = S_MOV_B32 33
- ; GFX908-NEXT: [[S_MOV_B32_28:%[0-9]+]]:sgpr_32 = S_MOV_B32 34
- ; GFX908-NEXT: [[S_MOV_B32_29:%[0-9]+]]:sgpr_32 = S_MOV_B32 35
- ; GFX908-NEXT: [[S_MOV_B32_30:%[0-9]+]]:sgpr_32 = S_MOV_B32 36
- ; GFX908-NEXT: [[S_MOV_B32_31:%[0-9]+]]:sgpr_32 = S_MOV_B32 37
- ; GFX908-NEXT: [[S_MOV_B32_32:%[0-9]+]]:sgpr_32 = S_MOV_B32 38
- ; GFX908-NEXT: [[S_MOV_B32_33:%[0-9]+]]:sgpr_32 = S_MOV_B32 39
- ; GFX908-NEXT: [[S_MOV_B32_34:%[0-9]+]]:sgpr_32 = S_MOV_B32 40
- ; GFX908-NEXT: [[S_MOV_B32_35:%[0-9]+]]:sgpr_32 = S_MOV_B32 41
- ; GFX908-NEXT: [[S_MOV_B32_36:%[0-9]+]]:sgpr_32 = S_MOV_B32 42
- ; GFX908-NEXT: [[S_MOV_B32_37:%[0-9]+]]:sgpr_32 = S_MOV_B32 43
- ; GFX908-NEXT: [[S_MOV_B32_38:%[0-9]+]]:sgpr_32 = S_MOV_B32 44
- ; GFX908-NEXT: [[S_MOV_B32_39:%[0-9]+]]:sgpr_32 = S_MOV_B32 45
- ; GFX908-NEXT: [[S_MOV_B32_40:%[0-9]+]]:sgpr_32 = S_MOV_B32 46
- ; GFX908-NEXT: [[S_MOV_B32_41:%[0-9]+]]:sgpr_32 = S_MOV_B32 47
- ; GFX908-NEXT: [[S_MOV_B32_42:%[0-9]+]]:sgpr_32 = S_MOV_B32 48
- ; GFX908-NEXT: [[S_MOV_B32_43:%[0-9]+]]:sgpr_32 = S_MOV_B32 49
- ; GFX908-NEXT: [[S_MOV_B32_44:%[0-9]+]]:sgpr_32 = S_MOV_B32 50
- ; GFX908-NEXT: [[S_MOV_B32_45:%[0-9]+]]:sgpr_32 = S_MOV_B32 51
- ; GFX908-NEXT: [[S_MOV_B32_46:%[0-9]+]]:sgpr_32 = S_MOV_B32 52
- ; GFX908-NEXT: [[S_MOV_B32_47:%[0-9]+]]:sgpr_32 = S_MOV_B32 53
- ; GFX908-NEXT: [[S_MOV_B32_48:%[0-9]+]]:sgpr_32 = S_MOV_B32 54
- ; GFX908-NEXT: [[S_MOV_B32_49:%[0-9]+]]:sgpr_32 = S_MOV_B32 55
- ; GFX908-NEXT: [[S_MOV_B32_50:%[0-9]+]]:sgpr_32 = S_MOV_B32 56
- ; GFX908-NEXT: [[S_MOV_B32_51:%[0-9]+]]:sgpr_32 = S_MOV_B32 57
- ; GFX908-NEXT: [[S_MOV_B32_52:%[0-9]+]]:sgpr_32 = S_MOV_B32 58
- ; GFX908-NEXT: [[S_MOV_B32_53:%[0-9]+]]:sgpr_32 = S_MOV_B32 59
- ; GFX908-NEXT: [[S_MOV_B32_54:%[0-9]+]]:sgpr_32 = S_MOV_B32 60
- ; GFX908-NEXT: [[S_MOV_B32_55:%[0-9]+]]:sgpr_32 = S_MOV_B32 61
- ; GFX908-NEXT: [[S_MOV_B32_56:%[0-9]+]]:sgpr_32 = S_MOV_B32 62
- ; GFX908-NEXT: [[S_MOV_B32_57:%[0-9]+]]:sgpr_32 = S_MOV_B32 63
- ; GFX908-NEXT: [[S_MOV_B32_58:%[0-9]+]]:sgpr_32 = S_MOV_B32 64
- ; GFX908-NEXT: [[S_MOV_B32_59:%[0-9]+]]:sgpr_32 = S_MOV_B32 65
- ; GFX908-NEXT: [[S_MOV_B32_60:%[0-9]+]]:sgpr_32 = S_MOV_B32 66
- ; GFX908-NEXT: [[S_MOV_B32_61:%[0-9]+]]:sgpr_32 = S_MOV_B32 67
- ; GFX908-NEXT: [[S_MOV_B32_62:%[0-9]+]]:sgpr_32 = S_MOV_B32 68
- ; GFX908-NEXT: [[S_MOV_B32_63:%[0-9]+]]:sgpr_32 = S_MOV_B32 69
- ; GFX908-NEXT: [[S_MOV_B32_64:%[0-9]+]]:sgpr_32 = S_MOV_B32 70
- ; GFX908-NEXT: [[S_MOV_B32_65:%[0-9]+]]:sgpr_32 = S_MOV_B32 71
- ; GFX908-NEXT: [[S_MOV_B32_66:%[0-9]+]]:sgpr_32 = S_MOV_B32 72
- ; GFX908-NEXT: [[S_MOV_B32_67:%[0-9]+]]:sgpr_32 = S_MOV_B32 73
- ; GFX908-NEXT: [[S_MOV_B32_68:%[0-9]+]]:sgpr_32 = S_MOV_B32 74
- ; GFX908-NEXT: [[S_MOV_B32_69:%[0-9]+]]:sgpr_32 = S_MOV_B32 75
- ; GFX908-NEXT: [[S_MOV_B32_70:%[0-9]+]]:sgpr_32 = S_MOV_B32 76
- ; GFX908-NEXT: [[S_MOV_B32_71:%[0-9]+]]:sgpr_32 = S_MOV_B32 77
- ; GFX908-NEXT: [[S_MOV_B32_72:%[0-9]+]]:sgpr_32 = S_MOV_B32 78
- ; GFX908-NEXT: [[S_MOV_B32_73:%[0-9]+]]:sgpr_32 = S_MOV_B32 79
- ; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: bb.1:
- ; GFX908-NEXT: [[S_MOV_B32_74:%[0-9]+]]:sgpr_32 = S_MOV_B32 0
- ; GFX908-NEXT: [[S_MOV_B32_75:%[0-9]+]]:sgpr_32 = S_MOV_B32 1
- ; GFX908-NEXT: [[S_MOV_B32_76:%[0-9]+]]:sgpr_32 = S_MOV_B32 2
- ; GFX908-NEXT: [[S_MOV_B32_77:%[0-9]+]]:sgpr_32 = S_MOV_B32 3
- ; GFX908-NEXT: [[S_MOV_B32_78:%[0-9]+]]:sgpr_32 = S_MOV_B32 4
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_74]], implicit [[S_MOV_B32_75]], implicit [[S_MOV_B32_76]], implicit [[S_MOV_B32_77]], implicit [[S_MOV_B32_78]]
- ; GFX908-NEXT: [[S_MOV_B32_79:%[0-9]+]]:sgpr_32 = S_MOV_B32 5
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_79]], implicit [[S_MOV_B32_]], implicit [[S_MOV_B32_1]], implicit [[S_MOV_B32_2]], implicit [[S_MOV_B32_3]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_4]], implicit [[S_MOV_B32_5]], implicit [[S_MOV_B32_6]], implicit [[S_MOV_B32_7]], implicit [[S_MOV_B32_8]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_9]], implicit [[S_MOV_B32_10]], implicit [[S_MOV_B32_11]], implicit [[S_MOV_B32_12]], implicit [[S_MOV_B32_13]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_14]], implicit [[S_MOV_B32_15]], implicit [[S_MOV_B32_16]], implicit [[S_MOV_B32_17]], implicit [[S_MOV_B32_18]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_19]], implicit [[S_MOV_B32_20]], implicit [[S_MOV_B32_21]], implicit [[S_MOV_B32_22]], implicit [[S_MOV_B32_23]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_24]], implicit [[S_MOV_B32_25]], implicit [[S_MOV_B32_26]], implicit [[S_MOV_B32_27]], implicit [[S_MOV_B32_28]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_29]], implicit [[S_MOV_B32_30]], implicit [[S_MOV_B32_31]], implicit [[S_MOV_B32_32]], implicit [[S_MOV_B32_33]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_34]], implicit [[S_MOV_B32_35]], implicit [[S_MOV_B32_36]], implicit [[S_MOV_B32_37]], implicit [[S_MOV_B32_38]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_39]], implicit [[S_MOV_B32_40]], implicit [[S_MOV_B32_41]], implicit [[S_MOV_B32_42]], implicit [[S_MOV_B32_43]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_44]], implicit [[S_MOV_B32_45]], implicit [[S_MOV_B32_46]], implicit [[S_MOV_B32_47]], implicit [[S_MOV_B32_48]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_49]], implicit [[S_MOV_B32_50]], implicit [[S_MOV_B32_51]], implicit [[S_MOV_B32_52]], implicit [[S_MOV_B32_53]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_54]], implicit [[S_MOV_B32_55]], implicit [[S_MOV_B32_56]], implicit [[S_MOV_B32_57]], implicit [[S_MOV_B32_58]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_59]], implicit [[S_MOV_B32_60]], implicit [[S_MOV_B32_61]], implicit [[S_MOV_B32_62]], implicit [[S_MOV_B32_63]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_64]], implicit [[S_MOV_B32_65]], implicit [[S_MOV_B32_66]], implicit [[S_MOV_B32_67]], implicit [[S_MOV_B32_68]]
- ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_69]], implicit [[S_MOV_B32_70]], implicit [[S_MOV_B32_71]], implicit [[S_MOV_B32_72]], implicit [[S_MOV_B32_73]]
- ; GFX908-NEXT: S_ENDPGM 0
- ;
- ; GFX90A-LABEL: name: small_num_sgprs_as_spill
- ; GFX90A: bb.0:
- ; GFX90A-NEXT: successors: %bb.1(0x80000000)
- ; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: [[S_MOV_B32_:%[0-9]+]]:sgpr_32 = S_MOV_B32 6
- ; GFX90A-NEXT: [[S_MOV_B32_1:%[0-9]+]]:sgpr_32 = S_MOV_B32 7
- ; GFX90A-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sgpr_32 = S_MOV_B32 8
- ; GFX90A-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sgpr_32 = S_MOV_B32 9
- ; GFX90A-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sgpr_32 = S_MOV_B32 10
- ; GFX90A-NEXT: [[S_MOV_B32_5:%[0-9]+]]:sgpr_32 = S_MOV_B32 11
- ; GFX90A-NEXT: [[S_MOV_B32_6:%[0-9]+]]:sgpr_32 = S_MOV_B32 12
- ; GFX90A-NEXT: [[S_MOV_B32_7:%[0-9]+]]:sgpr_32 = S_MOV_B32 13
- ; GFX90A-NEXT: [[S_MOV_B32_8:%[0-9]+]]:sgpr_32 = S_MOV_B32 14
- ; GFX90A-NEXT: [[S_MOV_B32_9:%[0-9]+]]:sgpr_32 = S_MOV_B32 15
- ; GFX90A-NEXT: [[S_MOV_B32_10:%[0-9]+]]:sgpr_32 = S_MOV_B32 16
- ; GFX90A-NEXT: [[S_MOV_B32_11:%[0-9]+]]:sgpr_32 = S_MOV_B32 17
- ; GFX90A-NEXT: [[S_MOV_B32_12:%[0-9]+]]:sgpr_32 = S_MOV_B32 18
- ; GFX90A-NEXT: [[S_MOV_B32_13:%[0-9]+]]:sgpr_32 = S_MOV_B32 19
- ; GFX90A-NEXT: [[S_MOV_B32_14:%[0-9]+]]:sgpr_32 = S_MOV_B32 20
- ; GFX90A-NEXT: [[S_MOV_B32_15:%[0-9]+]]:sgpr_32 = S_MOV_B32 21
- ; GFX90A-NEXT: [[S_MOV_B32_16:%[0-9]+]]:sgpr_32 = S_MOV_B32 22
- ; GFX90A-NEXT: [[S_MOV_B32_17:%[0-9]+]]:sgpr_32 = S_MOV_B32 23
- ; GFX90A-NEXT: [[S_MOV_B32_18:%[0-9]+]]:sgpr_32 = S_MOV_B32 24
- ; GFX90A-NEXT: [[S_MOV_B32_19:%[0-9]+]]:sgpr_32 = S_MOV_B32 25
- ; GFX90A-NEXT: [[S_MOV_B32_20:%[0-9]+]]:sgpr_32 = S_MOV_B32 26
- ; GFX90A-NEXT: [[S_MOV_B32_21:%[0-9]+]]:sgpr_32 = S_MOV_B32 27
- ; GFX90A-NEXT: [[S_MOV_B32_22:%[0-9]+]]:sgpr_32 = S_MOV_B32 28
- ; GFX90A-NEXT: [[S_MOV_B32_23:%[0-9]+]]:sgpr_32 = S_MOV_B32 29
- ; GFX90A-NEXT: [[S_MOV_B32_24:%[0-9]+]]:sgpr_32 = S_MOV_B32 30
- ; GFX90A-NEXT: [[S_MOV_B32_25:%[0-9]+]]:sgpr_32 = S_MOV_B32 31
- ; GFX90A-NEXT: [[S_MOV_B32_26:%[0-9]+]]:sgpr_32 = S_MOV_B32 32
- ; GFX90A-NEXT: [[S_MOV_B32_27:%[0-9]+]]:sgpr_32 = S_MOV_B32 33
- ; GFX90A-NEXT: [[S_MOV_B32_28:%[0-9]+]]:sgpr_32 = S_MOV_B32 34
- ; GFX90A-NEXT: [[S_MOV_B32_29:%[0-9]+]]:sgpr_32 = S_MOV_B32 35
- ; GFX90A-NEXT: [[S_MOV_B32_30:%[0-9]+]]:sgpr_32 = S_MOV_B32 36
- ; GFX90A-NEXT: [[S_MOV_B32_31:%[0-9]+]]:sgpr_32 = S_MOV_B32 37
- ; GFX90A-NEXT: [[S_MOV_B32_32:%[0-9]+]]:sgpr_32 = S_MOV_B32 38
- ; GFX90A-NEXT: [[S_MOV_B32_33:%[0-9]+]]:sgpr_32 = S_MOV_B32 39
- ; GFX90A-NEXT: [[S_MOV_B32_34:%[0-9]+]]:sgpr_32 = S_MOV_B32 40
- ; GFX90A-NEXT: [[S_MOV_B32_35:%[0-9]+]]:sgpr_32 = S_MOV_B32 41
- ; GFX90A-NEXT: [[S_MOV_B32_36:%[0-9]+]]:sgpr_32 = S_MOV_B32 42
- ; GFX90A-NEXT: [[S_MOV_B32_37:%[0-9]+]]:sgpr_32 = S_MOV_B32 43
- ; GFX90A-NEXT: [[S_MOV_B32_38:%[0-9]+]]:sgpr_32 = S_MOV_B32 44
- ; GFX90A-NEXT: [[S_MOV_B32_39:%[0-9]+]]:sgpr_32 = S_MOV_B32 45
- ; GFX90A-NEXT: [[S_MOV_B32_40:%[0-9]+]]:sgpr_32 = S_MOV_B32 46
- ; GFX90A-NEXT: [[S_MOV_B32_41:%[0-9]+]]:sgpr_32 = S_MOV_B32 47
- ; GFX90A-NEXT: [[S_MOV_B32_42:%[0-9]+]]:sgpr_32 = S_MOV_B32 48
- ; GFX90A-NEXT: [[S_MOV_B32_43:%[0-9]+]]:sgpr_32 = S_MOV_B32 49
- ; GFX90A-NEXT: [[S_MOV_B32_44:%[0-9]+]]:sgpr_32 = S_MOV_B32 50
- ; GFX90A-NEXT: [[S_MOV_B32_45:%[0-9]+]]:sgpr_32 = S_MOV_B32 51
- ; GFX90A-NEXT: [[S_MOV_B32_46:%[0-9]+]]:sgpr_32 = S_MOV_B32 52
- ; GFX90A-NEXT: [[S_MOV_B32_47:%[0-9]+]]:sgpr_32 = S_MOV_B32 53
- ; GFX90A-NEXT: [[S_MOV_B32_48:%[0-9]+]]:sgpr_32 = S_MOV_B32 54
- ; GFX90A-NEXT: [[S_MOV_B32_49:%[0-9]+]]:sgpr_32 = S_MOV_B32 55
- ; GFX90A-NEXT: [[S_MOV_B32_50:%[0-9]+]]:sgpr_32 = S_MOV_B32 56
- ; GFX90A-NEXT: [[S_MOV_B32_51:%[0-9]+]]:sgpr_32 = S_MOV_B32 57
- ; GFX90A-NEXT: [[S_MOV_B32_52:%[0-9]+]]:sgpr_32 = S_MOV_B32 58
- ; GFX90A-NEXT: [[S_MOV_B32_53:%[0-9]+]]:sgpr_32 = S_MOV_B32 59
- ; GFX90A-NEXT: [[S_MOV_B32_54:%[0-9]+]]:sgpr_32 = S_MOV_B32 60
- ; GFX90A-NEXT: [[S_MOV_B32_55:%[0-9]+]]:sgpr_32 = S_MOV_B32 61
- ; GFX90A-NEXT: [[S_MOV_B32_56:%[0-9]+]]:sgpr_32 = S_MOV_B32 62
- ; GFX90A-NEXT: [[S_MOV_B32_57:%[0-9]+]]:sgpr_32 = S_MOV_B32 63
- ; GFX90A-NEXT: [[S_MOV_B32_58:%[0-9]+]]:sgpr_32 = S_MOV_B32 64
- ; GFX90A-NEXT: [[S_MOV_B32_59:%[0-9]+]]:sgpr_32 = S_MOV_B32 65
- ; GFX90A-NEXT: [[S_MOV_B32_60:%[0-9]+]]:sgpr_32 = S_MOV_B32 66
- ; GFX90A-NEXT: [[S_MOV_B32_61:%[0-9]+]]:sgpr_32 = S_MOV_B32 67
- ; GFX90A-NEXT: [[S_MOV_B32_62:%[0-9]+]]:sgpr_32 = S_MOV_B32 68
- ; GFX90A-NEXT: [[S_MOV_B32_63:%[0-9]+]]:sgpr_32 = S_MOV_B32 69
- ; GFX90A-NEXT: [[S_MOV_B32_64:%[0-9]+]]:sgpr_32 = S_MOV_B32 70
- ; GFX90A-NEXT: [[S_MOV_B32_65:%[0-9]+]]:sgpr_32 = S_MOV_B32 71
- ; GFX90A-NEXT: [[S_MOV_B32_66:%[0-9]+]]:sgpr_32 = S_MOV_B32 72
- ; GFX90A-NEXT: [[S_MOV_B32_67:%[0-9]+]]:sgpr_32 = S_MOV_B32 73
- ; GFX90A-NEXT: [[S_MOV_B32_68:%[0-9]+]]:sgpr_32 = S_MOV_B32 74
- ; GFX90A-NEXT: [[S_MOV_B32_69:%[0-9]+]]:sgpr_32 = S_MOV_B32 75
- ; GFX90A-NEXT: [[S_MOV_B32_70:%[0-9]+]]:sgpr_32 = S_MOV_B32 76
- ; GFX90A-NEXT: [[S_MOV_B32_71:%[0-9]+]]:sgpr_32 = S_MOV_B32 77
- ; GFX90A-NEXT: [[S_MOV_B32_72:%[0-9]+]]:sgpr_32 = S_MOV_B32 78
- ; GFX90A-NEXT: [[S_MOV_B32_73:%[0-9]+]]:sgpr_32 = S_MOV_B32 79
- ; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: bb.1:
- ; GFX90A-NEXT: [[S_MOV_B32_74:%[0-9]+]]:sgpr_32 = S_MOV_B32 0
- ; GFX90A-NEXT: [[S_MOV_B32_75:%[0-9]+]]:sgpr_32 = S_MOV_B32 1
- ; GFX90A-NEXT: [[S_MOV_B32_76:%[0-9]+]]:sgpr_32 = S_MOV_B32 2
- ; GFX90A-NEXT: [[S_MOV_B32_77:%[0-9]+]]:sgpr_32 = S_MOV_B32 3
- ; GFX90A-NEXT: [[S_MOV_B32_78:%[0-9]+]]:sgpr_32 = S_MOV_B32 4
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_74]], implicit [[S_MOV_B32_75]], implicit [[S_MOV_B32_76]], implicit [[S_MOV_B32_77]], implicit [[S_MOV_B32_78]]
- ; GFX90A-NEXT: [[S_MOV_B32_79:%[0-9]+]]:sgpr_32 = S_MOV_B32 5
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_79]], implicit [[S_MOV_B32_]], implicit [[S_MOV_B32_1]], implicit [[S_MOV_B32_2]], implicit [[S_MOV_B32_3]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_4]], implicit [[S_MOV_B32_5]], implicit [[S_MOV_B32_6]], implicit [[S_MOV_B32_7]], implicit [[S_MOV_B32_8]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_9]], implicit [[S_MOV_B32_10]], implicit [[S_MOV_B32_11]], implicit [[S_MOV_B32_12]], implicit [[S_MOV_B32_13]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_14]], implicit [[S_MOV_B32_15]], implicit [[S_MOV_B32_16]], implicit [[S_MOV_B32_17]], implicit [[S_MOV_B32_18]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_19]], implicit [[S_MOV_B32_20]], implicit [[S_MOV_B32_21]], implicit [[S_MOV_B32_22]], implicit [[S_MOV_B32_23]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_24]], implicit [[S_MOV_B32_25]], implicit [[S_MOV_B32_26]], implicit [[S_MOV_B32_27]], implicit [[S_MOV_B32_28]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_29]], implicit [[S_MOV_B32_30]], implicit [[S_MOV_B32_31]], implicit [[S_MOV_B32_32]], implicit [[S_MOV_B32_33]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_34]], implicit [[S_MOV_B32_35]], implicit [[S_MOV_B32_36]], implicit [[S_MOV_B32_37]], implicit [[S_MOV_B32_38]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_39]], implicit [[S_MOV_B32_40]], implicit [[S_MOV_B32_41]], implicit [[S_MOV_B32_42]], implicit [[S_MOV_B32_43]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_44]], implicit [[S_MOV_B32_45]], implicit [[S_MOV_B32_46]], implicit [[S_MOV_B32_47]], implicit [[S_MOV_B32_48]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_49]], implicit [[S_MOV_B32_50]], implicit [[S_MOV_B32_51]], implicit [[S_MOV_B32_52]], implicit [[S_MOV_B32_53]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_54]], implicit [[S_MOV_B32_55]], implicit [[S_MOV_B32_56]], implicit [[S_MOV_B32_57]], implicit [[S_MOV_B32_58]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_59]], implicit [[S_MOV_B32_60]], implicit [[S_MOV_B32_61]], implicit [[S_MOV_B32_62]], implicit [[S_MOV_B32_63]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_64]], implicit [[S_MOV_B32_65]], implicit [[S_MOV_B32_66]], implicit [[S_MOV_B32_67]], implicit [[S_MOV_B32_68]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_69]], implicit [[S_MOV_B32_70]], implicit [[S_MOV_B32_71]], implicit [[S_MOV_B32_72]], implicit [[S_MOV_B32_73]]
- ; GFX90A-NEXT: S_ENDPGM 0
- bb.0:
- successors: %bb.1
-
- %0:sgpr_32 = S_MOV_B32 0
- %1:sgpr_32 = S_MOV_B32 1
- %2:sgpr_32 = S_MOV_B32 2
- %3:sgpr_32 = S_MOV_B32 3
- %4:sgpr_32 = S_MOV_B32 4
- %5:sgpr_32 = S_MOV_B32 5
- %6:sgpr_32 = S_MOV_B32 6
- %7:sgpr_32 = S_MOV_B32 7
- %8:sgpr_32 = S_MOV_B32 8
- %9:sgpr_32 = S_MOV_B32 9
- %10:sgpr_32 = S_MOV_B32 10
- %11:sgpr_32 = S_MOV_B32 11
- %12:sgpr_32 = S_MOV_B32 12
- %13:sgpr_32 = S_MOV_B32 13
- %14:sgpr_32 = S_MOV_B32 14
- %15:sgpr_32 = S_MOV_B32 15
- %16:sgpr_32 = S_MOV_B32 16
- %17:sgpr_32 = S_MOV_B32 17
- %18:sgpr_32 = S_MOV_B32 18
- %19:sgpr_32 = S_MOV_B32 19
- %20:sgpr_32 = S_MOV_B32 20
- %21:sgpr_32 = S_MOV_B32 21
- %22:sgpr_32 = S_MOV_B32 22
- %23:sgpr_32 = S_MOV_B32 23
- %24:sgpr_32 = S_MOV_B32 24
- %25:sgpr_32 = S_MOV_B32 25
- %26:sgpr_32 = S_MOV_B32 26
- %27:sgpr_32 = S_MOV_B32 27
- %28:sgpr_32 = S_MOV_B32 28
- %29:sgpr_32 = S_MOV_B32 29
- %30:sgpr_32 = S_MOV_B32 30
- %31:sgpr_32 = S_MOV_B32 31
- %32:sgpr_32 = S_MOV_B32 32
- %33:sgpr_32 = S_MOV_B32 33
- %34:sgpr_32 = S_MOV_B32 34
- %35:sgpr_32 = S_MOV_B32 35
- %36:sgpr_32 = S_MOV_B32 36
- %37:sgpr_32 = S_MOV_B32 37
- %38:sgpr_32 = S_MOV_B32 38
- %39:sgpr_32 = S_MOV_B32 39
- %40:sgpr_32 = S_MOV_B32 40
- %41:sgpr_32 = S_MOV_B32 41
- %42:sgpr_32 = S_MOV_B32 42
- %43:sgpr_32 = S_MOV_B32 43
- %44:sgpr_32 = S_MOV_B32 44
- %45:sgpr_32 = S_MOV_B32 45
- %46:sgpr_32 = S_MOV_B32 46
- %47:sgpr_32 = S_MOV_B32 47
- %48:sgpr_32 = S_MOV_B32 48
- %49:sgpr_32 = S_MOV_B32 49
- %50:sgpr_32 = S_MOV_B32 50
- %51:sgpr_32 = S_MOV_B32 51
- %52:sgpr_32 = S_MOV_B32 52
- %53:sgpr_32 = S_MOV_B32 53
- %54:sgpr_32 = S_MOV_B32 54
- %55:sgpr_32 = S_MOV_B32 55
- %56:sgpr_32 = S_MOV_B32 56
- %57:sgpr_32 = S_MOV_B32 57
- %58:sgpr_32 = S_MOV_B32 58
- %59:sgpr_32 = S_MOV_B32 59
- %60:sgpr_32 = S_MOV_B32 60
- %61:sgpr_32 = S_MOV_B32 61
- %62:sgpr_32 = S_MOV_B32 62
- %63:sgpr_32 = S_MOV_B32 63
- %64:sgpr_32 = S_MOV_B32 64
- %65:sgpr_32 = S_MOV_B32 65
- %66:sgpr_32 = S_MOV_B32 66
- %67:sgpr_32 = S_MOV_B32 67
- %68:sgpr_32 = S_MOV_B32 68
- %69:sgpr_32 = S_MOV_B32 69
- %70:sgpr_32 = S_MOV_B32 70
- %71:sgpr_32 = S_MOV_B32 71
- %72:sgpr_32 = S_MOV_B32 72
- %73:sgpr_32 = S_MOV_B32 73
- %74:sgpr_32 = S_MOV_B32 74
- %75:sgpr_32 = S_MOV_B32 75
- %76:sgpr_32 = S_MOV_B32 76
- %77:sgpr_32 = S_MOV_B32 77
- %78:sgpr_32 = S_MOV_B32 78
- %79:sgpr_32 = S_MOV_B32 79
-
- bb.1:
-
- S_NOP 0, implicit %0, implicit %1, implicit %2, implicit %3, implicit %4
- S_NOP 0, implicit %5, implicit %6, implicit %7, implicit %8, implicit %9
- S_NOP 0, implicit %10, implicit %11, implicit %12, implicit %13, implicit %14
- S_NOP 0, implicit %15, implicit %16, implicit %17, implicit %18, implicit %19
- S_NOP 0, implicit %20, implicit %21, implicit %22, implicit %23, implicit %24
- S_NOP 0, implicit %25, implicit %26, implicit %27, implicit %28, implicit %29
- S_NOP 0, implicit %30, implicit %31, implicit %32, implicit %33, implicit %34
- S_NOP 0, implicit %35, implicit %36, implicit %37, implicit %38, implicit %39
- S_NOP 0, implicit %40, implicit %41, implicit %42, implicit %43, implicit %44
- S_NOP 0, implicit %45, implicit %46, implicit %47, implicit %48, implicit %49
- S_NOP 0, implicit %50, implicit %51, implicit %52, implicit %53, implicit %54
- S_NOP 0, implicit %55, implicit %56, implicit %57, implicit %58, implicit %59
- S_NOP 0, implicit %60, implicit %61, implicit %62, implicit %63, implicit %64
- S_NOP 0, implicit %65, implicit %66, implicit %67, implicit %68, implicit %69
- S_NOP 0, implicit %70, implicit %71, implicit %72, implicit %73, implicit %74
- S_NOP 0, implicit %75, implicit %76, implicit %77, implicit %78, implicit %79
-
- S_ENDPGM 0
-...
-# User-requested maximum number of VGPRs need to be taken into account by
-# the scheduler's rematerialization stage. Register usage above that number
-# is considered like spill. On unified RF (gfx90a), the requested number is
-# understood "per-bank", effectively doubling its value, so no rematerialization
-# is necessary.
----
-name: small_num_vgprs_as_spill
-tracksRegLiveness: true
-machineFunctionInfo:
- isEntryFunction: true
-body: |
- ; GFX908-LABEL: name: small_num_vgprs_as_spill
- ; GFX908: bb.0:
- ; GFX908-NEXT: successors: %bb.1(0x80000000)
- ; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_3:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_6:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_7:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_8:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_9:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_10:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_11:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_12:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_13:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
- ; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: bb.1:
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]], implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]], implicit [[V_CVT_I32_F64_e32_4]]
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_5]], implicit [[V_CVT_I32_F64_e32_6]], implicit [[V_CVT_I32_F64_e32_7]], implicit [[V_CVT_I32_F64_e32_8]], implicit [[V_CVT_I32_F64_e32_9]]
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_14:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
- ; GFX908-NEXT: [[V_CVT_I32_F64_e32_15:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_10]], implicit [[V_CVT_I32_F64_e32_11]], implicit [[V_CVT_I32_F64_e32_12]], implicit [[V_CVT_I32_F64_e32_14]], implicit [[V_CVT_I32_F64_e32_15]]
- ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_13]]
- ; GFX908-NEXT: S_ENDPGM 0
- ;
- ; GFX90A-LABEL: name: small_num_vgprs_as_spill
- ; GFX90A: bb.0:
- ; GFX90A-NEXT: successors: %bb.1(0x80000000)
- ; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_3:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_6:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_7:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_8:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_9:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_10:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_11:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_12:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_13:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_14:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
- ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_15:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
- ; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: bb.1:
- ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]], implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]], implicit [[V_CVT_I32_F64_e32_4]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_5]], implicit [[V_CVT_I32_F64_e32_6]], implicit [[V_CVT_I32_F64_e32_7]], implicit [[V_CVT_I32_F64_e32_8]], implicit [[V_CVT_I32_F64_e32_9]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_10]], implicit [[V_CVT_I32_F64_e32_11]], implicit [[V_CVT_I32_F64_e32_12]], implicit [[V_CVT_I32_F64_e32_13]], implicit [[V_CVT_I32_F64_e32_14]]
- ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_15]]
- ; GFX90A-NEXT: S_ENDPGM 0
- bb.0:
- successors: %bb.1
-
- %0:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
- %1:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
- %2:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
- %3:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
- %4:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
- %5:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
- %6:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
- %7:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
- %8:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
- %9:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
- %10:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
- %11:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
- %12:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
- %13:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
- %14:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
- %15:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
-
- bb.1:
-
- S_NOP 0, implicit %0, implicit %1, implicit %2, implicit %3, implicit %4
- S_NOP 0, implicit %5, implicit %6, implicit %7, implicit %8, implicit %9
- S_NOP 0, implicit %10, implicit %11, implicit %12, implicit %13, implicit %14
- S_NOP 0, implicit %15
- S_ENDPGM 0
...
# Min/Max occupancy is 8, but user requests 7, the scheduler's rematerialization
# stage should not try to rematerialize instructions.
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-sgpr-stress.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-sgpr-stress.mir
new file mode 100644
index 0000000000000..92d0d1a936245
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-sgpr-stress.mir
@@ -0,0 +1,330 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -run-pass=machine-scheduler -amdgpu-disable-unclustered-high-rp-reschedule -verify-machineinstrs -amdgpu-stress-sgpr=83 %s -o - | FileCheck -check-prefix=GFX908 %s
+# RUN: llc -mtriple=amdgpu9.0a -run-pass=machine-scheduler -amdgpu-disable-unclustered-high-rp-reschedule -verify-machineinstrs -amdgpu-stress-sgpr=83 %s -o - | FileCheck -check-prefix=GFX90A %s
+
+--- |
+ define void @small_num_sgprs_as_spill() {
+ ret void
+ }
+...
+---
+# User-requested maximum number of SGPRs need to be taken into account by
+# the scheduler's rematerialization stage. Register usage above that number
+# is considered like spill.
+name: small_num_sgprs_as_spill
+tracksRegLiveness: true
+machineFunctionInfo:
+ isEntryFunction: true
+body: |
+ ; GFX908-LABEL: name: small_num_sgprs_as_spill
+ ; GFX908: bb.0:
+ ; GFX908-NEXT: successors: %bb.1(0x80000000)
+ ; GFX908-NEXT: {{ $}}
+ ; GFX908-NEXT: [[S_MOV_B32_:%[0-9]+]]:sgpr_32 = S_MOV_B32 6
+ ; GFX908-NEXT: [[S_MOV_B32_1:%[0-9]+]]:sgpr_32 = S_MOV_B32 7
+ ; GFX908-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sgpr_32 = S_MOV_B32 8
+ ; GFX908-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sgpr_32 = S_MOV_B32 9
+ ; GFX908-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sgpr_32 = S_MOV_B32 10
+ ; GFX908-NEXT: [[S_MOV_B32_5:%[0-9]+]]:sgpr_32 = S_MOV_B32 11
+ ; GFX908-NEXT: [[S_MOV_B32_6:%[0-9]+]]:sgpr_32 = S_MOV_B32 12
+ ; GFX908-NEXT: [[S_MOV_B32_7:%[0-9]+]]:sgpr_32 = S_MOV_B32 13
+ ; GFX908-NEXT: [[S_MOV_B32_8:%[0-9]+]]:sgpr_32 = S_MOV_B32 14
+ ; GFX908-NEXT: [[S_MOV_B32_9:%[0-9]+]]:sgpr_32 = S_MOV_B32 15
+ ; GFX908-NEXT: [[S_MOV_B32_10:%[0-9]+]]:sgpr_32 = S_MOV_B32 16
+ ; GFX908-NEXT: [[S_MOV_B32_11:%[0-9]+]]:sgpr_32 = S_MOV_B32 17
+ ; GFX908-NEXT: [[S_MOV_B32_12:%[0-9]+]]:sgpr_32 = S_MOV_B32 18
+ ; GFX908-NEXT: [[S_MOV_B32_13:%[0-9]+]]:sgpr_32 = S_MOV_B32 19
+ ; GFX908-NEXT: [[S_MOV_B32_14:%[0-9]+]]:sgpr_32 = S_MOV_B32 20
+ ; GFX908-NEXT: [[S_MOV_B32_15:%[0-9]+]]:sgpr_32 = S_MOV_B32 21
+ ; GFX908-NEXT: [[S_MOV_B32_16:%[0-9]+]]:sgpr_32 = S_MOV_B32 22
+ ; GFX908-NEXT: [[S_MOV_B32_17:%[0-9]+]]:sgpr_32 = S_MOV_B32 23
+ ; GFX908-NEXT: [[S_MOV_B32_18:%[0-9]+]]:sgpr_32 = S_MOV_B32 24
+ ; GFX908-NEXT: [[S_MOV_B32_19:%[0-9]+]]:sgpr_32 = S_MOV_B32 25
+ ; GFX908-NEXT: [[S_MOV_B32_20:%[0-9]+]]:sgpr_32 = S_MOV_B32 26
+ ; GFX908-NEXT: [[S_MOV_B32_21:%[0-9]+]]:sgpr_32 = S_MOV_B32 27
+ ; GFX908-NEXT: [[S_MOV_B32_22:%[0-9]+]]:sgpr_32 = S_MOV_B32 28
+ ; GFX908-NEXT: [[S_MOV_B32_23:%[0-9]+]]:sgpr_32 = S_MOV_B32 29
+ ; GFX908-NEXT: [[S_MOV_B32_24:%[0-9]+]]:sgpr_32 = S_MOV_B32 30
+ ; GFX908-NEXT: [[S_MOV_B32_25:%[0-9]+]]:sgpr_32 = S_MOV_B32 31
+ ; GFX908-NEXT: [[S_MOV_B32_26:%[0-9]+]]:sgpr_32 = S_MOV_B32 32
+ ; GFX908-NEXT: [[S_MOV_B32_27:%[0-9]+]]:sgpr_32 = S_MOV_B32 33
+ ; GFX908-NEXT: [[S_MOV_B32_28:%[0-9]+]]:sgpr_32 = S_MOV_B32 34
+ ; GFX908-NEXT: [[S_MOV_B32_29:%[0-9]+]]:sgpr_32 = S_MOV_B32 35
+ ; GFX908-NEXT: [[S_MOV_B32_30:%[0-9]+]]:sgpr_32 = S_MOV_B32 36
+ ; GFX908-NEXT: [[S_MOV_B32_31:%[0-9]+]]:sgpr_32 = S_MOV_B32 37
+ ; GFX908-NEXT: [[S_MOV_B32_32:%[0-9]+]]:sgpr_32 = S_MOV_B32 38
+ ; GFX908-NEXT: [[S_MOV_B32_33:%[0-9]+]]:sgpr_32 = S_MOV_B32 39
+ ; GFX908-NEXT: [[S_MOV_B32_34:%[0-9]+]]:sgpr_32 = S_MOV_B32 40
+ ; GFX908-NEXT: [[S_MOV_B32_35:%[0-9]+]]:sgpr_32 = S_MOV_B32 41
+ ; GFX908-NEXT: [[S_MOV_B32_36:%[0-9]+]]:sgpr_32 = S_MOV_B32 42
+ ; GFX908-NEXT: [[S_MOV_B32_37:%[0-9]+]]:sgpr_32 = S_MOV_B32 43
+ ; GFX908-NEXT: [[S_MOV_B32_38:%[0-9]+]]:sgpr_32 = S_MOV_B32 44
+ ; GFX908-NEXT: [[S_MOV_B32_39:%[0-9]+]]:sgpr_32 = S_MOV_B32 45
+ ; GFX908-NEXT: [[S_MOV_B32_40:%[0-9]+]]:sgpr_32 = S_MOV_B32 46
+ ; GFX908-NEXT: [[S_MOV_B32_41:%[0-9]+]]:sgpr_32 = S_MOV_B32 47
+ ; GFX908-NEXT: [[S_MOV_B32_42:%[0-9]+]]:sgpr_32 = S_MOV_B32 48
+ ; GFX908-NEXT: [[S_MOV_B32_43:%[0-9]+]]:sgpr_32 = S_MOV_B32 49
+ ; GFX908-NEXT: [[S_MOV_B32_44:%[0-9]+]]:sgpr_32 = S_MOV_B32 50
+ ; GFX908-NEXT: [[S_MOV_B32_45:%[0-9]+]]:sgpr_32 = S_MOV_B32 51
+ ; GFX908-NEXT: [[S_MOV_B32_46:%[0-9]+]]:sgpr_32 = S_MOV_B32 52
+ ; GFX908-NEXT: [[S_MOV_B32_47:%[0-9]+]]:sgpr_32 = S_MOV_B32 53
+ ; GFX908-NEXT: [[S_MOV_B32_48:%[0-9]+]]:sgpr_32 = S_MOV_B32 54
+ ; GFX908-NEXT: [[S_MOV_B32_49:%[0-9]+]]:sgpr_32 = S_MOV_B32 55
+ ; GFX908-NEXT: [[S_MOV_B32_50:%[0-9]+]]:sgpr_32 = S_MOV_B32 56
+ ; GFX908-NEXT: [[S_MOV_B32_51:%[0-9]+]]:sgpr_32 = S_MOV_B32 57
+ ; GFX908-NEXT: [[S_MOV_B32_52:%[0-9]+]]:sgpr_32 = S_MOV_B32 58
+ ; GFX908-NEXT: [[S_MOV_B32_53:%[0-9]+]]:sgpr_32 = S_MOV_B32 59
+ ; GFX908-NEXT: [[S_MOV_B32_54:%[0-9]+]]:sgpr_32 = S_MOV_B32 60
+ ; GFX908-NEXT: [[S_MOV_B32_55:%[0-9]+]]:sgpr_32 = S_MOV_B32 61
+ ; GFX908-NEXT: [[S_MOV_B32_56:%[0-9]+]]:sgpr_32 = S_MOV_B32 62
+ ; GFX908-NEXT: [[S_MOV_B32_57:%[0-9]+]]:sgpr_32 = S_MOV_B32 63
+ ; GFX908-NEXT: [[S_MOV_B32_58:%[0-9]+]]:sgpr_32 = S_MOV_B32 64
+ ; GFX908-NEXT: [[S_MOV_B32_59:%[0-9]+]]:sgpr_32 = S_MOV_B32 65
+ ; GFX908-NEXT: [[S_MOV_B32_60:%[0-9]+]]:sgpr_32 = S_MOV_B32 66
+ ; GFX908-NEXT: [[S_MOV_B32_61:%[0-9]+]]:sgpr_32 = S_MOV_B32 67
+ ; GFX908-NEXT: [[S_MOV_B32_62:%[0-9]+]]:sgpr_32 = S_MOV_B32 68
+ ; GFX908-NEXT: [[S_MOV_B32_63:%[0-9]+]]:sgpr_32 = S_MOV_B32 69
+ ; GFX908-NEXT: [[S_MOV_B32_64:%[0-9]+]]:sgpr_32 = S_MOV_B32 70
+ ; GFX908-NEXT: [[S_MOV_B32_65:%[0-9]+]]:sgpr_32 = S_MOV_B32 71
+ ; GFX908-NEXT: [[S_MOV_B32_66:%[0-9]+]]:sgpr_32 = S_MOV_B32 72
+ ; GFX908-NEXT: [[S_MOV_B32_67:%[0-9]+]]:sgpr_32 = S_MOV_B32 73
+ ; GFX908-NEXT: [[S_MOV_B32_68:%[0-9]+]]:sgpr_32 = S_MOV_B32 74
+ ; GFX908-NEXT: [[S_MOV_B32_69:%[0-9]+]]:sgpr_32 = S_MOV_B32 75
+ ; GFX908-NEXT: [[S_MOV_B32_70:%[0-9]+]]:sgpr_32 = S_MOV_B32 76
+ ; GFX908-NEXT: [[S_MOV_B32_71:%[0-9]+]]:sgpr_32 = S_MOV_B32 77
+ ; GFX908-NEXT: [[S_MOV_B32_72:%[0-9]+]]:sgpr_32 = S_MOV_B32 78
+ ; GFX908-NEXT: [[S_MOV_B32_73:%[0-9]+]]:sgpr_32 = S_MOV_B32 79
+ ; GFX908-NEXT: {{ $}}
+ ; GFX908-NEXT: bb.1:
+ ; GFX908-NEXT: [[S_MOV_B32_74:%[0-9]+]]:sgpr_32 = S_MOV_B32 0
+ ; GFX908-NEXT: [[S_MOV_B32_75:%[0-9]+]]:sgpr_32 = S_MOV_B32 1
+ ; GFX908-NEXT: [[S_MOV_B32_76:%[0-9]+]]:sgpr_32 = S_MOV_B32 2
+ ; GFX908-NEXT: [[S_MOV_B32_77:%[0-9]+]]:sgpr_32 = S_MOV_B32 3
+ ; GFX908-NEXT: [[S_MOV_B32_78:%[0-9]+]]:sgpr_32 = S_MOV_B32 4
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_74]], implicit [[S_MOV_B32_75]], implicit [[S_MOV_B32_76]], implicit [[S_MOV_B32_77]], implicit [[S_MOV_B32_78]]
+ ; GFX908-NEXT: [[S_MOV_B32_79:%[0-9]+]]:sgpr_32 = S_MOV_B32 5
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_79]], implicit [[S_MOV_B32_]], implicit [[S_MOV_B32_1]], implicit [[S_MOV_B32_2]], implicit [[S_MOV_B32_3]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_4]], implicit [[S_MOV_B32_5]], implicit [[S_MOV_B32_6]], implicit [[S_MOV_B32_7]], implicit [[S_MOV_B32_8]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_9]], implicit [[S_MOV_B32_10]], implicit [[S_MOV_B32_11]], implicit [[S_MOV_B32_12]], implicit [[S_MOV_B32_13]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_14]], implicit [[S_MOV_B32_15]], implicit [[S_MOV_B32_16]], implicit [[S_MOV_B32_17]], implicit [[S_MOV_B32_18]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_19]], implicit [[S_MOV_B32_20]], implicit [[S_MOV_B32_21]], implicit [[S_MOV_B32_22]], implicit [[S_MOV_B32_23]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_24]], implicit [[S_MOV_B32_25]], implicit [[S_MOV_B32_26]], implicit [[S_MOV_B32_27]], implicit [[S_MOV_B32_28]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_29]], implicit [[S_MOV_B32_30]], implicit [[S_MOV_B32_31]], implicit [[S_MOV_B32_32]], implicit [[S_MOV_B32_33]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_34]], implicit [[S_MOV_B32_35]], implicit [[S_MOV_B32_36]], implicit [[S_MOV_B32_37]], implicit [[S_MOV_B32_38]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_39]], implicit [[S_MOV_B32_40]], implicit [[S_MOV_B32_41]], implicit [[S_MOV_B32_42]], implicit [[S_MOV_B32_43]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_44]], implicit [[S_MOV_B32_45]], implicit [[S_MOV_B32_46]], implicit [[S_MOV_B32_47]], implicit [[S_MOV_B32_48]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_49]], implicit [[S_MOV_B32_50]], implicit [[S_MOV_B32_51]], implicit [[S_MOV_B32_52]], implicit [[S_MOV_B32_53]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_54]], implicit [[S_MOV_B32_55]], implicit [[S_MOV_B32_56]], implicit [[S_MOV_B32_57]], implicit [[S_MOV_B32_58]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_59]], implicit [[S_MOV_B32_60]], implicit [[S_MOV_B32_61]], implicit [[S_MOV_B32_62]], implicit [[S_MOV_B32_63]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_64]], implicit [[S_MOV_B32_65]], implicit [[S_MOV_B32_66]], implicit [[S_MOV_B32_67]], implicit [[S_MOV_B32_68]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[S_MOV_B32_69]], implicit [[S_MOV_B32_70]], implicit [[S_MOV_B32_71]], implicit [[S_MOV_B32_72]], implicit [[S_MOV_B32_73]]
+ ; GFX908-NEXT: S_ENDPGM 0
+ ;
+ ; GFX90A-LABEL: name: small_num_sgprs_as_spill
+ ; GFX90A: bb.0:
+ ; GFX90A-NEXT: successors: %bb.1(0x80000000)
+ ; GFX90A-NEXT: {{ $}}
+ ; GFX90A-NEXT: [[S_MOV_B32_:%[0-9]+]]:sgpr_32 = S_MOV_B32 6
+ ; GFX90A-NEXT: [[S_MOV_B32_1:%[0-9]+]]:sgpr_32 = S_MOV_B32 7
+ ; GFX90A-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sgpr_32 = S_MOV_B32 8
+ ; GFX90A-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sgpr_32 = S_MOV_B32 9
+ ; GFX90A-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sgpr_32 = S_MOV_B32 10
+ ; GFX90A-NEXT: [[S_MOV_B32_5:%[0-9]+]]:sgpr_32 = S_MOV_B32 11
+ ; GFX90A-NEXT: [[S_MOV_B32_6:%[0-9]+]]:sgpr_32 = S_MOV_B32 12
+ ; GFX90A-NEXT: [[S_MOV_B32_7:%[0-9]+]]:sgpr_32 = S_MOV_B32 13
+ ; GFX90A-NEXT: [[S_MOV_B32_8:%[0-9]+]]:sgpr_32 = S_MOV_B32 14
+ ; GFX90A-NEXT: [[S_MOV_B32_9:%[0-9]+]]:sgpr_32 = S_MOV_B32 15
+ ; GFX90A-NEXT: [[S_MOV_B32_10:%[0-9]+]]:sgpr_32 = S_MOV_B32 16
+ ; GFX90A-NEXT: [[S_MOV_B32_11:%[0-9]+]]:sgpr_32 = S_MOV_B32 17
+ ; GFX90A-NEXT: [[S_MOV_B32_12:%[0-9]+]]:sgpr_32 = S_MOV_B32 18
+ ; GFX90A-NEXT: [[S_MOV_B32_13:%[0-9]+]]:sgpr_32 = S_MOV_B32 19
+ ; GFX90A-NEXT: [[S_MOV_B32_14:%[0-9]+]]:sgpr_32 = S_MOV_B32 20
+ ; GFX90A-NEXT: [[S_MOV_B32_15:%[0-9]+]]:sgpr_32 = S_MOV_B32 21
+ ; GFX90A-NEXT: [[S_MOV_B32_16:%[0-9]+]]:sgpr_32 = S_MOV_B32 22
+ ; GFX90A-NEXT: [[S_MOV_B32_17:%[0-9]+]]:sgpr_32 = S_MOV_B32 23
+ ; GFX90A-NEXT: [[S_MOV_B32_18:%[0-9]+]]:sgpr_32 = S_MOV_B32 24
+ ; GFX90A-NEXT: [[S_MOV_B32_19:%[0-9]+]]:sgpr_32 = S_MOV_B32 25
+ ; GFX90A-NEXT: [[S_MOV_B32_20:%[0-9]+]]:sgpr_32 = S_MOV_B32 26
+ ; GFX90A-NEXT: [[S_MOV_B32_21:%[0-9]+]]:sgpr_32 = S_MOV_B32 27
+ ; GFX90A-NEXT: [[S_MOV_B32_22:%[0-9]+]]:sgpr_32 = S_MOV_B32 28
+ ; GFX90A-NEXT: [[S_MOV_B32_23:%[0-9]+]]:sgpr_32 = S_MOV_B32 29
+ ; GFX90A-NEXT: [[S_MOV_B32_24:%[0-9]+]]:sgpr_32 = S_MOV_B32 30
+ ; GFX90A-NEXT: [[S_MOV_B32_25:%[0-9]+]]:sgpr_32 = S_MOV_B32 31
+ ; GFX90A-NEXT: [[S_MOV_B32_26:%[0-9]+]]:sgpr_32 = S_MOV_B32 32
+ ; GFX90A-NEXT: [[S_MOV_B32_27:%[0-9]+]]:sgpr_32 = S_MOV_B32 33
+ ; GFX90A-NEXT: [[S_MOV_B32_28:%[0-9]+]]:sgpr_32 = S_MOV_B32 34
+ ; GFX90A-NEXT: [[S_MOV_B32_29:%[0-9]+]]:sgpr_32 = S_MOV_B32 35
+ ; GFX90A-NEXT: [[S_MOV_B32_30:%[0-9]+]]:sgpr_32 = S_MOV_B32 36
+ ; GFX90A-NEXT: [[S_MOV_B32_31:%[0-9]+]]:sgpr_32 = S_MOV_B32 37
+ ; GFX90A-NEXT: [[S_MOV_B32_32:%[0-9]+]]:sgpr_32 = S_MOV_B32 38
+ ; GFX90A-NEXT: [[S_MOV_B32_33:%[0-9]+]]:sgpr_32 = S_MOV_B32 39
+ ; GFX90A-NEXT: [[S_MOV_B32_34:%[0-9]+]]:sgpr_32 = S_MOV_B32 40
+ ; GFX90A-NEXT: [[S_MOV_B32_35:%[0-9]+]]:sgpr_32 = S_MOV_B32 41
+ ; GFX90A-NEXT: [[S_MOV_B32_36:%[0-9]+]]:sgpr_32 = S_MOV_B32 42
+ ; GFX90A-NEXT: [[S_MOV_B32_37:%[0-9]+]]:sgpr_32 = S_MOV_B32 43
+ ; GFX90A-NEXT: [[S_MOV_B32_38:%[0-9]+]]:sgpr_32 = S_MOV_B32 44
+ ; GFX90A-NEXT: [[S_MOV_B32_39:%[0-9]+]]:sgpr_32 = S_MOV_B32 45
+ ; GFX90A-NEXT: [[S_MOV_B32_40:%[0-9]+]]:sgpr_32 = S_MOV_B32 46
+ ; GFX90A-NEXT: [[S_MOV_B32_41:%[0-9]+]]:sgpr_32 = S_MOV_B32 47
+ ; GFX90A-NEXT: [[S_MOV_B32_42:%[0-9]+]]:sgpr_32 = S_MOV_B32 48
+ ; GFX90A-NEXT: [[S_MOV_B32_43:%[0-9]+]]:sgpr_32 = S_MOV_B32 49
+ ; GFX90A-NEXT: [[S_MOV_B32_44:%[0-9]+]]:sgpr_32 = S_MOV_B32 50
+ ; GFX90A-NEXT: [[S_MOV_B32_45:%[0-9]+]]:sgpr_32 = S_MOV_B32 51
+ ; GFX90A-NEXT: [[S_MOV_B32_46:%[0-9]+]]:sgpr_32 = S_MOV_B32 52
+ ; GFX90A-NEXT: [[S_MOV_B32_47:%[0-9]+]]:sgpr_32 = S_MOV_B32 53
+ ; GFX90A-NEXT: [[S_MOV_B32_48:%[0-9]+]]:sgpr_32 = S_MOV_B32 54
+ ; GFX90A-NEXT: [[S_MOV_B32_49:%[0-9]+]]:sgpr_32 = S_MOV_B32 55
+ ; GFX90A-NEXT: [[S_MOV_B32_50:%[0-9]+]]:sgpr_32 = S_MOV_B32 56
+ ; GFX90A-NEXT: [[S_MOV_B32_51:%[0-9]+]]:sgpr_32 = S_MOV_B32 57
+ ; GFX90A-NEXT: [[S_MOV_B32_52:%[0-9]+]]:sgpr_32 = S_MOV_B32 58
+ ; GFX90A-NEXT: [[S_MOV_B32_53:%[0-9]+]]:sgpr_32 = S_MOV_B32 59
+ ; GFX90A-NEXT: [[S_MOV_B32_54:%[0-9]+]]:sgpr_32 = S_MOV_B32 60
+ ; GFX90A-NEXT: [[S_MOV_B32_55:%[0-9]+]]:sgpr_32 = S_MOV_B32 61
+ ; GFX90A-NEXT: [[S_MOV_B32_56:%[0-9]+]]:sgpr_32 = S_MOV_B32 62
+ ; GFX90A-NEXT: [[S_MOV_B32_57:%[0-9]+]]:sgpr_32 = S_MOV_B32 63
+ ; GFX90A-NEXT: [[S_MOV_B32_58:%[0-9]+]]:sgpr_32 = S_MOV_B32 64
+ ; GFX90A-NEXT: [[S_MOV_B32_59:%[0-9]+]]:sgpr_32 = S_MOV_B32 65
+ ; GFX90A-NEXT: [[S_MOV_B32_60:%[0-9]+]]:sgpr_32 = S_MOV_B32 66
+ ; GFX90A-NEXT: [[S_MOV_B32_61:%[0-9]+]]:sgpr_32 = S_MOV_B32 67
+ ; GFX90A-NEXT: [[S_MOV_B32_62:%[0-9]+]]:sgpr_32 = S_MOV_B32 68
+ ; GFX90A-NEXT: [[S_MOV_B32_63:%[0-9]+]]:sgpr_32 = S_MOV_B32 69
+ ; GFX90A-NEXT: [[S_MOV_B32_64:%[0-9]+]]:sgpr_32 = S_MOV_B32 70
+ ; GFX90A-NEXT: [[S_MOV_B32_65:%[0-9]+]]:sgpr_32 = S_MOV_B32 71
+ ; GFX90A-NEXT: [[S_MOV_B32_66:%[0-9]+]]:sgpr_32 = S_MOV_B32 72
+ ; GFX90A-NEXT: [[S_MOV_B32_67:%[0-9]+]]:sgpr_32 = S_MOV_B32 73
+ ; GFX90A-NEXT: [[S_MOV_B32_68:%[0-9]+]]:sgpr_32 = S_MOV_B32 74
+ ; GFX90A-NEXT: [[S_MOV_B32_69:%[0-9]+]]:sgpr_32 = S_MOV_B32 75
+ ; GFX90A-NEXT: [[S_MOV_B32_70:%[0-9]+]]:sgpr_32 = S_MOV_B32 76
+ ; GFX90A-NEXT: [[S_MOV_B32_71:%[0-9]+]]:sgpr_32 = S_MOV_B32 77
+ ; GFX90A-NEXT: [[S_MOV_B32_72:%[0-9]+]]:sgpr_32 = S_MOV_B32 78
+ ; GFX90A-NEXT: [[S_MOV_B32_73:%[0-9]+]]:sgpr_32 = S_MOV_B32 79
+ ; GFX90A-NEXT: {{ $}}
+ ; GFX90A-NEXT: bb.1:
+ ; GFX90A-NEXT: [[S_MOV_B32_74:%[0-9]+]]:sgpr_32 = S_MOV_B32 0
+ ; GFX90A-NEXT: [[S_MOV_B32_75:%[0-9]+]]:sgpr_32 = S_MOV_B32 1
+ ; GFX90A-NEXT: [[S_MOV_B32_76:%[0-9]+]]:sgpr_32 = S_MOV_B32 2
+ ; GFX90A-NEXT: [[S_MOV_B32_77:%[0-9]+]]:sgpr_32 = S_MOV_B32 3
+ ; GFX90A-NEXT: [[S_MOV_B32_78:%[0-9]+]]:sgpr_32 = S_MOV_B32 4
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_74]], implicit [[S_MOV_B32_75]], implicit [[S_MOV_B32_76]], implicit [[S_MOV_B32_77]], implicit [[S_MOV_B32_78]]
+ ; GFX90A-NEXT: [[S_MOV_B32_79:%[0-9]+]]:sgpr_32 = S_MOV_B32 5
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_79]], implicit [[S_MOV_B32_]], implicit [[S_MOV_B32_1]], implicit [[S_MOV_B32_2]], implicit [[S_MOV_B32_3]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_4]], implicit [[S_MOV_B32_5]], implicit [[S_MOV_B32_6]], implicit [[S_MOV_B32_7]], implicit [[S_MOV_B32_8]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_9]], implicit [[S_MOV_B32_10]], implicit [[S_MOV_B32_11]], implicit [[S_MOV_B32_12]], implicit [[S_MOV_B32_13]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_14]], implicit [[S_MOV_B32_15]], implicit [[S_MOV_B32_16]], implicit [[S_MOV_B32_17]], implicit [[S_MOV_B32_18]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_19]], implicit [[S_MOV_B32_20]], implicit [[S_MOV_B32_21]], implicit [[S_MOV_B32_22]], implicit [[S_MOV_B32_23]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_24]], implicit [[S_MOV_B32_25]], implicit [[S_MOV_B32_26]], implicit [[S_MOV_B32_27]], implicit [[S_MOV_B32_28]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_29]], implicit [[S_MOV_B32_30]], implicit [[S_MOV_B32_31]], implicit [[S_MOV_B32_32]], implicit [[S_MOV_B32_33]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_34]], implicit [[S_MOV_B32_35]], implicit [[S_MOV_B32_36]], implicit [[S_MOV_B32_37]], implicit [[S_MOV_B32_38]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_39]], implicit [[S_MOV_B32_40]], implicit [[S_MOV_B32_41]], implicit [[S_MOV_B32_42]], implicit [[S_MOV_B32_43]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_44]], implicit [[S_MOV_B32_45]], implicit [[S_MOV_B32_46]], implicit [[S_MOV_B32_47]], implicit [[S_MOV_B32_48]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_49]], implicit [[S_MOV_B32_50]], implicit [[S_MOV_B32_51]], implicit [[S_MOV_B32_52]], implicit [[S_MOV_B32_53]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_54]], implicit [[S_MOV_B32_55]], implicit [[S_MOV_B32_56]], implicit [[S_MOV_B32_57]], implicit [[S_MOV_B32_58]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_59]], implicit [[S_MOV_B32_60]], implicit [[S_MOV_B32_61]], implicit [[S_MOV_B32_62]], implicit [[S_MOV_B32_63]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_64]], implicit [[S_MOV_B32_65]], implicit [[S_MOV_B32_66]], implicit [[S_MOV_B32_67]], implicit [[S_MOV_B32_68]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[S_MOV_B32_69]], implicit [[S_MOV_B32_70]], implicit [[S_MOV_B32_71]], implicit [[S_MOV_B32_72]], implicit [[S_MOV_B32_73]]
+ ; GFX90A-NEXT: S_ENDPGM 0
+ bb.0:
+ successors: %bb.1
+
+ %0:sgpr_32 = S_MOV_B32 0
+ %1:sgpr_32 = S_MOV_B32 1
+ %2:sgpr_32 = S_MOV_B32 2
+ %3:sgpr_32 = S_MOV_B32 3
+ %4:sgpr_32 = S_MOV_B32 4
+ %5:sgpr_32 = S_MOV_B32 5
+ %6:sgpr_32 = S_MOV_B32 6
+ %7:sgpr_32 = S_MOV_B32 7
+ %8:sgpr_32 = S_MOV_B32 8
+ %9:sgpr_32 = S_MOV_B32 9
+ %10:sgpr_32 = S_MOV_B32 10
+ %11:sgpr_32 = S_MOV_B32 11
+ %12:sgpr_32 = S_MOV_B32 12
+ %13:sgpr_32 = S_MOV_B32 13
+ %14:sgpr_32 = S_MOV_B32 14
+ %15:sgpr_32 = S_MOV_B32 15
+ %16:sgpr_32 = S_MOV_B32 16
+ %17:sgpr_32 = S_MOV_B32 17
+ %18:sgpr_32 = S_MOV_B32 18
+ %19:sgpr_32 = S_MOV_B32 19
+ %20:sgpr_32 = S_MOV_B32 20
+ %21:sgpr_32 = S_MOV_B32 21
+ %22:sgpr_32 = S_MOV_B32 22
+ %23:sgpr_32 = S_MOV_B32 23
+ %24:sgpr_32 = S_MOV_B32 24
+ %25:sgpr_32 = S_MOV_B32 25
+ %26:sgpr_32 = S_MOV_B32 26
+ %27:sgpr_32 = S_MOV_B32 27
+ %28:sgpr_32 = S_MOV_B32 28
+ %29:sgpr_32 = S_MOV_B32 29
+ %30:sgpr_32 = S_MOV_B32 30
+ %31:sgpr_32 = S_MOV_B32 31
+ %32:sgpr_32 = S_MOV_B32 32
+ %33:sgpr_32 = S_MOV_B32 33
+ %34:sgpr_32 = S_MOV_B32 34
+ %35:sgpr_32 = S_MOV_B32 35
+ %36:sgpr_32 = S_MOV_B32 36
+ %37:sgpr_32 = S_MOV_B32 37
+ %38:sgpr_32 = S_MOV_B32 38
+ %39:sgpr_32 = S_MOV_B32 39
+ %40:sgpr_32 = S_MOV_B32 40
+ %41:sgpr_32 = S_MOV_B32 41
+ %42:sgpr_32 = S_MOV_B32 42
+ %43:sgpr_32 = S_MOV_B32 43
+ %44:sgpr_32 = S_MOV_B32 44
+ %45:sgpr_32 = S_MOV_B32 45
+ %46:sgpr_32 = S_MOV_B32 46
+ %47:sgpr_32 = S_MOV_B32 47
+ %48:sgpr_32 = S_MOV_B32 48
+ %49:sgpr_32 = S_MOV_B32 49
+ %50:sgpr_32 = S_MOV_B32 50
+ %51:sgpr_32 = S_MOV_B32 51
+ %52:sgpr_32 = S_MOV_B32 52
+ %53:sgpr_32 = S_MOV_B32 53
+ %54:sgpr_32 = S_MOV_B32 54
+ %55:sgpr_32 = S_MOV_B32 55
+ %56:sgpr_32 = S_MOV_B32 56
+ %57:sgpr_32 = S_MOV_B32 57
+ %58:sgpr_32 = S_MOV_B32 58
+ %59:sgpr_32 = S_MOV_B32 59
+ %60:sgpr_32 = S_MOV_B32 60
+ %61:sgpr_32 = S_MOV_B32 61
+ %62:sgpr_32 = S_MOV_B32 62
+ %63:sgpr_32 = S_MOV_B32 63
+ %64:sgpr_32 = S_MOV_B32 64
+ %65:sgpr_32 = S_MOV_B32 65
+ %66:sgpr_32 = S_MOV_B32 66
+ %67:sgpr_32 = S_MOV_B32 67
+ %68:sgpr_32 = S_MOV_B32 68
+ %69:sgpr_32 = S_MOV_B32 69
+ %70:sgpr_32 = S_MOV_B32 70
+ %71:sgpr_32 = S_MOV_B32 71
+ %72:sgpr_32 = S_MOV_B32 72
+ %73:sgpr_32 = S_MOV_B32 73
+ %74:sgpr_32 = S_MOV_B32 74
+ %75:sgpr_32 = S_MOV_B32 75
+ %76:sgpr_32 = S_MOV_B32 76
+ %77:sgpr_32 = S_MOV_B32 77
+ %78:sgpr_32 = S_MOV_B32 78
+ %79:sgpr_32 = S_MOV_B32 79
+
+ bb.1:
+
+ S_NOP 0, implicit %0, implicit %1, implicit %2, implicit %3, implicit %4
+ S_NOP 0, implicit %5, implicit %6, implicit %7, implicit %8, implicit %9
+ S_NOP 0, implicit %10, implicit %11, implicit %12, implicit %13, implicit %14
+ S_NOP 0, implicit %15, implicit %16, implicit %17, implicit %18, implicit %19
+ S_NOP 0, implicit %20, implicit %21, implicit %22, implicit %23, implicit %24
+ S_NOP 0, implicit %25, implicit %26, implicit %27, implicit %28, implicit %29
+ S_NOP 0, implicit %30, implicit %31, implicit %32, implicit %33, implicit %34
+ S_NOP 0, implicit %35, implicit %36, implicit %37, implicit %38, implicit %39
+ S_NOP 0, implicit %40, implicit %41, implicit %42, implicit %43, implicit %44
+ S_NOP 0, implicit %45, implicit %46, implicit %47, implicit %48, implicit %49
+ S_NOP 0, implicit %50, implicit %51, implicit %52, implicit %53, implicit %54
+ S_NOP 0, implicit %55, implicit %56, implicit %57, implicit %58, implicit %59
+ S_NOP 0, implicit %60, implicit %61, implicit %62, implicit %63, implicit %64
+ S_NOP 0, implicit %65, implicit %66, implicit %67, implicit %68, implicit %69
+ S_NOP 0, implicit %70, implicit %71, implicit %72, implicit %73, implicit %74
+ S_NOP 0, implicit %75, implicit %76, implicit %77, implicit %78, implicit %79
+
+ S_ENDPGM 0
+...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-vgpr-stress.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-vgpr-stress.mir
new file mode 100644
index 0000000000000..b1851aafe5e84
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats-vgpr-stress.mir
@@ -0,0 +1,101 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -run-pass=machine-scheduler -amdgpu-disable-unclustered-high-rp-reschedule -verify-machineinstrs -amdgpu-stress-vgpr=14 -amdgpu-stress-agpr=14 %s -o - | FileCheck -check-prefix=GFX908 %s
+# RUN: llc -mtriple=amdgpu9.0a -run-pass=machine-scheduler -amdgpu-disable-unclustered-high-rp-reschedule -verify-machineinstrs -amdgpu-stress-vgpr=14 -amdgpu-stress-agpr=14 %s -o - | FileCheck -check-prefix=GFX90A %s
+
+--- |
+ define void @small_num_vgprs_as_spill() {
+ ret void
+ }
+...
+# User-requested maximum number of VGPRs need to be taken into account by
+# the scheduler's rematerialization stage. Register usage above that number
+# is considered like spill.
+---
+name: small_num_vgprs_as_spill
+tracksRegLiveness: true
+machineFunctionInfo:
+ isEntryFunction: true
+body: |
+ ; GFX908-LABEL: name: small_num_vgprs_as_spill
+ ; GFX908: bb.0:
+ ; GFX908-NEXT: successors: %bb.1(0x80000000)
+ ; GFX908-NEXT: {{ $}}
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_3:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_6:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_7:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_8:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_9:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_10:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_11:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_12:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_13:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
+ ; GFX908-NEXT: {{ $}}
+ ; GFX908-NEXT: bb.1:
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]], implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]], implicit [[V_CVT_I32_F64_e32_4]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_5]], implicit [[V_CVT_I32_F64_e32_6]], implicit [[V_CVT_I32_F64_e32_7]], implicit [[V_CVT_I32_F64_e32_8]], implicit [[V_CVT_I32_F64_e32_9]]
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_14:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
+ ; GFX908-NEXT: [[V_CVT_I32_F64_e32_15:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_10]], implicit [[V_CVT_I32_F64_e32_11]], implicit [[V_CVT_I32_F64_e32_12]], implicit [[V_CVT_I32_F64_e32_14]], implicit [[V_CVT_I32_F64_e32_15]]
+ ; GFX908-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_13]]
+ ; GFX908-NEXT: S_ENDPGM 0
+ ;
+ ; GFX90A-LABEL: name: small_num_vgprs_as_spill
+ ; GFX90A: bb.0:
+ ; GFX90A-NEXT: successors: %bb.1(0x80000000)
+ ; GFX90A-NEXT: {{ $}}
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_1:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_2:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_3:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_4:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_5:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_6:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_7:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_8:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_9:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_10:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_11:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_12:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_13:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_14:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
+ ; GFX90A-NEXT: [[V_CVT_I32_F64_e32_15:%[0-9]+]]:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
+ ; GFX90A-NEXT: {{ $}}
+ ; GFX90A-NEXT: bb.1:
+ ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_]], implicit [[V_CVT_I32_F64_e32_1]], implicit [[V_CVT_I32_F64_e32_2]], implicit [[V_CVT_I32_F64_e32_3]], implicit [[V_CVT_I32_F64_e32_4]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_5]], implicit [[V_CVT_I32_F64_e32_6]], implicit [[V_CVT_I32_F64_e32_7]], implicit [[V_CVT_I32_F64_e32_8]], implicit [[V_CVT_I32_F64_e32_9]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_10]], implicit [[V_CVT_I32_F64_e32_11]], implicit [[V_CVT_I32_F64_e32_12]], implicit [[V_CVT_I32_F64_e32_13]], implicit [[V_CVT_I32_F64_e32_14]]
+ ; GFX90A-NEXT: S_NOP 0, implicit [[V_CVT_I32_F64_e32_15]]
+ ; GFX90A-NEXT: S_ENDPGM 0
+ bb.0:
+ successors: %bb.1
+
+ %0:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 0, implicit $exec, implicit $mode, implicit-def $m0
+ %1:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 1, implicit $exec, implicit $mode, implicit-def $m0
+ %2:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 2, implicit $exec, implicit $mode, implicit-def $m0
+ %3:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 3, implicit $exec, implicit $mode, implicit-def $m0
+ %4:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 4, implicit $exec, implicit $mode, implicit-def $m0
+ %5:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 5, implicit $exec, implicit $mode, implicit-def $m0
+ %6:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 6, implicit $exec, implicit $mode, implicit-def $m0
+ %7:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 7, implicit $exec, implicit $mode, implicit-def $m0
+ %8:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 8, implicit $exec, implicit $mode, implicit-def $m0
+ %9:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 9, implicit $exec, implicit $mode, implicit-def $m0
+ %10:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 10, implicit $exec, implicit $mode, implicit-def $m0
+ %11:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 11, implicit $exec, implicit $mode, implicit-def $m0
+ %12:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 12, implicit $exec, implicit $mode, implicit-def $m0
+ %13:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 13, implicit $exec, implicit $mode
+ %14:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 14, implicit $exec, implicit $mode
+ %15:vgpr_32 = nofpexcept V_CVT_I32_F64_e32 15, implicit $exec, implicit $mode
+
+ bb.1:
+
+ S_NOP 0, implicit %0, implicit %1, implicit %2, implicit %3, implicit %4
+ S_NOP 0, implicit %5, implicit %6, implicit %7, implicit %8, implicit %9
+ S_NOP 0, implicit %10, implicit %11, implicit %12, implicit %13, implicit %14
+ S_NOP 0, implicit %15
+ S_ENDPGM 0
+...
diff --git a/llvm/test/CodeGen/AMDGPU/nsa-reassign-stress.ll b/llvm/test/CodeGen/AMDGPU/nsa-reassign-stress.ll
new file mode 100644
index 0000000000000..f86f5a3818e82
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/nsa-reassign-stress.ll
@@ -0,0 +1,24 @@
+; RUN: llc -mtriple=amdgpu10.10 --amdgpu-xnack=false -enable-misched=0 -amdgpu-stress-vgpr=10 < %s | FileCheck -check-prefix=GCN %s
+
+; GCN-LABEL: {{^}}sample_contig_nsa_10vgprs:
+; GCN-DAG: image_sample_c_l v{{[0-9]+}}, v[{{[0-9]+:[0-9]+}}],
+; GCN-DAG: image_sample v{{[0-9]+}}, v[{{[0-9]+:[0-9]+}}],
+define amdgpu_ps <2 x float> @sample_contig_nsa_10vgprs(<8 x i32> inreg %rsrc, <4 x i32> inreg %samp, float %zcompare, float %s1, float %t1, float %r1, float %lod, float %r2, float %s2, float %t2) {
+main_body:
+ %zcompare.1 = fadd float %zcompare, 1.0
+ %s1.1 = fadd float %s1, 1.0
+ %t1.1 = fadd float %t1, 1.0
+ %r1.1 = fadd float %r1, 1.0
+ %s2.1 = fadd float %s2, 1.0
+ %t2.1 = fadd float %t2, 1.0
+ %r2.1 = fadd float %r2, 1.0
+ %lod.1 = fadd float %lod, 1.0
+ %v1 = call float @llvm.amdgcn.image.sample.c.l.3d.f32.f32(i32 1, float %zcompare.1, float %s1.1, float %t1.1, float %r1.1, float %lod.1, <8 x i32> %rsrc, <4 x i32> %samp, i1 0, i32 0, i32 0)
+ %v2 = call float @llvm.amdgcn.image.sample.3d.f32.f32(i32 1, float %s2.1, float %t2.1, float %r2.1, <8 x i32> %rsrc, <4 x i32> %samp, i1 0, i32 0, i32 0)
+ %r.0 = insertelement <2 x float> poison, float %v1, i32 0
+ %r = insertelement <2 x float> %r.0, float %v2, i32 1
+ ret <2 x float> %r
+}
+
+declare float @llvm.amdgcn.image.sample.3d.f32.f32(i32, float, float, float, <8 x i32>, <4 x i32>, i1, i32, i32)
+declare float @llvm.amdgcn.image.sample.c.l.3d.f32.f32(i32, float, float, float, float, float, <8 x i32>, <4 x i32>, i1, i32, i32)
diff --git a/llvm/test/CodeGen/AMDGPU/nsa-reassign.ll b/llvm/test/CodeGen/AMDGPU/nsa-reassign.ll
index 87004befd0d9e..c3740f28d95b9 100644
--- a/llvm/test/CodeGen/AMDGPU/nsa-reassign.ll
+++ b/llvm/test/CodeGen/AMDGPU/nsa-reassign.ll
@@ -20,26 +20,6 @@ main_body:
ret <2 x float> %r
}
-; GCN-LABEL: {{^}}sample_contig_nsa_10vgprs:
-; GCN-DAG: image_sample_c_l v{{[0-9]+}}, v[{{[0-9]+:[0-9]+}}],
-; GCN-DAG: image_sample v{{[0-9]+}}, v[{{[0-9]+:[0-9]+}}],
-define amdgpu_ps <2 x float> @sample_contig_nsa_10vgprs(<8 x i32> inreg %rsrc, <4 x i32> inreg %samp, float %zcompare, float %s1, float %t1, float %r1, float %lod, float %r2, float %s2, float %t2) #0 {
-main_body:
- %zcompare.1 = fadd float %zcompare, 1.0
- %s1.1 = fadd float %s1, 1.0
- %t1.1 = fadd float %t1, 1.0
- %r1.1 = fadd float %r1, 1.0
- %s2.1 = fadd float %s2, 1.0
- %t2.1 = fadd float %t2, 1.0
- %r2.1 = fadd float %r2, 1.0
- %lod.1 = fadd float %lod, 1.0
- %v1 = call float @llvm.amdgcn.image.sample.c.l.3d.f32.f32(i32 1, float %zcompare.1, float %s1.1, float %t1.1, float %r1.1, float %lod.1, <8 x i32> %rsrc, <4 x i32> %samp, i1 0, i32 0, i32 0)
- %v2 = call float @llvm.amdgcn.image.sample.3d.f32.f32(i32 1, float %s2.1, float %t2.1, float %r2.1, <8 x i32> %rsrc, <4 x i32> %samp, i1 0, i32 0, i32 0)
- %r.0 = insertelement <2 x float> poison, float %v1, i32 0
- %r = insertelement <2 x float> %r.0, float %v2, i32 1
- ret <2 x float> %r
-}
-
; GCN-LABEL: {{^}}sample_contig_nsa_conflict:
; GCN-DAG: image_sample v{{[0-9]+}}, v[{{[0-9]+:[0-9]+}}],
; GCN-DAG: image_sample v{{[0-9]+}}, [{{v[0-9]+, v[0-9]+, v[0-9]+}}],
@@ -98,5 +78,3 @@ main_body:
declare float @llvm.amdgcn.image.sample.3d.f32.f32(i32, float, float, float, <8 x i32>, <4 x i32>, i1, i32, i32)
declare float @llvm.amdgcn.image.sample.c.l.3d.f32.f32(i32, float, float, float, float, float, <8 x i32>, <4 x i32>, i1, i32, i32)
-
-attributes #0 = {"amdgpu-num-vgpr"="10"}
diff --git a/llvm/test/CodeGen/AMDGPU/nsa-reassign.mir b/llvm/test/CodeGen/AMDGPU/nsa-reassign.mir
index d284fd82d0e96..b84bdeeaa7a63 100644
--- a/llvm/test/CodeGen/AMDGPU/nsa-reassign.mir
+++ b/llvm/test/CodeGen/AMDGPU/nsa-reassign.mir
@@ -1,10 +1,8 @@
-# RUN: llc -mtriple=amdgpu10.10 -verify-machineinstrs -run-pass greedy,amdgpu-nsa-reassign,virtregrewriter,si-shrink-instructions -o - %s | FileCheck -check-prefix=GCN %s
+# RUN: llc -mtriple=amdgpu10.10 -verify-machineinstrs -run-pass greedy,amdgpu-nsa-reassign,virtregrewriter,si-shrink-instructions -o - -amdgpu-stress-vgpr=8 %s | FileCheck -check-prefix=GCN %s
--- |
- define amdgpu_kernel void @nsa_reassign() #0 { ret void }
- define amdgpu_kernel void @do_not_reassign_spill() #0 { ret void }
-
- attributes #0 = { "amdgpu-num-vgpr"="8" }
+ define amdgpu_kernel void @nsa_reassign() { ret void }
+ define amdgpu_kernel void @do_not_reassign_spill() { ret void }
...
# GCN-LABEL: name: nsa_reassign
diff --git a/llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll b/llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll
index 9431658b0e6b5..c9f549fc93c22 100644
--- a/llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll
+++ b/llvm/test/CodeGen/AMDGPU/partial-regcopy-and-spill-missed-at-regalloc.ll
@@ -1,8 +1,8 @@
; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-;RUN: llc -mtriple=amdgpu9.08-amd-amdhsa --stop-after=greedy,2 < %s | FileCheck -check-prefix=REGALLOC-GFX908 %s
-;RUN: llc -mtriple=amdgpu9.08-amd-amdhsa --stop-after=prolog-epilog < %s | FileCheck -check-prefix=PEI-GFX908 %s
-;RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa -amdgpu-mfma-vgpr-form=0 --stop-after=greedy,2 < %s | FileCheck -check-prefix=REGALLOC-GFX90A %s
-;RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa -amdgpu-mfma-vgpr-form=0 --stop-after=prolog-epilog < %s | FileCheck -check-prefix=PEI-GFX90A %s
+;RUN: llc -mtriple=amdgpu9.08-amd-amdhsa --stop-after=greedy,2 -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 < %s | FileCheck -check-prefix=REGALLOC-GFX908 %s
+;RUN: llc -mtriple=amdgpu9.08-amd-amdhsa --stop-after=prolog-epilog -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 < %s | FileCheck -check-prefix=PEI-GFX908 %s
+;RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa -amdgpu-mfma-vgpr-form=0 --stop-after=greedy,2 -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 < %s | FileCheck -check-prefix=REGALLOC-GFX90A %s
+;RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa -amdgpu-mfma-vgpr-form=0 --stop-after=prolog-epilog -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 < %s | FileCheck -check-prefix=PEI-GFX90A %s
; Partial reg copy and spill missed during regalloc handled later at frame lowering.
define amdgpu_kernel void @partial_copy(<4 x i32> %arg) #0 {
@@ -104,4 +104,4 @@ define amdgpu_kernel void @partial_copy(<4 x i32> %arg) #0 {
declare <4 x i32> @llvm.amdgcn.mfma.i32.4x4x4i8(i32, i32, <4 x i32>, i32, i32, i32)
-attributes #0 = { nounwind "amdgpu-num-vgpr"="5" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-x" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+attributes #0 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-x" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
diff --git a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr1.mir b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr1.mir
new file mode 100644
index 0000000000000..f993cd2bb6f86
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr1.mir
@@ -0,0 +1,93 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=1 -amdgpu-stress-agpr=1 -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=MUBUF-V2A %s
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=1 -amdgpu-stress-agpr=1 -mattr=+enable-flat-scratch -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=FLATSCR-V2A %s
+
+--- |
+
+ define void @test_spill_v2_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ define void @test_spill_v3_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ attributes #0 = { nounwind }
+...
+
+---
+name: test_spill_v2_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 8, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v2_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit killed $vgpr0_vgpr1
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v2_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1
+ ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1 = IMPLICIT_DEF
+ SI_SPILL_V64_SAVE killed $vgpr0_vgpr1, %stack.0, $sgpr32, 0, implicit $exec :: (store (s64) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1 = SI_SPILL_V64_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s64) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
+---
+name: test_spill_v3_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 12, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v3_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v3_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX2_SADDR killed $vgpr0_vgpr1, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" store (s64) into %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = SCRATCH_LOAD_DWORDX2_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" load (s64) from %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
+ SI_SPILL_V96_SAVE killed $vgpr0_vgpr1_vgpr2, %stack.0, $sgpr32, 0, implicit $exec :: (store (s96) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2 = SI_SPILL_V96_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s96) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
\ No newline at end of file
diff --git a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr3.mir b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr3.mir
new file mode 100644
index 0000000000000..839fc0cd9432e
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr3.mir
@@ -0,0 +1,109 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=3 -amdgpu-stress-agpr=3 -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=MUBUF-V2A %s
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=3 -amdgpu-stress-agpr=3 -mattr=+enable-flat-scratch -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=FLATSCR-V2A %s
+
+--- |
+
+ define void @test_spill_v4_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ define void @test_spill_v5_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ attributes #0 = { nounwind }
+...
+
+---
+name: test_spill_v4_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 16, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v4_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v4_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
+ ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
+ SI_SPILL_V128_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3, %stack.0, $sgpr32, 0, implicit $exec :: (store (s128) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2_vgpr3 = SI_SPILL_V128_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s128) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
+---
+name: test_spill_v5_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 20, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v5_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v5_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX2_SADDR killed $vgpr0_vgpr1, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" store (s64) into %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = SCRATCH_LOAD_DWORDX2_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" load (s64) from %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
+ SI_SPILL_V160_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, %stack.0, $sgpr32, 0, implicit $exec :: (store (s160) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = SI_SPILL_V160_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s160) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
\ No newline at end of file
diff --git a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr4.mir b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr4.mir
new file mode 100644
index 0000000000000..9934a9ff9a2d3
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr4.mir
@@ -0,0 +1,68 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=4 -amdgpu-stress-agpr=4 -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=MUBUF-V2A %s
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=4 -amdgpu-stress-agpr=4 -mattr=+enable-flat-scratch -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=FLATSCR-V2A %s
+
+--- |
+
+ define void @test_spill_v8_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ attributes #0 = { nounwind }
+...
+
+---
+name: test_spill_v8_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 32, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v8_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr2, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 8, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr3, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 12, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr6, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr7, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr2 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 8, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr3 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 12, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr6 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr7 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v8_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr0_vgpr1_vgpr2_vgpr3, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" store (s128) into %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr7, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr6, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" load (s128) from %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr7 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $vgpr6 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
+ SI_SPILL_V256_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, %stack.0, $sgpr32, 0, implicit $exec :: (store (s256) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = SI_SPILL_V256_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s256) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
\ No newline at end of file
diff --git a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr5.mir b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr5.mir
new file mode 100644
index 0000000000000..e1fe3f86b323c
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr-vgpr5.mir
@@ -0,0 +1,147 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=MUBUF-V2A %s
+# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 -mattr=+enable-flat-scratch -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=FLATSCR-V2A %s
+
+--- |
+
+ define void @test_spill_v6_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ define void @test_spill_v16_partial_agpr() #0 {
+ entry:
+ unreachable
+ }
+
+ attributes #0 = { nounwind }
+...
+
+---
+name: test_spill_v6_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 24, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v6_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v6_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
+ SI_SPILL_V192_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, %stack.0, $sgpr32, 0, implicit $exec :: (store (s196) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = SI_SPILL_V192_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s196) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
+---
+name: test_spill_v16_partial_agpr
+tracksRegLiveness: true
+stack:
+ - { id: 0, name: '', type: spill-slot, offset: 0, size: 64, alignment: 4 }
+machineFunctionInfo:
+ scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
+ stackPtrOffsetReg: '$sgpr32'
+ hasSpilledVGPRs: true
+body: |
+ bb.0.entry:
+ ; MUBUF-V2A-LABEL: name: test_spill_v16_partial_agpr
+ ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
+ ; MUBUF-V2A-NEXT: {{ $}}
+ ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr2, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 8, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr3, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 12, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr4, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 16, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 16, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr5, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 20, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 20, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr6, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 24, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 24, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr7, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 28, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 28, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr8, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 32, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 32, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr9, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 36, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 36, addrspace 5)
+ ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr10, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 40, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 40, addrspace 5)
+ ; MUBUF-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr11, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr12, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr13, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr14, implicit $exec
+ ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr15, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr2 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 8, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr3 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 12, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr4 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 16, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 16, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr5 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 20, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 20, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr6 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 24, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 24, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr7 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 28, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 28, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr8 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 32, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 32, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr9 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 36, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 36, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr10 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 40, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 40, addrspace 5)
+ ; MUBUF-V2A-NEXT: $vgpr11 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr12 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr13 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr14 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
+ ; MUBUF-V2A-NEXT: $vgpr15 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; MUBUF-V2A-NEXT: S_ENDPGM 0
+ ;
+ ; FLATSCR-V2A-LABEL: name: test_spill_v16_partial_agpr
+ ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
+ ; FLATSCR-V2A-NEXT: {{ $}}
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr0_vgpr1_vgpr2_vgpr3, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" store (s128) into %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr4_vgpr5_vgpr6_vgpr7, $sgpr32, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s128) into %stack.0 + 16, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr11, implicit $exec
+ ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX3_SADDR killed $vgpr8_vgpr9_vgpr10, $sgpr32, 32, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s96) into %stack.0 + 32, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr15, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr14, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr13, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr12, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" load (s128) from %stack.0, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr4_vgpr5_vgpr6_vgpr7 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" load (s128) from %stack.0 + 16, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr11 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
+ ; FLATSCR-V2A-NEXT: $vgpr8_vgpr9_vgpr10 = SCRATCH_LOAD_DWORDX3_SADDR $sgpr32, 32, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" load (s96) from %stack.0 + 32, align 4, addrspace 5)
+ ; FLATSCR-V2A-NEXT: $vgpr15 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $vgpr14 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $vgpr13 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: $vgpr12 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
+ ; FLATSCR-V2A-NEXT: S_ENDPGM 0
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
+ SI_SPILL_V512_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, %stack.0, $sgpr32, 0, implicit $exec :: (store (s512) into %stack.0, align 4, addrspace 5)
+ $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = SI_SPILL_V512_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s512) from %stack.0, align 4, addrspace 5)
+ S_ENDPGM 0
+
+...
\ No newline at end of file
diff --git a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr.mir b/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr.mir
deleted file mode 100644
index 317c965dfa738..0000000000000
--- a/llvm/test/CodeGen/AMDGPU/pei-build-spill-partial-agpr.mir
+++ /dev/null
@@ -1,399 +0,0 @@
-# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=MUBUF-V2A %s
-# RUN: llc -mtriple=amdgpu9.08 -verify-machineinstrs -mattr=+enable-flat-scratch -run-pass=prolog-epilog -o - %s | FileCheck -check-prefix=FLATSCR-V2A %s
-
---- |
-
- define void @test_spill_v2_partial_agpr() #1 {
- entry:
- unreachable
- }
-
- define void @test_spill_v3_partial_agpr() #1 {
- entry:
- unreachable
- }
-
- define void @test_spill_v4_partial_agpr() #2 {
- entry:
- unreachable
- }
-
- define void @test_spill_v5_partial_agpr() #2 {
- entry:
- unreachable
- }
-
- define void @test_spill_v6_partial_agpr() #4 {
- entry:
- unreachable
- }
-
- define void @test_spill_v8_partial_agpr() #3 {
- entry:
- unreachable
- }
-
- define void @test_spill_v16_partial_agpr() #4 {
- entry:
- unreachable
- }
-
- attributes #1 = { nounwind "amdgpu-num-vgpr"="1" }
- attributes #2 = { nounwind "amdgpu-num-vgpr"="3" }
- attributes #3 = { nounwind "amdgpu-num-vgpr"="4" }
- attributes #4 = { nounwind "amdgpu-num-vgpr"="5" }
-...
-
----
-name: test_spill_v2_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 8, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v2_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit killed $vgpr0_vgpr1
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v2_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1
- ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1, implicit $vgpr0_vgpr1 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1 = IMPLICIT_DEF
- SI_SPILL_V64_SAVE killed $vgpr0_vgpr1, %stack.0, $sgpr32, 0, implicit $exec :: (store (s64) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1 = SI_SPILL_V64_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s64) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v3_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 12, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v3_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v3_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX2_SADDR killed $vgpr0_vgpr1, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" store (s64) into %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = SCRATCH_LOAD_DWORDX2_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2, implicit $vgpr0_vgpr1_vgpr2 :: ("amdgpu-thread-private" load (s64) from %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2 = IMPLICIT_DEF
- SI_SPILL_V96_SAVE killed $vgpr0_vgpr1_vgpr2, %stack.0, $sgpr32, 0, implicit $exec :: (store (s96) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2 = SI_SPILL_V96_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s96) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v4_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 16, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v4_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v4_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3
- ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3, implicit $vgpr0_vgpr1_vgpr2_vgpr3 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2_vgpr3 = IMPLICIT_DEF
- SI_SPILL_V128_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3, %stack.0, $sgpr32, 0, implicit $exec :: (store (s128) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2_vgpr3 = SI_SPILL_V128_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s128) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v5_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 20, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v5_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v5_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX2_SADDR killed $vgpr0_vgpr1, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" store (s64) into %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1 = SCRATCH_LOAD_DWORDX2_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 :: ("amdgpu-thread-private" load (s64) from %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = IMPLICIT_DEF
- SI_SPILL_V160_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4, %stack.0, $sgpr32, 0, implicit $exec :: (store (s160) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4 = SI_SPILL_V160_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s160) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v6_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 24, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v6_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v6_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr3, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr0, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $vgpr3 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $vgpr2 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $vgpr1 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = IMPLICIT_DEF
- SI_SPILL_V192_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5, %stack.0, $sgpr32, 0, implicit $exec :: (store (s196) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5 = SI_SPILL_V192_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s196) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v8_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 32, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v8_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr2, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 8, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr3, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 12, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr6, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr7, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr2 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 8, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr3 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 12, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr6 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr7 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v8_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr0_vgpr1_vgpr2_vgpr3, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" store (s128) into %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr7, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr6, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr5, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr4, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 :: ("amdgpu-thread-private" load (s128) from %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr7 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $vgpr6 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $vgpr5 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: $vgpr4 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = IMPLICIT_DEF
- SI_SPILL_V256_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7, %stack.0, $sgpr32, 0, implicit $exec :: (store (s256) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7 = SI_SPILL_V256_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s256) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
-
----
-name: test_spill_v16_partial_agpr
-tracksRegLiveness: true
-stack:
- - { id: 0, name: '', type: spill-slot, offset: 0, size: 64, alignment: 4 }
-machineFunctionInfo:
- scratchRSrcReg: $sgpr0_sgpr1_sgpr2_sgpr3
- stackPtrOffsetReg: '$sgpr32'
- hasSpilledVGPRs: true
-body: |
- bb.0.entry:
- ; MUBUF-V2A-LABEL: name: test_spill_v16_partial_agpr
- ; MUBUF-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
- ; MUBUF-V2A-NEXT: {{ $}}
- ; MUBUF-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" store (s32) into %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr1, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr2, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 8, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr3, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 12, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr4, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 16, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 16, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr5, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 20, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 20, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr6, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 24, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 24, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr7, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 28, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 28, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr8, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 32, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 32, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr9, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 36, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 36, addrspace 5)
- ; MUBUF-V2A-NEXT: BUFFER_STORE_DWORD_OFFSET killed $vgpr10, $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 40, 0, 0, implicit $exec :: ("amdgpu-thread-private" store (s32) into %stack.0 + 40, addrspace 5)
- ; MUBUF-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr11, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr12, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr13, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr14, implicit $exec
- ; MUBUF-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr15, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; MUBUF-V2A-NEXT: $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 0, 0, 0, implicit $exec, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr1 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 4, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 4, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr2 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 8, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 8, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr3 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 12, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 12, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr4 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 16, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 16, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr5 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 20, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 20, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr6 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 24, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 24, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr7 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 28, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 28, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr8 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 32, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 32, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr9 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 36, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 36, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr10 = BUFFER_LOAD_DWORD_OFFSET $sgpr0_sgpr1_sgpr2_sgpr3, $sgpr32, 40, 0, 0, implicit $exec :: ("amdgpu-thread-private" load (s32) from %stack.0 + 40, addrspace 5)
- ; MUBUF-V2A-NEXT: $vgpr11 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr12 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr13 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr14 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec
- ; MUBUF-V2A-NEXT: $vgpr15 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; MUBUF-V2A-NEXT: S_ENDPGM 0
- ;
- ; FLATSCR-V2A-LABEL: name: test_spill_v16_partial_agpr
- ; FLATSCR-V2A: liveins: $agpr0, $agpr1, $agpr2, $agpr3, $agpr4
- ; FLATSCR-V2A-NEXT: {{ $}}
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr0_vgpr1_vgpr2_vgpr3, $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" store (s128) into %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX4_SADDR killed $vgpr4_vgpr5_vgpr6_vgpr7, $sgpr32, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s128) into %stack.0 + 16, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $agpr4 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr11, implicit $exec
- ; FLATSCR-V2A-NEXT: SCRATCH_STORE_DWORDX3_SADDR killed $vgpr8_vgpr9_vgpr10, $sgpr32, 32, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s96) into %stack.0 + 32, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $agpr0 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr15, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $agpr1 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr14, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $agpr2 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr13, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $agpr3 = V_ACCVGPR_WRITE_B32_e64 killed $vgpr12, implicit $exec, implicit killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $vgpr0_vgpr1_vgpr2_vgpr3 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit-def $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 :: ("amdgpu-thread-private" load (s128) from %stack.0, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr4_vgpr5_vgpr6_vgpr7 = SCRATCH_LOAD_DWORDX4_SADDR $sgpr32, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" load (s128) from %stack.0 + 16, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr11 = V_ACCVGPR_READ_B32_e64 $agpr4, implicit $exec
- ; FLATSCR-V2A-NEXT: $vgpr8_vgpr9_vgpr10 = SCRATCH_LOAD_DWORDX3_SADDR $sgpr32, 32, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" load (s96) from %stack.0 + 32, align 4, addrspace 5)
- ; FLATSCR-V2A-NEXT: $vgpr15 = V_ACCVGPR_READ_B32_e64 $agpr0, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $vgpr14 = V_ACCVGPR_READ_B32_e64 $agpr1, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $vgpr13 = V_ACCVGPR_READ_B32_e64 $agpr2, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: $vgpr12 = V_ACCVGPR_READ_B32_e64 $agpr3, implicit $exec, implicit $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15
- ; FLATSCR-V2A-NEXT: S_ENDPGM 0
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = IMPLICIT_DEF
- SI_SPILL_V512_SAVE killed $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15, %stack.0, $sgpr32, 0, implicit $exec :: (store (s512) into %stack.0, align 4, addrspace 5)
- $vgpr0_vgpr1_vgpr2_vgpr3_vgpr4_vgpr5_vgpr6_vgpr7_vgpr8_vgpr9_vgpr10_vgpr11_vgpr12_vgpr13_vgpr14_vgpr15 = SI_SPILL_V512_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s512) from %stack.0, align 4, addrspace 5)
- S_ENDPGM 0
-
-...
diff --git a/llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll b/llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll
index c22e48557b876..756f04f584ff6 100644
--- a/llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll
+++ b/llvm/test/CodeGen/AMDGPU/preserve-wwm-copy-dst-reg.ll
@@ -1,6 +1,6 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa < %s | FileCheck -check-prefix=GFX906 %s
-; RUN: llc -mtriple=amdgpu9.08-amd-amdhsa < %s | FileCheck -check-prefix=GFX908 %s
+; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -amdgpu-stress-vgpr=42 -amdgpu-stress-sgpr=40 < %s | FileCheck -check-prefix=GFX906 %s
+; RUN: llc -mtriple=amdgpu9.08-amd-amdhsa -amdgpu-stress-vgpr=42 -amdgpu-stress-sgpr=40 -amdgpu-stress-agpr=42 < %s | FileCheck -check-prefix=GFX908 %s
; Due to high register pressure, regalloc would split the liverange of wwm VGPR register used for SGPR spills
; and introduce a copy. The copy should be of whole-wave with exec mask manipulation around it.
@@ -828,5 +828,5 @@ define void @preserve_wwm_copy_dstreg(ptr %parg0, ptr %parg1, ptr %parg2) #0 {
declare void @foo()
-attributes #0 = { nounwind "amdgpu-num-vgpr"="42" "amdgpu-num-sgpr"="40"}
+attributes #0 = { nounwind}
diff --git a/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers-stress.ll b/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers-stress.ll
new file mode 100644
index 0000000000000..424fb6f8dc050
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers-stress.ll
@@ -0,0 +1,38 @@
+; RUN: llc -mtriple=amdgpu11.00--amdpal -verify-misched -amdgpu-stress-vgpr=64 < %s | FileCheck --check-prefixes=GFX11-PAL %s
+; RUN: llc -mtriple=amdgpu11.00--amdpal -amdgpu-use-amdgpu-trackers=1 -verify-misched -amdgpu-stress-vgpr=64 < %s | FileCheck --check-prefixes=GFX11-PAL-GCNTRACKERS %s
+
+; GCN Trackers are sensitive to minor changes in RP, and will avoid scheduling certain instructions, which, if scheduled,
+; allow scheduling of other instructions which reduce RP
+
+; CHECK-LABEL: {{^}}return_72xi32:
+; GFX11-PAL: NumSgprs: 33
+; GFX11-PAL-GCNTRACKERS: NumSgprs: 33
+; GFX11-PAL: NumVgprs: 64
+; GFX11-PAL-GCNTRACKERS: NumVgprs: 64
+; GFX11-PAL: ScratchSize: 220
+; GFX11-PAL-GCNTRACKERS: ScratchSize: 248
+
+
+; CHECK-LABEL: {{^}}call_72xi32:
+; GFX11-PAL: NumSgprs: 40
+; GFX11-PAL-GCNTRACKERS: NumSgprs: 37
+; GFX11-PAL: NumVgprs: 64
+; GFX11-PAL-GCNTRACKERS: NumVgprs: 64
+; GFX11-PAL: ScratchSize: 2780
+; GFX11-PAL-GCNTRACKERS: ScratchSize: 2808
+
+
+define amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val) #1 {
+ ret <72 x i32> %val
+}
+
+define amdgpu_gfx void @call_72xi32() #1 {
+entry:
+ %ret.0 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> zeroinitializer)
+ %val.0 = insertelement <72 x i32> %ret.0, i32 42, i32 0
+ %val.1 = insertelement <72 x i32> %val.0, i32 24, i32 58
+ %ret.1 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val.1)
+ ret void
+}
+
+attributes #1 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers.ll b/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers.ll
index 7044517b6a432..50903aa3bc14c 100644
--- a/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers.ll
+++ b/llvm/test/CodeGen/AMDGPU/schedule-amdgpu-trackers.ll
@@ -1,5 +1,3 @@
-; RUN: llc -mtriple=amdgpu11.00--amdpal -verify-misched < %s | FileCheck --check-prefixes=GFX11-PAL %s
-; RUN: llc -mtriple=amdgpu11.00--amdpal -amdgpu-use-amdgpu-trackers=1 -verify-misched < %s | FileCheck --check-prefixes=GFX11-PAL-GCNTRACKERS %s
; RUN: llc -mtriple=amdgpu8.02 -amdgpu-scalarize-global-loads=false -verify-misched < %s | FileCheck --check-prefixes=TONGA %s
; RUN: llc -mtriple=amdgpu8.02 -amdgpu-scalarize-global-loads=false -amdgpu-use-amdgpu-trackers=1 -verify-misched < %s | FileCheck --check-prefixes=TONGA-GCNTRACKERS %s
; RUN: llc -mtriple=amdgpu9.08 -verify-misched < %s | FileCheck --check-prefixes=GFX908 %s
@@ -10,37 +8,6 @@
; GCN Trackers are sensitive to minor changes in RP, and will avoid scheduling certain instructions, which, if scheduled,
; allow scheduling of other instructions which reduce RP
-; CHECK-LABEL: {{^}}return_72xi32:
-; GFX11-PAL: NumSgprs: 33
-; GFX11-PAL-GCNTRACKERS: NumSgprs: 33
-; GFX11-PAL: NumVgprs: 64
-; GFX11-PAL-GCNTRACKERS: NumVgprs: 64
-; GFX11-PAL: ScratchSize: 220
-; GFX11-PAL-GCNTRACKERS: ScratchSize: 248
-
-
-; CHECK-LABEL: {{^}}call_72xi32:
-; GFX11-PAL: NumSgprs: 40
-; GFX11-PAL-GCNTRACKERS: NumSgprs: 37
-; GFX11-PAL: NumVgprs: 64
-; GFX11-PAL-GCNTRACKERS: NumVgprs: 64
-; GFX11-PAL: ScratchSize: 2780
-; GFX11-PAL-GCNTRACKERS: ScratchSize: 2808
-
-
-define amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val) #1 {
- ret <72 x i32> %val
-}
-
-define amdgpu_gfx void @call_72xi32() #1 {
-entry:
- %ret.0 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> zeroinitializer)
- %val.0 = insertelement <72 x i32> %ret.0, i32 42, i32 0
- %val.1 = insertelement <72 x i32> %val.0, i32 24, i32 58
- %ret.1 = call amdgpu_gfx <72 x i32> @return_72xi32(<72 x i32> %val.1)
- ret void
-}
-
; CHECK-LABEL: {{^}}global_extload_v16f16_to_v16f64:
; TONGA: NumSgprs: 96
; TONGA-GCNTRACKERS: NumSgprs: 96
@@ -632,6 +599,5 @@ declare align 4 ptr addrspace(4) @llvm.amdgcn.implicitarg.ptr() #3
!5 = !{i32 0, i32 1024}
attributes #0 = { nounwind "amdgpu-waves-per-eu"="1,1" "amdgpu-flat-work-group-size"="1,1" }
-attributes #1 = { nounwind "amdgpu-num-vgpr"="64" }
attributes #2 = { nofree nosync nounwind readnone speculatable willreturn }
attributes #3 = { nounwind readnone speculatable willreturn }
diff --git a/llvm/test/CodeGen/AMDGPU/schedule-regpressure-limit-clustering.ll b/llvm/test/CodeGen/AMDGPU/schedule-regpressure-limit-clustering.ll
index 9625d3ccf4eca..d31827816cdc0 100644
--- a/llvm/test/CodeGen/AMDGPU/schedule-regpressure-limit-clustering.ll
+++ b/llvm/test/CodeGen/AMDGPU/schedule-regpressure-limit-clustering.ll
@@ -1,4 +1,4 @@
-; RUN: llc -mtriple=amdgpu9.00 < %s | FileCheck -check-prefix=GCN %s
+; RUN: llc -mtriple=amdgpu9.00 -amdgpu-stress-vgpr=9 < %s | FileCheck -check-prefix=GCN %s
; Interleave loads and stores to fit into 9 VGPR limit.
; This requires to avoid load/store clustering.
@@ -15,7 +15,7 @@
; GCN: NumVgprs: {{[0-9]$}}
; GCN: ScratchSize: 0{{$}}
-define amdgpu_kernel void @load_store_max_9vgprs(ptr addrspace(1) nocapture noalias readonly %arg, ptr addrspace(1) nocapture noalias %arg1, i1 %cnd) #1 {
+define amdgpu_kernel void @load_store_max_9vgprs(ptr addrspace(1) nocapture noalias readonly %arg, ptr addrspace(1) nocapture noalias %arg1, i1 %cnd) {
bb:
%id = call i32 @llvm.amdgcn.workitem.id.x()
%base = getelementptr inbounds <4 x i32>, ptr addrspace(1) %arg, i32 %id
@@ -42,4 +42,3 @@ bb2:
declare i32 @llvm.amdgcn.workitem.id.x() #0
attributes #0 = { nounwind readnone }
-attributes #1 = { "amdgpu-num-vgpr"="9" }
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-sgpr-spills-cycle-header.ll b/llvm/test/CodeGen/AMDGPU/si-lower-sgpr-spills-cycle-header.ll
index ba51f6e018a51..6e3004d21244b 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-sgpr-spills-cycle-header.ll
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-sgpr-spills-cycle-header.ll
@@ -1,6 +1,7 @@
; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 6
; REQUIRES: amdgpu-registered-target
; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -O0 -amdgpu-spill-sgpr-to-vgpr=1 \
+; RUN: -amdgpu-stress-vgpr=64 -amdgpu-stress-sgpr=24 \
; RUN: -verify-machineinstrs -stop-after=si-lower-sgpr-spills -o - %s | FileCheck %s
;; Ensure that si-lower-sgpr-spills prevents IMPLICIT_DEF assignments from clobbering
@@ -160,7 +161,7 @@ define amdgpu_kernel void @loop_sgpr_spill_implicit_def_in_preheader(
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed renamable $sgpr6
; CHECK-NEXT: GLOBAL_STORE_DWORD_SADDR [[V_MOV_B32_e32_]], [[COPY]], killed renamable $sgpr4_sgpr5, 0, 0, implicit $exec :: (volatile store (s32) into %ir.out.load, addrspace 1)
; CHECK-NEXT: S_ENDPGM 0
- ptr addrspace(1) %out) local_unnamed_addr #0 {
+ ptr addrspace(1) %out) local_unnamed_addr {
entry:
%a0 = load i32, ptr addrspace(4) poison
%a1 = load i32, ptr addrspace(4) poison
@@ -203,4 +204,3 @@ exit:
ret void
}
-attributes #0 = { nounwind "amdgpu-num-sgpr"="24" "amdgpu-num-vgpr"="64" }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr10.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr10.ll
new file mode 100644
index 0000000000000..0b370ad335d95
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr10.ll
@@ -0,0 +1,81 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=10 -amdgpu-stress-agpr=10 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.0a -amdgpu-mfma-vgpr-form=0 -amdgpu-stress-vgpr=10 -amdgpu-stress-agpr=10 < %s | FileCheck -check-prefixes=GCN,GFX90A %s
+
+define amdgpu_kernel void @max_10_vgprs_used_9a() #1 {
+; GFX908-LABEL: max_10_vgprs_used_9a:
+; GFX908: ; %bb.0:
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_accvgpr_read_b32 v0, a0
+; GFX908-NEXT: v_accvgpr_read_b32 v1, a1
+; GFX908-NEXT: v_accvgpr_read_b32 v2, a2
+; GFX908-NEXT: v_accvgpr_read_b32 v3, a3
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a2
+; GFX908-NEXT: v_accvgpr_read_b32 v4, a1
+; GFX908-NEXT: v_accvgpr_write_b32 a1, v0
+; GFX908-NEXT: v_accvgpr_write_b32 a4, v3
+; GFX908-NEXT: v_accvgpr_write_b32 a3, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a2, v1
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_accvgpr_write_b32 a0, v4
+; GFX908-NEXT: v_accvgpr_write_b32 a1, v5
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_endpgm
+;
+; GFX90A-LABEL: max_10_vgprs_used_9a:
+; GFX90A: ; %bb.0:
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_accvgpr_read_b32 v0, a0
+; GFX90A-NEXT: v_accvgpr_read_b32 v1, a1
+; GFX90A-NEXT: v_accvgpr_read_b32 v2, a2
+; GFX90A-NEXT: v_accvgpr_read_b32 v3, a3
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_accvgpr_read_b32 v5, a3
+; GFX90A-NEXT: v_accvgpr_read_b32 v4, a2
+; GFX90A-NEXT: v_accvgpr_write_b32 a5, v3
+; GFX90A-NEXT: v_accvgpr_write_b32 a4, v2
+; GFX90A-NEXT: v_accvgpr_write_b32 a3, v1
+; GFX90A-NEXT: v_accvgpr_write_b32 a2, v0
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_accvgpr_write_b32 a0, v4
+; GFX90A-NEXT: v_accvgpr_write_b32 a1, v5
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: s_endpgm
+ %a1 = call <4 x i32> asm sideeffect "", "=a"()
+ %a2 = call <4 x i32> asm sideeffect "", "=a"()
+ %a3 = call i32 asm sideeffect "", "=a"()
+ %a4 = call <2 x i32> asm sideeffect "", "=a"()
+ call void asm sideeffect "", "a,a,a"(<4 x i32> %a1, <4 x i32> %a2, i32 %a3)
+ call void asm sideeffect "", "a"(<2 x i32> %a4)
+ ret void
+}
+; GCN: ScratchSize: 0
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32, i32, i32)
+declare <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float, float, <4 x float>, i32, i32, i32)
+declare <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float, float, <32 x float>, i32, i32, i32)
+
+
+attributes #1 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+
+attributes #1 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; GCN: {{.*}}
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr12.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr12.ll
new file mode 100644
index 0000000000000..b4873227b2e32
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr12.ll
@@ -0,0 +1,135 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=12 -amdgpu-stress-agpr=12 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.0a -amdgpu-mfma-vgpr-form=0 -amdgpu-stress-vgpr=12 -amdgpu-stress-agpr=12 < %s | FileCheck -check-prefixes=GCN,GFX90A %s
+
+define amdgpu_kernel void @max_12regs_13a_used(i32 %cond, ptr addrspace(1) %arg, ptr addrspace(1) %out) #2 {
+; GFX908-LABEL: max_12regs_13a_used:
+; GFX908: ; %bb.0: ; %bb
+; GFX908-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x2c
+; GFX908-NEXT: v_mov_b32_e32 v0, 1.0
+; GFX908-NEXT: s_load_dword s0, s[0:1], 0x24
+; GFX908-NEXT: s_waitcnt lgkmcnt(0)
+; GFX908-NEXT: s_cmp_lg_u32 s0, 0
+; GFX908-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
+; GFX908-NEXT: s_waitcnt lgkmcnt(0)
+; GFX908-NEXT: v_mov_b32_e32 v5, s8
+; GFX908-NEXT: v_mov_b32_e32 v1, s9
+; GFX908-NEXT: v_mov_b32_e32 v2, s10
+; GFX908-NEXT: v_accvgpr_write_b32 a0, v5
+; GFX908-NEXT: v_mov_b32_e32 v5, s11
+; GFX908-NEXT: v_accvgpr_write_b32 a1, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a2, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a3, v5
+; GFX908-NEXT: s_nop 0
+; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
+; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[4:7], v0, v0, a[0:3]
+; GFX908-NEXT: s_cbranch_scc0 .LBB0_2
+; GFX908-NEXT: ; %bb.1: ; %st
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_endpgm
+; GFX908-NEXT: .LBB0_2: ; %use
+; GFX908-NEXT: s_nop 2
+; GFX908-NEXT: v_accvgpr_read_b32 v1, a4
+; GFX908-NEXT: v_accvgpr_read_b32 v2, a5
+; GFX908-NEXT: v_accvgpr_read_b32 v3, a6
+; GFX908-NEXT: v_accvgpr_read_b32 v4, a7
+; GFX908-NEXT: v_accvgpr_write_b32 a4, 4
+; GFX908-NEXT: v_accvgpr_write_b32 a8, 5
+; GFX908-NEXT: v_accvgpr_write_b32 a9, 1
+; GFX908-NEXT: v_accvgpr_write_b32 a10, 2
+; GFX908-NEXT: v_accvgpr_write_b32 a11, 3
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_accvgpr_write_b32 a7, v4
+; GFX908-NEXT: v_accvgpr_write_b32 a6, v3
+; GFX908-NEXT: v_accvgpr_write_b32 a5, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
+; GFX908-NEXT: v_mov_b32_e32 v1, v0
+; GFX908-NEXT: v_mov_b32_e32 v2, v0
+; GFX908-NEXT: v_mov_b32_e32 v3, v0
+; GFX908-NEXT: v_mov_b32_e32 v4, 0
+; GFX908-NEXT: global_store_dwordx4 v4, v[0:3], s[6:7]
+; GFX908-NEXT: s_waitcnt vmcnt(0)
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_endpgm
+;
+; GFX90A-LABEL: max_12regs_13a_used:
+; GFX90A: ; %bb.0: ; %bb
+; GFX90A-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x2c
+; GFX90A-NEXT: v_mov_b32_e32 v0, 1.0
+; GFX90A-NEXT: s_load_dword s0, s[0:1], 0x24
+; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90A-NEXT: s_cmp_lg_u32 s0, 0
+; GFX90A-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
+; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90A-NEXT: v_accvgpr_write_b32 a0, s8
+; GFX90A-NEXT: v_accvgpr_write_b32 a1, s9
+; GFX90A-NEXT: v_accvgpr_write_b32 a2, s10
+; GFX90A-NEXT: v_accvgpr_write_b32 a3, s11
+; GFX90A-NEXT: s_nop 1
+; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
+; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[4:7], v0, v0, a[0:3]
+; GFX90A-NEXT: s_cbranch_scc0 .LBB0_2
+; GFX90A-NEXT: ; %bb.1: ; %st
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: s_endpgm
+; GFX90A-NEXT: .LBB0_2: ; %use
+; GFX90A-NEXT: s_nop 3
+; GFX90A-NEXT: v_accvgpr_read_b32 v9, a7
+; GFX90A-NEXT: v_accvgpr_read_b32 v8, a6
+; GFX90A-NEXT: v_accvgpr_read_b32 v7, a5
+; GFX90A-NEXT: v_accvgpr_read_b32 v6, a4
+; GFX90A-NEXT: v_accvgpr_write_b32 a4, 4
+; GFX90A-NEXT: v_mov_b32_e32 v4, 0
+; GFX90A-NEXT: v_accvgpr_write_b32 a8, 5
+; GFX90A-NEXT: v_accvgpr_write_b32 a9, 1
+; GFX90A-NEXT: v_accvgpr_write_b32 a10, 2
+; GFX90A-NEXT: v_accvgpr_write_b32 a11, 3
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_accvgpr_write_b32 a4, v6
+; GFX90A-NEXT: v_accvgpr_write_b32 a5, v7
+; GFX90A-NEXT: v_accvgpr_write_b32 a6, v8
+; GFX90A-NEXT: v_accvgpr_write_b32 a7, v9
+; GFX90A-NEXT: v_mov_b32_e32 v1, v0
+; GFX90A-NEXT: v_mov_b32_e32 v2, v0
+; GFX90A-NEXT: v_mov_b32_e32 v3, v0
+; GFX90A-NEXT: global_store_dwordx4 v4, v[0:3], s[6:7]
+; GFX90A-NEXT: s_waitcnt vmcnt(0)
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: s_endpgm
+bb:
+ %in.1 = load <4 x float>, ptr addrspace(1) %arg
+ %mai.1 = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %in.1, i32 0, i32 0, i32 0)
+ %mai.2 = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %mai.1, i32 0, i32 0, i32 0)
+ %cmp = icmp eq i32 %cond, 0
+ br i1 %cmp, label %use, label %st
+
+use:
+ call void asm sideeffect "", "a,a,a,a,a"(i32 1, i32 2, i32 3, i32 4, i32 5)
+ store volatile <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, ptr addrspace(1) %out
+ br label %st
+
+st:
+ %gep1 = getelementptr <4 x float>, ptr addrspace(1) %out, i64 16
+ %gep2 = getelementptr <4 x float>, ptr addrspace(1) %out, i64 32
+ call void asm sideeffect "", "a,a"(<4 x float> %mai.1, <4 x float> %mai.2)
+ ret void
+}
+; GCN: ScratchSize: 0
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32, i32, i32)
+declare <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float, float, <4 x float>, i32, i32, i32)
+declare <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float, float, <32 x float>, i32, i32, i32)
+
+
+attributes #2 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+
+attributes #2 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; GCN: {{.*}}
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr32.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr32.ll
new file mode 100644
index 0000000000000..65141f897bd72
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr32.ll
@@ -0,0 +1,186 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=32 -amdgpu-stress-agpr=32 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.0a -amdgpu-mfma-vgpr-form=0 -amdgpu-stress-vgpr=32 -amdgpu-stress-agpr=32 < %s | FileCheck -check-prefixes=GCN,GFX90A %s
+
+define amdgpu_kernel void @max_32regs_mfma32(ptr addrspace(1) %arg) #3 {
+; GFX908-LABEL: max_32regs_mfma32:
+; GFX908: ; %bb.0: ; %bb
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x40400000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x40c00000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x40e00000
+; GFX908-NEXT: v_accvgpr_write_b32 a2, v1
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x40a00000
+; GFX908-NEXT: v_accvgpr_write_b32 a5, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a6, v3
+; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41000000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41100000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41200000
+; GFX908-NEXT: v_accvgpr_write_b32 a7, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a8, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a9, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41300000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41400000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41500000
+; GFX908-NEXT: v_accvgpr_write_b32 a10, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a11, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a12, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41600000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41700000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41800000
+; GFX908-NEXT: v_accvgpr_write_b32 a13, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a14, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a15, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41880000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41900000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41980000
+; GFX908-NEXT: v_accvgpr_write_b32 a16, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a17, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a18, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41a00000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41a80000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41b00000
+; GFX908-NEXT: v_accvgpr_write_b32 a19, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a20, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a21, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41b80000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41c00000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41c80000
+; GFX908-NEXT: v_accvgpr_write_b32 a22, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a23, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a24, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41d00000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41d80000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41e00000
+; GFX908-NEXT: v_accvgpr_write_b32 a25, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a26, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a27, v3
+; GFX908-NEXT: v_mov_b32_e32 v1, 0x41e80000
+; GFX908-NEXT: v_mov_b32_e32 v2, 0x41f00000
+; GFX908-NEXT: v_mov_b32_e32 v3, 0x41f80000
+; GFX908-NEXT: v_mov_b32_e32 v4, 1.0
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a0
+; GFX908-NEXT: v_accvgpr_write_b32 a0, 1.0
+; GFX908-NEXT: v_accvgpr_write_b32 a1, 2.0
+; GFX908-NEXT: v_accvgpr_write_b32 a3, 4.0
+; GFX908-NEXT: v_accvgpr_write_b32 a28, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a29, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a30, v3
+; GFX908-NEXT: v_accvgpr_write_b32 a31, 2.0
+; GFX908-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
+; GFX908-NEXT: v_mfma_f32_32x32x1f32 a[0:31], v4, v4, a[0:31]
+; GFX908-NEXT: v_mov_b32_e32 v0, 0
+; GFX908-NEXT: s_nop 13
+; GFX908-NEXT: v_accvgpr_write_b32 a1, v5
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: v_accvgpr_read_b32 v1, a0
+; GFX908-NEXT: s_waitcnt lgkmcnt(0)
+; GFX908-NEXT: s_nop 0
+; GFX908-NEXT: global_store_dword v0, v1, s[2:3]
+; GFX908-NEXT: s_endpgm
+;
+; GFX90A-LABEL: max_32regs_mfma32:
+; GFX90A: ; %bb.0: ; %bb
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40400000
+; GFX90A-NEXT: v_accvgpr_write_b32 a2, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40a00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a4, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40c00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a5, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40e00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a6, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41000000
+; GFX90A-NEXT: v_accvgpr_write_b32 a7, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41100000
+; GFX90A-NEXT: v_accvgpr_write_b32 a8, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41200000
+; GFX90A-NEXT: v_accvgpr_write_b32 a9, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41300000
+; GFX90A-NEXT: v_accvgpr_write_b32 a10, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41400000
+; GFX90A-NEXT: v_accvgpr_write_b32 a11, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41500000
+; GFX90A-NEXT: v_accvgpr_write_b32 a12, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41600000
+; GFX90A-NEXT: v_accvgpr_write_b32 a13, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41700000
+; GFX90A-NEXT: v_accvgpr_write_b32 a14, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41800000
+; GFX90A-NEXT: v_accvgpr_write_b32 a15, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41880000
+; GFX90A-NEXT: v_accvgpr_write_b32 a16, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41900000
+; GFX90A-NEXT: v_accvgpr_write_b32 a17, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41980000
+; GFX90A-NEXT: v_accvgpr_write_b32 a18, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41a00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a19, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41a80000
+; GFX90A-NEXT: v_accvgpr_write_b32 a20, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41b00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a21, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41b80000
+; GFX90A-NEXT: v_accvgpr_write_b32 a22, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41c00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a23, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41c80000
+; GFX90A-NEXT: v_accvgpr_write_b32 a24, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41d00000
+; GFX90A-NEXT: v_accvgpr_write_b32 a25, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41d80000
+; GFX90A-NEXT: v_accvgpr_write_b32 a26, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41e00000
+; GFX90A-NEXT: v_mov_b32_e32 v2, 1.0
+; GFX90A-NEXT: v_accvgpr_write_b32 a27, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41e80000
+; GFX90A-NEXT: v_accvgpr_write_b32 a28, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41f00000
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: v_accvgpr_write_b32 a1, 2.0
+; GFX90A-NEXT: v_accvgpr_write_b32 a29, v1
+; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41f80000
+; GFX90A-NEXT: v_accvgpr_read_b32 v3, a0
+; GFX90A-NEXT: v_accvgpr_write_b32 a0, 1.0
+; GFX90A-NEXT: v_accvgpr_write_b32 a3, 4.0
+; GFX90A-NEXT: v_accvgpr_write_b32 a30, v1
+; GFX90A-NEXT: v_accvgpr_mov_b32 a31, a1
+; GFX90A-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
+; GFX90A-NEXT: v_mov_b32_e32 v0, 0
+; GFX90A-NEXT: v_mfma_f32_32x32x1f32 a[0:31], v2, v2, a[0:31]
+; GFX90A-NEXT: s_nop 15
+; GFX90A-NEXT: s_nop 2
+; GFX90A-NEXT: v_accvgpr_write_b32 a1, v3
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90A-NEXT: global_store_dword v0, a0, s[2:3]
+; GFX90A-NEXT: s_endpgm
+bb:
+ %v = call i32 asm sideeffect "", "=a"()
+ br label %use
+
+use:
+ %mai.1 = tail call <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float 1.0, float 1.0, <32 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0, float 9.0, float 10.0, float 11.0, float 12.0, float 13.0, float 14.0, float 15.0, float 16.0, float 17.0, float 18.0, float 19.0, float 20.0, float 21.0, float 22.0, float 23.0, float 24.0, float 25.0, float 26.0, float 27.0, float 28.0, float 29.0, float 30.0, float 31.0, float 2.0>, i32 0, i32 0, i32 0)
+ call void asm sideeffect "", "a"(i32 %v)
+ %elt1 = extractelement <32 x float> %mai.1, i32 0
+ store float %elt1, ptr addrspace(1) %arg
+ ret void
+}
+; GCN: ScratchSize: 0
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32, i32, i32)
+declare <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float, float, <4 x float>, i32, i32, i32)
+declare <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float, float, <32 x float>, i32, i32, i32)
+
+
+attributes #3 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+
+attributes #3 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; GCN: {{.*}}
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr6.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr6.ll
new file mode 100644
index 0000000000000..f3a4687ba4520
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-agpr-vgpr6.ll
@@ -0,0 +1,130 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=6 -amdgpu-stress-agpr=6 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.0a -amdgpu-mfma-vgpr-form=0 -amdgpu-stress-vgpr=6 -amdgpu-stress-agpr=6 < %s | FileCheck -check-prefixes=GCN,GFX90A %s
+
+; Should spill agprs to memory for both gfx908 and gfx90a.
+define amdgpu_kernel void @max_6regs_used_8a(ptr addrspace(1) %arg) #4 {
+; GFX908-LABEL: max_6regs_used_8a:
+; GFX908: ; %bb.0:
+; GFX908-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ; def v1
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: v_lshlrev_b32_e32 v4, 4, v0
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ; def a[0:3]
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_mov_b32 s8, SCRATCH_RSRC_DWORD0
+; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
+; GFX908-NEXT: s_waitcnt lgkmcnt(0)
+; GFX908-NEXT: global_load_dwordx4 v[0:3], v4, s[2:3]
+; GFX908-NEXT: s_mov_b32 s9, SCRATCH_RSRC_DWORD1
+; GFX908-NEXT: s_mov_b32 s10, -1
+; GFX908-NEXT: s_mov_b32 s11, 0xe00000
+; GFX908-NEXT: s_add_u32 s8, s8, s5
+; GFX908-NEXT: s_addc_u32 s9, s9, 0
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a0 ; Reload Reuse
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 ; 4-byte Folded Spill
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a1 ; Reload Reuse
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:4 ; 4-byte Folded Spill
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a2 ; Reload Reuse
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:8 ; 4-byte Folded Spill
+; GFX908-NEXT: v_accvgpr_read_b32 v5, a3 ; Reload Reuse
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:12 ; 4-byte Folded Spill
+; GFX908-NEXT: s_waitcnt vmcnt(4)
+; GFX908-NEXT: v_accvgpr_write_b32 a0, v0
+; GFX908-NEXT: v_accvgpr_write_b32 a1, v1
+; GFX908-NEXT: v_accvgpr_write_b32 a2, v2
+; GFX908-NEXT: v_accvgpr_write_b32 a3, v3
+; GFX908-NEXT: v_mov_b32_e32 v0, 1.0
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
+; GFX908-NEXT: s_nop 3
+; GFX908-NEXT: v_accvgpr_read_b32 v0, a0
+; GFX908-NEXT: v_accvgpr_read_b32 v1, a1
+; GFX908-NEXT: v_accvgpr_read_b32 v2, a2
+; GFX908-NEXT: v_accvgpr_read_b32 v3, a3
+; GFX908-NEXT: s_nop 1
+; GFX908-NEXT: global_store_dwordx4 v4, v[0:3], s[2:3]
+; GFX908-NEXT: buffer_load_dword v0, off, s[8:11], 0 ; 4-byte Folded Reload
+; GFX908-NEXT: s_nop 0
+; GFX908-NEXT: buffer_load_dword v1, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
+; GFX908-NEXT: buffer_load_dword v2, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
+; GFX908-NEXT: buffer_load_dword v3, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
+; GFX908-NEXT: s_waitcnt vmcnt(0)
+; GFX908-NEXT: global_store_dwordx4 v[0:1], v[0:3], off
+; GFX908-NEXT: s_waitcnt vmcnt(0)
+; GFX908-NEXT: v_accvgpr_read_b32 v0, a4
+; GFX908-NEXT: ;;#ASMSTART
+; GFX908-NEXT: ; use v0
+; GFX908-NEXT: ;;#ASMEND
+; GFX908-NEXT: s_endpgm
+;
+; GFX90A-LABEL: max_6regs_used_8a:
+; GFX90A: ; %bb.0:
+; GFX90A-NEXT: s_mov_b32 s8, SCRATCH_RSRC_DWORD0
+; GFX90A-NEXT: s_mov_b32 s9, SCRATCH_RSRC_DWORD1
+; GFX90A-NEXT: s_mov_b32 s10, -1
+; GFX90A-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
+; GFX90A-NEXT: s_mov_b32 s11, 0xe00000
+; GFX90A-NEXT: s_add_u32 s8, s8, s5
+; GFX90A-NEXT: s_addc_u32 s9, s9, 0
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ; def v1
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ; def a[0:3]
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: buffer_store_dword a0, off, s[8:11], 0 ; 4-byte Folded Spill
+; GFX90A-NEXT: s_nop 0
+; GFX90A-NEXT: buffer_store_dword a1, off, s[8:11], 0 offset:4 ; 4-byte Folded Spill
+; GFX90A-NEXT: buffer_store_dword a2, off, s[8:11], 0 offset:8 ; 4-byte Folded Spill
+; GFX90A-NEXT: buffer_store_dword a3, off, s[8:11], 0 offset:12 ; 4-byte Folded Spill
+; GFX90A-NEXT: v_lshlrev_b32_e32 v0, 4, v0
+; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90A-NEXT: global_load_dwordx4 a[0:3], v0, s[2:3]
+; GFX90A-NEXT: v_mov_b32_e32 v2, 1.0
+; GFX90A-NEXT: s_waitcnt vmcnt(0)
+; GFX90A-NEXT: s_nop 0
+; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v2, v2, a[0:3]
+; GFX90A-NEXT: buffer_load_dword v2, off, s[8:11], 0 ; 4-byte Folded Reload
+; GFX90A-NEXT: buffer_load_dword v3, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
+; GFX90A-NEXT: buffer_load_dword v4, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
+; GFX90A-NEXT: buffer_load_dword v5, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
+; GFX90A-NEXT: s_nop 0
+; GFX90A-NEXT: global_store_dwordx4 v0, a[0:3], s[2:3]
+; GFX90A-NEXT: s_waitcnt vmcnt(1)
+; GFX90A-NEXT: global_store_dwordx4 v[0:1], v[2:5], off
+; GFX90A-NEXT: s_waitcnt vmcnt(0)
+; GFX90A-NEXT: ;;#ASMSTART
+; GFX90A-NEXT: ; use v1
+; GFX90A-NEXT: ;;#ASMEND
+; GFX90A-NEXT: s_endpgm
+ %tid = call i32 @llvm.amdgcn.workitem.id.x()
+ %v0 = call float asm sideeffect "; def $0", "=v"()
+ %a4 = call <4 x float> asm sideeffect "; def $0", "=a"()
+ %gep = getelementptr inbounds <4 x float>, ptr addrspace(1) %arg, i32 %tid
+ %mai.in = load <4 x float>, ptr addrspace(1) %gep
+ %mai.out = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %mai.in, i32 0, i32 0, i32 0)
+ store <4 x float> %mai.out, ptr addrspace(1) %gep
+ store volatile <4 x float> %a4, ptr addrspace(1) poison
+ call void asm sideeffect "; use $0", "v"(float %v0);
+ ret void
+}
+; GCN: ScratchSize: 20
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32, i32, i32)
+declare <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float, float, <4 x float>, i32, i32, i32)
+declare <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float, float, <32 x float>, i32, i32, i32)
+
+
+attributes #4 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+
+attributes #4 = { nounwind "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
+;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
+; GCN: {{.*}}
diff --git a/llvm/test/CodeGen/AMDGPU/spill-agpr.ll b/llvm/test/CodeGen/AMDGPU/spill-agpr.ll
deleted file mode 100644
index ea6216ac4ac40..0000000000000
--- a/llvm/test/CodeGen/AMDGPU/spill-agpr.ll
+++ /dev/null
@@ -1,493 +0,0 @@
-; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
-; RUN: llc -mtriple=amdgpu9.08 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
-; RUN: llc -mtriple=amdgpu9.0a -amdgpu-mfma-vgpr-form=0 < %s | FileCheck -check-prefixes=GCN,GFX90A %s
-
-define amdgpu_kernel void @max_12regs_13a_used(i32 %cond, ptr addrspace(1) %arg, ptr addrspace(1) %out) #2 {
-; GFX908-LABEL: max_12regs_13a_used:
-; GFX908: ; %bb.0: ; %bb
-; GFX908-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x2c
-; GFX908-NEXT: v_mov_b32_e32 v0, 1.0
-; GFX908-NEXT: s_load_dword s0, s[0:1], 0x24
-; GFX908-NEXT: s_waitcnt lgkmcnt(0)
-; GFX908-NEXT: s_cmp_lg_u32 s0, 0
-; GFX908-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
-; GFX908-NEXT: s_waitcnt lgkmcnt(0)
-; GFX908-NEXT: v_mov_b32_e32 v5, s8
-; GFX908-NEXT: v_mov_b32_e32 v1, s9
-; GFX908-NEXT: v_mov_b32_e32 v2, s10
-; GFX908-NEXT: v_accvgpr_write_b32 a0, v5
-; GFX908-NEXT: v_mov_b32_e32 v5, s11
-; GFX908-NEXT: v_accvgpr_write_b32 a1, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a2, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a3, v5
-; GFX908-NEXT: s_nop 0
-; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
-; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[4:7], v0, v0, a[0:3]
-; GFX908-NEXT: s_cbranch_scc0 .LBB0_2
-; GFX908-NEXT: ; %bb.1: ; %st
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_endpgm
-; GFX908-NEXT: .LBB0_2: ; %use
-; GFX908-NEXT: s_nop 2
-; GFX908-NEXT: v_accvgpr_read_b32 v1, a4
-; GFX908-NEXT: v_accvgpr_read_b32 v2, a5
-; GFX908-NEXT: v_accvgpr_read_b32 v3, a6
-; GFX908-NEXT: v_accvgpr_read_b32 v4, a7
-; GFX908-NEXT: v_accvgpr_write_b32 a4, 4
-; GFX908-NEXT: v_accvgpr_write_b32 a8, 5
-; GFX908-NEXT: v_accvgpr_write_b32 a9, 1
-; GFX908-NEXT: v_accvgpr_write_b32 a10, 2
-; GFX908-NEXT: v_accvgpr_write_b32 a11, 3
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_accvgpr_write_b32 a7, v4
-; GFX908-NEXT: v_accvgpr_write_b32 a6, v3
-; GFX908-NEXT: v_accvgpr_write_b32 a5, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
-; GFX908-NEXT: v_mov_b32_e32 v1, v0
-; GFX908-NEXT: v_mov_b32_e32 v2, v0
-; GFX908-NEXT: v_mov_b32_e32 v3, v0
-; GFX908-NEXT: v_mov_b32_e32 v4, 0
-; GFX908-NEXT: global_store_dwordx4 v4, v[0:3], s[6:7]
-; GFX908-NEXT: s_waitcnt vmcnt(0)
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_endpgm
-;
-; GFX90A-LABEL: max_12regs_13a_used:
-; GFX90A: ; %bb.0: ; %bb
-; GFX90A-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x2c
-; GFX90A-NEXT: v_mov_b32_e32 v0, 1.0
-; GFX90A-NEXT: s_load_dword s0, s[0:1], 0x24
-; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
-; GFX90A-NEXT: s_cmp_lg_u32 s0, 0
-; GFX90A-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
-; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
-; GFX90A-NEXT: v_accvgpr_write_b32 a0, s8
-; GFX90A-NEXT: v_accvgpr_write_b32 a1, s9
-; GFX90A-NEXT: v_accvgpr_write_b32 a2, s10
-; GFX90A-NEXT: v_accvgpr_write_b32 a3, s11
-; GFX90A-NEXT: s_nop 1
-; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
-; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[4:7], v0, v0, a[0:3]
-; GFX90A-NEXT: s_cbranch_scc0 .LBB0_2
-; GFX90A-NEXT: ; %bb.1: ; %st
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: s_endpgm
-; GFX90A-NEXT: .LBB0_2: ; %use
-; GFX90A-NEXT: s_nop 3
-; GFX90A-NEXT: v_accvgpr_read_b32 v9, a7
-; GFX90A-NEXT: v_accvgpr_read_b32 v8, a6
-; GFX90A-NEXT: v_accvgpr_read_b32 v7, a5
-; GFX90A-NEXT: v_accvgpr_read_b32 v6, a4
-; GFX90A-NEXT: v_accvgpr_write_b32 a4, 4
-; GFX90A-NEXT: v_mov_b32_e32 v4, 0
-; GFX90A-NEXT: v_accvgpr_write_b32 a8, 5
-; GFX90A-NEXT: v_accvgpr_write_b32 a9, 1
-; GFX90A-NEXT: v_accvgpr_write_b32 a10, 2
-; GFX90A-NEXT: v_accvgpr_write_b32 a11, 3
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: v_accvgpr_write_b32 a4, v6
-; GFX90A-NEXT: v_accvgpr_write_b32 a5, v7
-; GFX90A-NEXT: v_accvgpr_write_b32 a6, v8
-; GFX90A-NEXT: v_accvgpr_write_b32 a7, v9
-; GFX90A-NEXT: v_mov_b32_e32 v1, v0
-; GFX90A-NEXT: v_mov_b32_e32 v2, v0
-; GFX90A-NEXT: v_mov_b32_e32 v3, v0
-; GFX90A-NEXT: global_store_dwordx4 v4, v[0:3], s[6:7]
-; GFX90A-NEXT: s_waitcnt vmcnt(0)
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: s_endpgm
-bb:
- %in.1 = load <4 x float>, ptr addrspace(1) %arg
- %mai.1 = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %in.1, i32 0, i32 0, i32 0)
- %mai.2 = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %mai.1, i32 0, i32 0, i32 0)
- %cmp = icmp eq i32 %cond, 0
- br i1 %cmp, label %use, label %st
-
-use:
- call void asm sideeffect "", "a,a,a,a,a"(i32 1, i32 2, i32 3, i32 4, i32 5)
- store volatile <4 x float> <float 1.0, float 1.0, float 1.0, float 1.0>, ptr addrspace(1) %out
- br label %st
-
-st:
- %gep1 = getelementptr <4 x float>, ptr addrspace(1) %out, i64 16
- %gep2 = getelementptr <4 x float>, ptr addrspace(1) %out, i64 32
- call void asm sideeffect "", "a,a"(<4 x float> %mai.1, <4 x float> %mai.2)
- ret void
-}
-; GCN: ScratchSize: 0
-
-define amdgpu_kernel void @max_10_vgprs_used_9a() #1 {
-; GFX908-LABEL: max_10_vgprs_used_9a:
-; GFX908: ; %bb.0:
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_accvgpr_read_b32 v0, a0
-; GFX908-NEXT: v_accvgpr_read_b32 v1, a1
-; GFX908-NEXT: v_accvgpr_read_b32 v2, a2
-; GFX908-NEXT: v_accvgpr_read_b32 v3, a3
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a2
-; GFX908-NEXT: v_accvgpr_read_b32 v4, a1
-; GFX908-NEXT: v_accvgpr_write_b32 a1, v0
-; GFX908-NEXT: v_accvgpr_write_b32 a4, v3
-; GFX908-NEXT: v_accvgpr_write_b32 a3, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a2, v1
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_accvgpr_write_b32 a0, v4
-; GFX908-NEXT: v_accvgpr_write_b32 a1, v5
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_endpgm
-;
-; GFX90A-LABEL: max_10_vgprs_used_9a:
-; GFX90A: ; %bb.0:
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: v_accvgpr_read_b32 v0, a0
-; GFX90A-NEXT: v_accvgpr_read_b32 v1, a1
-; GFX90A-NEXT: v_accvgpr_read_b32 v2, a2
-; GFX90A-NEXT: v_accvgpr_read_b32 v3, a3
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: v_accvgpr_read_b32 v5, a3
-; GFX90A-NEXT: v_accvgpr_read_b32 v4, a2
-; GFX90A-NEXT: v_accvgpr_write_b32 a5, v3
-; GFX90A-NEXT: v_accvgpr_write_b32 a4, v2
-; GFX90A-NEXT: v_accvgpr_write_b32 a3, v1
-; GFX90A-NEXT: v_accvgpr_write_b32 a2, v0
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: v_accvgpr_write_b32 a0, v4
-; GFX90A-NEXT: v_accvgpr_write_b32 a1, v5
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: s_endpgm
- %a1 = call <4 x i32> asm sideeffect "", "=a"()
- %a2 = call <4 x i32> asm sideeffect "", "=a"()
- %a3 = call i32 asm sideeffect "", "=a"()
- %a4 = call <2 x i32> asm sideeffect "", "=a"()
- call void asm sideeffect "", "a,a,a"(<4 x i32> %a1, <4 x i32> %a2, i32 %a3)
- call void asm sideeffect "", "a"(<2 x i32> %a4)
- ret void
-}
-; GCN: ScratchSize: 0
-
-define amdgpu_kernel void @max_32regs_mfma32(ptr addrspace(1) %arg) #3 {
-; GFX908-LABEL: max_32regs_mfma32:
-; GFX908: ; %bb.0: ; %bb
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x40400000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x40c00000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x40e00000
-; GFX908-NEXT: v_accvgpr_write_b32 a2, v1
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x40a00000
-; GFX908-NEXT: v_accvgpr_write_b32 a5, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a6, v3
-; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41000000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41100000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41200000
-; GFX908-NEXT: v_accvgpr_write_b32 a7, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a8, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a9, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41300000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41400000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41500000
-; GFX908-NEXT: v_accvgpr_write_b32 a10, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a11, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a12, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41600000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41700000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41800000
-; GFX908-NEXT: v_accvgpr_write_b32 a13, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a14, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a15, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41880000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41900000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41980000
-; GFX908-NEXT: v_accvgpr_write_b32 a16, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a17, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a18, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41a00000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41a80000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41b00000
-; GFX908-NEXT: v_accvgpr_write_b32 a19, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a20, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a21, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41b80000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41c00000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41c80000
-; GFX908-NEXT: v_accvgpr_write_b32 a22, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a23, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a24, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41d00000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41d80000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41e00000
-; GFX908-NEXT: v_accvgpr_write_b32 a25, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a26, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a27, v3
-; GFX908-NEXT: v_mov_b32_e32 v1, 0x41e80000
-; GFX908-NEXT: v_mov_b32_e32 v2, 0x41f00000
-; GFX908-NEXT: v_mov_b32_e32 v3, 0x41f80000
-; GFX908-NEXT: v_mov_b32_e32 v4, 1.0
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a0
-; GFX908-NEXT: v_accvgpr_write_b32 a0, 1.0
-; GFX908-NEXT: v_accvgpr_write_b32 a1, 2.0
-; GFX908-NEXT: v_accvgpr_write_b32 a3, 4.0
-; GFX908-NEXT: v_accvgpr_write_b32 a28, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a29, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a30, v3
-; GFX908-NEXT: v_accvgpr_write_b32 a31, 2.0
-; GFX908-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
-; GFX908-NEXT: v_mfma_f32_32x32x1f32 a[0:31], v4, v4, a[0:31]
-; GFX908-NEXT: v_mov_b32_e32 v0, 0
-; GFX908-NEXT: s_nop 13
-; GFX908-NEXT: v_accvgpr_write_b32 a1, v5
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: v_accvgpr_read_b32 v1, a0
-; GFX908-NEXT: s_waitcnt lgkmcnt(0)
-; GFX908-NEXT: s_nop 0
-; GFX908-NEXT: global_store_dword v0, v1, s[2:3]
-; GFX908-NEXT: s_endpgm
-;
-; GFX90A-LABEL: max_32regs_mfma32:
-; GFX90A: ; %bb.0: ; %bb
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40400000
-; GFX90A-NEXT: v_accvgpr_write_b32 a2, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40a00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a4, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40c00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a5, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x40e00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a6, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41000000
-; GFX90A-NEXT: v_accvgpr_write_b32 a7, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41100000
-; GFX90A-NEXT: v_accvgpr_write_b32 a8, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41200000
-; GFX90A-NEXT: v_accvgpr_write_b32 a9, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41300000
-; GFX90A-NEXT: v_accvgpr_write_b32 a10, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41400000
-; GFX90A-NEXT: v_accvgpr_write_b32 a11, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41500000
-; GFX90A-NEXT: v_accvgpr_write_b32 a12, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41600000
-; GFX90A-NEXT: v_accvgpr_write_b32 a13, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41700000
-; GFX90A-NEXT: v_accvgpr_write_b32 a14, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41800000
-; GFX90A-NEXT: v_accvgpr_write_b32 a15, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41880000
-; GFX90A-NEXT: v_accvgpr_write_b32 a16, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41900000
-; GFX90A-NEXT: v_accvgpr_write_b32 a17, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41980000
-; GFX90A-NEXT: v_accvgpr_write_b32 a18, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41a00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a19, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41a80000
-; GFX90A-NEXT: v_accvgpr_write_b32 a20, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41b00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a21, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41b80000
-; GFX90A-NEXT: v_accvgpr_write_b32 a22, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41c00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a23, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41c80000
-; GFX90A-NEXT: v_accvgpr_write_b32 a24, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41d00000
-; GFX90A-NEXT: v_accvgpr_write_b32 a25, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41d80000
-; GFX90A-NEXT: v_accvgpr_write_b32 a26, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41e00000
-; GFX90A-NEXT: v_mov_b32_e32 v2, 1.0
-; GFX90A-NEXT: v_accvgpr_write_b32 a27, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41e80000
-; GFX90A-NEXT: v_accvgpr_write_b32 a28, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41f00000
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: v_accvgpr_write_b32 a1, 2.0
-; GFX90A-NEXT: v_accvgpr_write_b32 a29, v1
-; GFX90A-NEXT: v_mov_b32_e32 v1, 0x41f80000
-; GFX90A-NEXT: v_accvgpr_read_b32 v3, a0
-; GFX90A-NEXT: v_accvgpr_write_b32 a0, 1.0
-; GFX90A-NEXT: v_accvgpr_write_b32 a3, 4.0
-; GFX90A-NEXT: v_accvgpr_write_b32 a30, v1
-; GFX90A-NEXT: v_accvgpr_mov_b32 a31, a1
-; GFX90A-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
-; GFX90A-NEXT: v_mov_b32_e32 v0, 0
-; GFX90A-NEXT: v_mfma_f32_32x32x1f32 a[0:31], v2, v2, a[0:31]
-; GFX90A-NEXT: s_nop 15
-; GFX90A-NEXT: s_nop 2
-; GFX90A-NEXT: v_accvgpr_write_b32 a1, v3
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
-; GFX90A-NEXT: global_store_dword v0, a0, s[2:3]
-; GFX90A-NEXT: s_endpgm
-bb:
- %v = call i32 asm sideeffect "", "=a"()
- br label %use
-
-use:
- %mai.1 = tail call <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float 1.0, float 1.0, <32 x float> <float 1.0, float 2.0, float 3.0, float 4.0, float 5.0, float 6.0, float 7.0, float 8.0, float 9.0, float 10.0, float 11.0, float 12.0, float 13.0, float 14.0, float 15.0, float 16.0, float 17.0, float 18.0, float 19.0, float 20.0, float 21.0, float 22.0, float 23.0, float 24.0, float 25.0, float 26.0, float 27.0, float 28.0, float 29.0, float 30.0, float 31.0, float 2.0>, i32 0, i32 0, i32 0)
- call void asm sideeffect "", "a"(i32 %v)
- %elt1 = extractelement <32 x float> %mai.1, i32 0
- store float %elt1, ptr addrspace(1) %arg
- ret void
-}
-; GCN: ScratchSize: 0
-
-; Should spill agprs to memory for both gfx908 and gfx90a.
-define amdgpu_kernel void @max_6regs_used_8a(ptr addrspace(1) %arg) #4 {
-; GFX908-LABEL: max_6regs_used_8a:
-; GFX908: ; %bb.0:
-; GFX908-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ; def v1
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: v_lshlrev_b32_e32 v4, 4, v0
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ; def a[0:3]
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_mov_b32 s8, SCRATCH_RSRC_DWORD0
-; GFX908-NEXT: v_accvgpr_write_b32 a4, v1
-; GFX908-NEXT: s_waitcnt lgkmcnt(0)
-; GFX908-NEXT: global_load_dwordx4 v[0:3], v4, s[2:3]
-; GFX908-NEXT: s_mov_b32 s9, SCRATCH_RSRC_DWORD1
-; GFX908-NEXT: s_mov_b32 s10, -1
-; GFX908-NEXT: s_mov_b32 s11, 0xe00000
-; GFX908-NEXT: s_add_u32 s8, s8, s5
-; GFX908-NEXT: s_addc_u32 s9, s9, 0
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a0 ; Reload Reuse
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 ; 4-byte Folded Spill
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a1 ; Reload Reuse
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:4 ; 4-byte Folded Spill
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a2 ; Reload Reuse
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:8 ; 4-byte Folded Spill
-; GFX908-NEXT: v_accvgpr_read_b32 v5, a3 ; Reload Reuse
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: buffer_store_dword v5, off, s[8:11], 0 offset:12 ; 4-byte Folded Spill
-; GFX908-NEXT: s_waitcnt vmcnt(4)
-; GFX908-NEXT: v_accvgpr_write_b32 a0, v0
-; GFX908-NEXT: v_accvgpr_write_b32 a1, v1
-; GFX908-NEXT: v_accvgpr_write_b32 a2, v2
-; GFX908-NEXT: v_accvgpr_write_b32 a3, v3
-; GFX908-NEXT: v_mov_b32_e32 v0, 1.0
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v0, v0, a[0:3]
-; GFX908-NEXT: s_nop 3
-; GFX908-NEXT: v_accvgpr_read_b32 v0, a0
-; GFX908-NEXT: v_accvgpr_read_b32 v1, a1
-; GFX908-NEXT: v_accvgpr_read_b32 v2, a2
-; GFX908-NEXT: v_accvgpr_read_b32 v3, a3
-; GFX908-NEXT: s_nop 1
-; GFX908-NEXT: global_store_dwordx4 v4, v[0:3], s[2:3]
-; GFX908-NEXT: buffer_load_dword v0, off, s[8:11], 0 ; 4-byte Folded Reload
-; GFX908-NEXT: s_nop 0
-; GFX908-NEXT: buffer_load_dword v1, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
-; GFX908-NEXT: buffer_load_dword v2, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
-; GFX908-NEXT: buffer_load_dword v3, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
-; GFX908-NEXT: s_waitcnt vmcnt(0)
-; GFX908-NEXT: global_store_dwordx4 v[0:1], v[0:3], off
-; GFX908-NEXT: s_waitcnt vmcnt(0)
-; GFX908-NEXT: v_accvgpr_read_b32 v0, a4
-; GFX908-NEXT: ;;#ASMSTART
-; GFX908-NEXT: ; use v0
-; GFX908-NEXT: ;;#ASMEND
-; GFX908-NEXT: s_endpgm
-;
-; GFX90A-LABEL: max_6regs_used_8a:
-; GFX90A: ; %bb.0:
-; GFX90A-NEXT: s_mov_b32 s8, SCRATCH_RSRC_DWORD0
-; GFX90A-NEXT: s_mov_b32 s9, SCRATCH_RSRC_DWORD1
-; GFX90A-NEXT: s_mov_b32 s10, -1
-; GFX90A-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x24
-; GFX90A-NEXT: s_mov_b32 s11, 0xe00000
-; GFX90A-NEXT: s_add_u32 s8, s8, s5
-; GFX90A-NEXT: s_addc_u32 s9, s9, 0
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ; def v1
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ; def a[0:3]
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: buffer_store_dword a0, off, s[8:11], 0 ; 4-byte Folded Spill
-; GFX90A-NEXT: s_nop 0
-; GFX90A-NEXT: buffer_store_dword a1, off, s[8:11], 0 offset:4 ; 4-byte Folded Spill
-; GFX90A-NEXT: buffer_store_dword a2, off, s[8:11], 0 offset:8 ; 4-byte Folded Spill
-; GFX90A-NEXT: buffer_store_dword a3, off, s[8:11], 0 offset:12 ; 4-byte Folded Spill
-; GFX90A-NEXT: v_lshlrev_b32_e32 v0, 4, v0
-; GFX90A-NEXT: s_waitcnt lgkmcnt(0)
-; GFX90A-NEXT: global_load_dwordx4 a[0:3], v0, s[2:3]
-; GFX90A-NEXT: v_mov_b32_e32 v2, 1.0
-; GFX90A-NEXT: s_waitcnt vmcnt(0)
-; GFX90A-NEXT: s_nop 0
-; GFX90A-NEXT: v_mfma_f32_4x4x1f32 a[0:3], v2, v2, a[0:3]
-; GFX90A-NEXT: buffer_load_dword v2, off, s[8:11], 0 ; 4-byte Folded Reload
-; GFX90A-NEXT: buffer_load_dword v3, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
-; GFX90A-NEXT: buffer_load_dword v4, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
-; GFX90A-NEXT: buffer_load_dword v5, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
-; GFX90A-NEXT: s_nop 0
-; GFX90A-NEXT: global_store_dwordx4 v0, a[0:3], s[2:3]
-; GFX90A-NEXT: s_waitcnt vmcnt(1)
-; GFX90A-NEXT: global_store_dwordx4 v[0:1], v[2:5], off
-; GFX90A-NEXT: s_waitcnt vmcnt(0)
-; GFX90A-NEXT: ;;#ASMSTART
-; GFX90A-NEXT: ; use v1
-; GFX90A-NEXT: ;;#ASMEND
-; GFX90A-NEXT: s_endpgm
- %tid = call i32 @llvm.amdgcn.workitem.id.x()
- %v0 = call float asm sideeffect "; def $0", "=v"()
- %a4 = call <4 x float> asm sideeffect "; def $0", "=a"()
- %gep = getelementptr inbounds <4 x float>, ptr addrspace(1) %arg, i32 %tid
- %mai.in = load <4 x float>, ptr addrspace(1) %gep
- %mai.out = tail call <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float 1.0, float 1.0, <4 x float> %mai.in, i32 0, i32 0, i32 0)
- store <4 x float> %mai.out, ptr addrspace(1) %gep
- store volatile <4 x float> %a4, ptr addrspace(1) poison
- call void asm sideeffect "; use $0", "v"(float %v0);
- ret void
-}
-; GCN: ScratchSize: 20
-
-declare i32 @llvm.amdgcn.workitem.id.x()
-declare <16 x float> @llvm.amdgcn.mfma.f32.16x16x1f32(float, float, <16 x float>, i32, i32, i32)
-declare <4 x float> @llvm.amdgcn.mfma.f32.4x4x1f32(float, float, <4 x float>, i32, i32, i32)
-declare <32 x float> @llvm.amdgcn.mfma.f32.32x32x1f32(float, float, <32 x float>, i32, i32, i32)
-
-
-attributes #1 = { nounwind "amdgpu-num-vgpr"="10" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #2 = { nounwind "amdgpu-num-vgpr"="12" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #3 = { nounwind "amdgpu-num-vgpr"="32" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #4 = { nounwind "amdgpu-num-vgpr"="6" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-cluster-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-cluster-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-cluster-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-
-attributes #1 = { nounwind "amdgpu-num-vgpr"="10" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #2 = { nounwind "amdgpu-num-vgpr"="12" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #3 = { nounwind "amdgpu-num-vgpr"="32" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-attributes #4 = { nounwind "amdgpu-num-vgpr"="6" "amdgpu-no-dispatch-id" "amdgpu-no-dispatch-ptr" "amdgpu-no-implicitarg-ptr" "amdgpu-no-lds-kernel-id" "amdgpu-no-queue-ptr" "amdgpu-no-workgroup-id-x" "amdgpu-no-workgroup-id-y" "amdgpu-no-workgroup-id-z" "amdgpu-no-workitem-id-y" "amdgpu-no-workitem-id-z" }
-;; NOTE: These prefixes are unused and the list is autogenerated. Do not add tests below this line:
-; GCN: {{.*}}
diff --git a/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr16.ll b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr16.ll
new file mode 100644
index 0000000000000..85ca59b8fdb4d
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr16.ll
@@ -0,0 +1,90 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -post-RA-scheduler=0 -stress-regalloc=8 -amdgpu-stress-vgpr=8 -amdgpu-stress-sgpr=16 < %s | FileCheck -check-prefix=MUBUF %s
+; RUN: llc -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -post-RA-scheduler=0 -stress-regalloc=8 -amdgpu-stress-vgpr=8 -amdgpu-stress-sgpr=16 -mattr=+enable-flat-scratch < %s | FileCheck -check-prefix=FLATSCR %s
+
+; Test that the VGPR spiller correctly switches to SGPR offsets when the
+; instruction offset field would overflow, and that it accounts for memory
+; swizzling.
+
+define void @test_sgpr_offset_function_scavenge_fail_func() #0 {
+; MUBUF-LABEL: test_sgpr_offset_function_scavenge_fail_func:
+; MUBUF: ; %bb.0: ; %entry
+; MUBUF-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 glc
+; MUBUF-NEXT: s_waitcnt vmcnt(0)
+; MUBUF-NEXT: s_add_i32 s10, s32, 0x40100
+; MUBUF-NEXT: buffer_store_dword v0, off, s[0:3], s10 ; 4-byte Folded Spill
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: s_add_i32 s10, s32, 0x40100
+; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s10 ; 4-byte Folded Reload
+; MUBUF-NEXT: s_waitcnt vmcnt(0)
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: s_setpc_b64 s[30:31]
+;
+; FLATSCR-LABEL: test_sgpr_offset_function_scavenge_fail_func:
+; FLATSCR: ; %bb.0: ; %entry
+; FLATSCR-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: scratch_load_dword v0, off, s32 offset:8 glc
+; FLATSCR-NEXT: s_waitcnt vmcnt(0)
+; FLATSCR-NEXT: s_add_i32 s8, s32, 0x1004
+; FLATSCR-NEXT: scratch_store_dword off, v0, s8 ; 4-byte Folded Spill
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: s_add_i32 s8, s32, 0x1004
+; FLATSCR-NEXT: scratch_load_dword v0, off, s8 ; 4-byte Folded Reload
+; FLATSCR-NEXT: s_waitcnt vmcnt(0)
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: s_setpc_b64 s[30:31]
+entry:
+ ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
+ ; fit in the instruction, and has to live in the SGPR offset.
+ %alloca = alloca i8, i32 4096, align 4, addrspace(5)
+
+ %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
+
+ %asm.0 = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
+ %asm0.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 0
+ %asm1.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 1
+ %asm2.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 2
+ %asm3.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 3
+ %asm4.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 4
+ %asm5.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 5
+ %asm6.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 6
+ %asm7.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 7
+
+ ; 0x40000 / 64 = 4096 (for wave64)
+ %a = load volatile i32, ptr addrspace(5) %aptr
+ call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0.0, i32 %asm1.0, i32 %asm2.0, i32 %asm3.0, i32 %asm4.0, i32 %asm5.0, i32 %asm6.0, i32 %asm7.0, i32 %a)
+
+ %asm = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
+ %asm0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 0
+ %asm1 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 1
+ %asm2 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 2
+ %asm3 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 3
+ %asm4 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 4
+ %asm5 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 5
+ %asm6 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 6
+ %asm7 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 7
+
+ call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}"() #0
+ ; Force %a to spill with no free SGPRs
+ call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0, i32 %asm1, i32 %asm2, i32 %asm3, i32 %asm4, i32 %asm5, i32 %asm6, i32 %asm7, i32 %a)
+ ret void
+}
+
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr18.ll b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr18.ll
new file mode 100644
index 0000000000000..2ac411d65d2b9
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation-sgpr18.ll
@@ -0,0 +1,91 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
+; RUN: llc -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -post-RA-scheduler=0 -stress-regalloc=8 -amdgpu-stress-vgpr=8 -amdgpu-stress-sgpr=18 < %s | FileCheck -check-prefix=MUBUF %s
+; RUN: llc -mtriple=amdgpu9.00-amd-amdhsa -enable-misched=0 -post-RA-scheduler=0 -stress-regalloc=8 -amdgpu-stress-vgpr=8 -amdgpu-stress-sgpr=18 -mattr=+enable-flat-scratch < %s | FileCheck -check-prefix=FLATSCR %s
+
+; Test that the VGPR spiller correctly switches to SGPR offsets when the
+; instruction offset field would overflow, and that it accounts for memory
+; swizzling.
+
+define amdgpu_kernel void @test_sgpr_offset_function_scavenge_fail_kernel() #0 {
+; MUBUF-LABEL: test_sgpr_offset_function_scavenge_fail_kernel:
+; MUBUF: ; %bb.0: ; %entry
+; MUBUF-NEXT: s_add_u32 s0, s0, s17
+; MUBUF-NEXT: s_addc_u32 s1, s1, 0
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
+; MUBUF-NEXT: s_waitcnt vmcnt(0)
+; MUBUF-NEXT: s_mov_b32 s10, 0x40100
+; MUBUF-NEXT: buffer_store_dword v0, off, s[0:3], s10 ; 4-byte Folded Spill
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s10 ; 4-byte Folded Reload
+; MUBUF-NEXT: s_waitcnt vmcnt(0)
+; MUBUF-NEXT: ;;#ASMSTART
+; MUBUF-NEXT: ;;#ASMEND
+; MUBUF-NEXT: s_endpgm
+;
+; FLATSCR-LABEL: test_sgpr_offset_function_scavenge_fail_kernel:
+; FLATSCR: ; %bb.0: ; %entry
+; FLATSCR-NEXT: s_add_u32 flat_scratch_lo, s8, s13
+; FLATSCR-NEXT: s_addc_u32 flat_scratch_hi, s9, 0
+; FLATSCR-NEXT: s_mov_b32 s8, 0
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: scratch_load_dword v0, off, s8 offset:8 glc
+; FLATSCR-NEXT: s_waitcnt vmcnt(0)
+; FLATSCR-NEXT: s_movk_i32 s8, 0x1004
+; FLATSCR-NEXT: scratch_store_dword off, v0, s8 ; 4-byte Folded Spill
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: scratch_load_dword v0, off, s8 ; 4-byte Folded Reload
+; FLATSCR-NEXT: s_waitcnt vmcnt(0)
+; FLATSCR-NEXT: ;;#ASMSTART
+; FLATSCR-NEXT: ;;#ASMEND
+; FLATSCR-NEXT: s_endpgm
+entry:
+ ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
+ ; fit in the instruction, and has to live in the SGPR offset.
+ %alloca = alloca i8, i32 4096, align 4, addrspace(5)
+
+ %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
+
+ %asm.0 = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
+ %asm0.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 0
+ %asm1.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 1
+ %asm2.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 2
+ %asm3.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 3
+ %asm4.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 4
+ %asm5.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 5
+ %asm6.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 6
+ %asm7.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 7
+
+ ; 0x40000 / 64 = 4096 (for wave64)
+ %a = load volatile i32, ptr addrspace(5) %aptr
+ call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0.0, i32 %asm1.0, i32 %asm2.0, i32 %asm3.0, i32 %asm4.0, i32 %asm5.0, i32 %asm6.0, i32 %asm7.0, i32 %a)
+
+ %asm = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
+ %asm0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 0
+ %asm1 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 1
+ %asm2 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 2
+ %asm3 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 3
+ %asm4 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 4
+ %asm5 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 5
+ %asm6 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 6
+ %asm7 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 7
+
+ call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}"() #0
+ ; Force %a to spill with no free SGPRs
+ call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0, i32 %asm1, i32 %asm2, i32 %asm3, i32 %asm4, i32 %asm5, i32 %asm6, i32 %asm7, i32 %a)
+ ret void
+}
+
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-offset-calculation.ll b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation.ll
index 968092c7922be..34c07daa2a0a2 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-offset-calculation.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-offset-calculation.ll
@@ -109,169 +109,6 @@ entry:
ret void
}
-define void @test_sgpr_offset_function_scavenge_fail_func() #2 {
-; MUBUF-LABEL: test_sgpr_offset_function_scavenge_fail_func:
-; MUBUF: ; %bb.0: ; %entry
-; MUBUF-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 glc
-; MUBUF-NEXT: s_waitcnt vmcnt(0)
-; MUBUF-NEXT: s_add_i32 s10, s32, 0x40100
-; MUBUF-NEXT: buffer_store_dword v0, off, s[0:3], s10 ; 4-byte Folded Spill
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: s_add_i32 s10, s32, 0x40100
-; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s10 ; 4-byte Folded Reload
-; MUBUF-NEXT: s_waitcnt vmcnt(0)
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: s_setpc_b64 s[30:31]
-;
-; FLATSCR-LABEL: test_sgpr_offset_function_scavenge_fail_func:
-; FLATSCR: ; %bb.0: ; %entry
-; FLATSCR-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: scratch_load_dword v0, off, s32 offset:8 glc
-; FLATSCR-NEXT: s_waitcnt vmcnt(0)
-; FLATSCR-NEXT: s_add_i32 s8, s32, 0x1004
-; FLATSCR-NEXT: scratch_store_dword off, v0, s8 ; 4-byte Folded Spill
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: s_add_i32 s8, s32, 0x1004
-; FLATSCR-NEXT: scratch_load_dword v0, off, s8 ; 4-byte Folded Reload
-; FLATSCR-NEXT: s_waitcnt vmcnt(0)
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: s_setpc_b64 s[30:31]
-entry:
- ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
- ; fit in the instruction, and has to live in the SGPR offset.
- %alloca = alloca i8, i32 4096, align 4, addrspace(5)
-
- %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
-
- %asm.0 = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
- %asm0.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 0
- %asm1.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 1
- %asm2.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 2
- %asm3.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 3
- %asm4.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 4
- %asm5.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 5
- %asm6.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 6
- %asm7.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 7
-
- ; 0x40000 / 64 = 4096 (for wave64)
- %a = load volatile i32, ptr addrspace(5) %aptr
- call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0.0, i32 %asm1.0, i32 %asm2.0, i32 %asm3.0, i32 %asm4.0, i32 %asm5.0, i32 %asm6.0, i32 %asm7.0, i32 %a)
-
- %asm = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
- %asm0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 0
- %asm1 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 1
- %asm2 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 2
- %asm3 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 3
- %asm4 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 4
- %asm5 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 5
- %asm6 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 6
- %asm7 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 7
-
- call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}"() #0
- ; Force %a to spill with no free SGPRs
- call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0, i32 %asm1, i32 %asm2, i32 %asm3, i32 %asm4, i32 %asm5, i32 %asm6, i32 %asm7, i32 %a)
- ret void
-}
-
-define amdgpu_kernel void @test_sgpr_offset_function_scavenge_fail_kernel() #3 {
-; MUBUF-LABEL: test_sgpr_offset_function_scavenge_fail_kernel:
-; MUBUF: ; %bb.0: ; %entry
-; MUBUF-NEXT: s_add_u32 s0, s0, s17
-; MUBUF-NEXT: s_addc_u32 s1, s1, 0
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], 0 offset:8 glc
-; MUBUF-NEXT: s_waitcnt vmcnt(0)
-; MUBUF-NEXT: s_mov_b32 s10, 0x40100
-; MUBUF-NEXT: buffer_store_dword v0, off, s[0:3], s10 ; 4-byte Folded Spill
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: buffer_load_dword v0, off, s[0:3], s10 ; 4-byte Folded Reload
-; MUBUF-NEXT: s_waitcnt vmcnt(0)
-; MUBUF-NEXT: ;;#ASMSTART
-; MUBUF-NEXT: ;;#ASMEND
-; MUBUF-NEXT: s_endpgm
-;
-; FLATSCR-LABEL: test_sgpr_offset_function_scavenge_fail_kernel:
-; FLATSCR: ; %bb.0: ; %entry
-; FLATSCR-NEXT: s_add_u32 flat_scratch_lo, s8, s13
-; FLATSCR-NEXT: s_addc_u32 flat_scratch_hi, s9, 0
-; FLATSCR-NEXT: s_mov_b32 s8, 0
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: scratch_load_dword v0, off, s8 offset:8 glc
-; FLATSCR-NEXT: s_waitcnt vmcnt(0)
-; FLATSCR-NEXT: s_movk_i32 s8, 0x1004
-; FLATSCR-NEXT: scratch_store_dword off, v0, s8 ; 4-byte Folded Spill
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: scratch_load_dword v0, off, s8 ; 4-byte Folded Reload
-; FLATSCR-NEXT: s_waitcnt vmcnt(0)
-; FLATSCR-NEXT: ;;#ASMSTART
-; FLATSCR-NEXT: ;;#ASMEND
-; FLATSCR-NEXT: s_endpgm
-entry:
- ; Occupy 4096 bytes of scratch, so the offset of the spill of %a does not
- ; fit in the instruction, and has to live in the SGPR offset.
- %alloca = alloca i8, i32 4096, align 4, addrspace(5)
-
- %aptr = getelementptr i32, ptr addrspace(5) %alloca, i32 1
-
- %asm.0 = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
- %asm0.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 0
- %asm1.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 1
- %asm2.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 2
- %asm3.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 3
- %asm4.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 4
- %asm5.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 5
- %asm6.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 6
- %asm7.0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm.0, 7
-
- ; 0x40000 / 64 = 4096 (for wave64)
- %a = load volatile i32, ptr addrspace(5) %aptr
- call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0.0, i32 %asm1.0, i32 %asm2.0, i32 %asm3.0, i32 %asm4.0, i32 %asm5.0, i32 %asm6.0, i32 %asm7.0, i32 %a)
-
- %asm = call { i32, i32, i32, i32, i32, i32, i32, i32 } asm sideeffect "", "=s,=s,=s,=s,=s,=s,=s,=s"()
- %asm0 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 0
- %asm1 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 1
- %asm2 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 2
- %asm3 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 3
- %asm4 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 4
- %asm5 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 5
- %asm6 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 6
- %asm7 = extractvalue { i32, i32, i32, i32, i32, i32, i32, i32 } %asm, 7
-
- call void asm sideeffect "", "~{v0},~{v1},~{v2},~{v3},~{v4},~{v5},~{v6},~{v7}"() #0
- ; Force %a to spill with no free SGPRs
- call void asm sideeffect "", "s,s,s,s,s,s,s,s,v"(i32 %asm0, i32 %asm1, i32 %asm2, i32 %asm3, i32 %asm4, i32 %asm5, i32 %asm6, i32 %asm7, i32 %a)
- ret void
-}
-
define amdgpu_kernel void @test_sgpr_offset_subregs_kernel() {
; MUBUF-LABEL: test_sgpr_offset_subregs_kernel:
; MUBUF: ; %bb.0: ; %entry
@@ -651,6 +488,3 @@ entry:
}
attributes #0 = { nounwind }
-attributes #1 = { nounwind "amdgpu-num-sgpr"="17" "amdgpu-num-vgpr"="8" }
-attributes #2 = { nounwind "amdgpu-num-sgpr"="16" "amdgpu-num-vgpr"="8" }
-attributes #3 = { nounwind "amdgpu-num-sgpr"="18" "amdgpu-num-vgpr"="8" }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-sgpr-stack-no-sgpr.ll b/llvm/test/CodeGen/AMDGPU/spill-sgpr-stack-no-sgpr.ll
index de8212919ed1d..37cd82287480e 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-sgpr-stack-no-sgpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-sgpr-stack-no-sgpr.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgpu10.10 -mattr=+wavefrontsize64 < %s | FileCheck -check-prefix=GFX10 %s
+; RUN: llc -mtriple=amdgpu10.10 -mattr=+wavefrontsize64 -amdgpu-stress-vgpr=8 -amdgpu-stress-sgpr=16 < %s | FileCheck -check-prefix=GFX10 %s
; The test was originally written to spill an SGPR to scratch without
; having spare SGPRs available to save exec. This scenario won't be
@@ -41,4 +41,4 @@ define amdgpu_kernel void @test() #1 {
; number of incoming arguments is larger than the number of permitted
; registers.
attributes #0 = { nounwind }
-attributes #1 = { nounwind "amdgpu-num-sgpr"="16" "amdgpu-num-vgpr"="8" "amdgpu-no-queue-ptr" "amdgpu-no-dispatch-id" }
+attributes #1 = { nounwind "amdgpu-no-queue-ptr" "amdgpu-no-dispatch-id" }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll b/llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll
index 29c5092609a82..d488943237bb8 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-vector-superclass.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py
-; RUN: llc -mtriple=amdgpu9.08-amd-amdhsa -stop-after=greedy,1 -o - %s | FileCheck -check-prefix=GCN %s
+; RUN: llc -mtriple=amdgpu9.08-amd-amdhsa -stop-after=greedy,1 -o - -amdgpu-stress-vgpr=5 -amdgpu-stress-agpr=5 %s | FileCheck -check-prefix=GCN %s
; Convert AV spills into VGPR spills by introducing appropriate copies in between.
define amdgpu_kernel void @test_spill_av_class(<4 x i32> %arg) #0 {
@@ -27,4 +27,4 @@ define amdgpu_kernel void @test_spill_av_class(<4 x i32> %arg) #0 {
declare <4 x i32> @llvm.amdgcn.mfma.i32.4x4x4i8(i32, i32, <4 x i32>, i32, i32, i32)
-attributes #0 = { nounwind "amdgpu-num-vgpr"="5" }
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress10.ll b/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress10.ll
new file mode 100644
index 0000000000000..6457f7f5ce3cf
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress10.ll
@@ -0,0 +1,27 @@
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=10 -amdgpu-stress-agpr=10 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.00 -amdgpu-stress-vgpr=10 < %s | FileCheck -check-prefixes=GCN,GFX900 %s
+
+; GCN-LABEL: {{^}}max_10_vgprs_spill_v32:
+; GCN-DAG: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD0
+; GCN-DAG: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD1
+; GCN: buffer_store_dword v{{[0-9]}},
+; GFX908-DAG: v_accvgpr_write_b32 a0, v{{[0-9]}}
+; GFX908-DAG: v_accvgpr_write_b32 a9, v{{[0-9]}}
+; GCN-NOT: a10
+
+; GFX908: NumVgprs: 10
+; GFX900: ScratchSize: 100
+; GFX908: ScratchSize: 68
+; GFX908: VGPRBlocks: 2
+; GFX908: NumVGPRsForWavesPerEU: 10
+define amdgpu_kernel void @max_10_vgprs_spill_v32(ptr addrspace(1) %p) #0 {
+ %tid = call i32 @llvm.amdgcn.workitem.id.x()
+ %gep = getelementptr inbounds <32 x float>, ptr addrspace(1) %p, i32 %tid
+ %v = load volatile <32 x float>, ptr addrspace(1) %gep
+ store volatile <32 x float> %v, ptr addrspace(1) poison
+ ret void
+}
+
+declare i32 @llvm.amdgcn.workitem.id.x()
+
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress11.ll b/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress11.ll
new file mode 100644
index 0000000000000..ea21c3d3ad45a
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/spill-vgpr-stress11.ll
@@ -0,0 +1,56 @@
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=11 -amdgpu-stress-agpr=11 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
+; RUN: llc -mtriple=amdgpu9.00 -amdgpu-stress-vgpr=11 < %s | FileCheck -check-prefixes=GCN,GFX900 %s
+
+; GCN-LABEL: {{^}}max_11_vgprs:
+; GFX900-NOT: SCRATCH_RSRC
+; GFX908-NOT: SCRATCH_RSRC
+; GFX908-DAG: v_accvgpr_write_b32 [[A_REG:a[0-9]+]], v{{[0-9]}}
+; GFX900-NOT: buffer_
+; GFX908-NOT: buffer_
+; GFX908-DAG: v_mov_b32_e32 v{{[0-9]}}, [[V_REG:v[0-9]+]]
+; GFX908-DAG: v_accvgpr_read_b32 [[V_REG]], [[A_REG]]
+
+; GFX900: NumVgprs: 11
+; GFX908: NumVgprs: 10
+; GFX900: ScratchSize: 0
+; GFX908: ScratchSize: 0
+; GCN: VGPRBlocks: 2
+; GFX900: NumVGPRsForWavesPerEU: 11
+; GFX908: NumVGPRsForWavesPerEU: 10
+define amdgpu_kernel void @max_11_vgprs(ptr addrspace(1) %p) #0 {
+ %tid = load volatile i32, ptr addrspace(1) poison
+ %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
+ %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
+ %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
+ %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
+ %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
+ %p6 = getelementptr inbounds i32, ptr addrspace(1) %p5, i32 20
+ %p7 = getelementptr inbounds i32, ptr addrspace(1) %p6, i32 24
+ %p8 = getelementptr inbounds i32, ptr addrspace(1) %p7, i32 28
+ %p9 = getelementptr inbounds i32, ptr addrspace(1) %p8, i32 32
+ %p10 = getelementptr inbounds i32, ptr addrspace(1) %p9, i32 36
+ %v1 = load volatile i32, ptr addrspace(1) %p1
+ %v2 = load volatile i32, ptr addrspace(1) %p2
+ %v3 = load volatile i32, ptr addrspace(1) %p3
+ %v4 = load volatile i32, ptr addrspace(1) %p4
+ %v5 = load volatile i32, ptr addrspace(1) %p5
+ %v6 = load volatile i32, ptr addrspace(1) %p6
+ %v7 = load volatile i32, ptr addrspace(1) %p7
+ %v8 = load volatile i32, ptr addrspace(1) %p8
+ %v9 = load volatile i32, ptr addrspace(1) %p9
+ %v10 = load volatile i32, ptr addrspace(1) %p10
+ call void asm sideeffect "", "v,v,v,v,v,v,v,v,v,v"(i32 %v1, i32 %v2, i32 %v3, i32 %v4, i32 %v5, i32 %v6, i32 %v7, i32 %v8, i32 %v9, i32 %v10)
+ store volatile i32 %v1, ptr addrspace(1) poison
+ store volatile i32 %v2, ptr addrspace(1) poison
+ store volatile i32 %v3, ptr addrspace(1) poison
+ store volatile i32 %v4, ptr addrspace(1) poison
+ store volatile i32 %v5, ptr addrspace(1) poison
+ store volatile i32 %v6, ptr addrspace(1) poison
+ store volatile i32 %v7, ptr addrspace(1) poison
+ store volatile i32 %v8, ptr addrspace(1) poison
+ store volatile i32 %v9, ptr addrspace(1) poison
+ store volatile i32 %v10, ptr addrspace(1) poison
+ ret void
+}
+
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr.ll b/llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr.ll
index d528217f3724c..66e3903bc1f1f 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-vgpr-to-agpr.ll
@@ -1,4 +1,4 @@
-; RUN: llc -mtriple=amdgpu9.08 < %s | FileCheck -check-prefixes=GFX908 %s
+; RUN: llc -mtriple=amdgpu9.08 -amdgpu-stress-vgpr=11 -amdgpu-stress-agpr=11 < %s | FileCheck -check-prefixes=GFX908 %s
; GFX908-LABEL: {{^}}max_11_vgprs_used_9a:
; GFX908-NOT: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD0
@@ -105,4 +105,4 @@ define amdgpu_kernel void @max_11_vgprs_used_1a_partial_spill(ptr addrspace(1) %
ret void
}
-attributes #0 = { nounwind "amdgpu-num-vgpr"="11" }
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/spill-vgpr.ll b/llvm/test/CodeGen/AMDGPU/spill-vgpr.ll
index 05dba10e9d9ab..68a0f0db2aac8 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-vgpr.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-vgpr.ll
@@ -1,78 +1,6 @@
; RUN: llc -mtriple=amdgpu9.08 < %s | FileCheck -check-prefixes=GCN,GFX908 %s
; RUN: llc -mtriple=amdgpu9.00 < %s | FileCheck -check-prefixes=GCN,GFX900 %s
-; GCN-LABEL: {{^}}max_11_vgprs:
-; GFX900-NOT: SCRATCH_RSRC
-; GFX908-NOT: SCRATCH_RSRC
-; GFX908-DAG: v_accvgpr_write_b32 [[A_REG:a[0-9]+]], v{{[0-9]}}
-; GFX900-NOT: buffer_
-; GFX908-NOT: buffer_
-; GFX908-DAG: v_mov_b32_e32 v{{[0-9]}}, [[V_REG:v[0-9]+]]
-; GFX908-DAG: v_accvgpr_read_b32 [[V_REG]], [[A_REG]]
-
-; GFX900: NumVgprs: 11
-; GFX908: NumVgprs: 10
-; GFX900: ScratchSize: 0
-; GFX908: ScratchSize: 0
-; GCN: VGPRBlocks: 2
-; GFX900: NumVGPRsForWavesPerEU: 11
-; GFX908: NumVGPRsForWavesPerEU: 10
-define amdgpu_kernel void @max_11_vgprs(ptr addrspace(1) %p) #2 {
- %tid = load volatile i32, ptr addrspace(1) poison
- %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
- %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
- %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
- %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
- %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
- %p6 = getelementptr inbounds i32, ptr addrspace(1) %p5, i32 20
- %p7 = getelementptr inbounds i32, ptr addrspace(1) %p6, i32 24
- %p8 = getelementptr inbounds i32, ptr addrspace(1) %p7, i32 28
- %p9 = getelementptr inbounds i32, ptr addrspace(1) %p8, i32 32
- %p10 = getelementptr inbounds i32, ptr addrspace(1) %p9, i32 36
- %v1 = load volatile i32, ptr addrspace(1) %p1
- %v2 = load volatile i32, ptr addrspace(1) %p2
- %v3 = load volatile i32, ptr addrspace(1) %p3
- %v4 = load volatile i32, ptr addrspace(1) %p4
- %v5 = load volatile i32, ptr addrspace(1) %p5
- %v6 = load volatile i32, ptr addrspace(1) %p6
- %v7 = load volatile i32, ptr addrspace(1) %p7
- %v8 = load volatile i32, ptr addrspace(1) %p8
- %v9 = load volatile i32, ptr addrspace(1) %p9
- %v10 = load volatile i32, ptr addrspace(1) %p10
- call void asm sideeffect "", "v,v,v,v,v,v,v,v,v,v"(i32 %v1, i32 %v2, i32 %v3, i32 %v4, i32 %v5, i32 %v6, i32 %v7, i32 %v8, i32 %v9, i32 %v10)
- store volatile i32 %v1, ptr addrspace(1) poison
- store volatile i32 %v2, ptr addrspace(1) poison
- store volatile i32 %v3, ptr addrspace(1) poison
- store volatile i32 %v4, ptr addrspace(1) poison
- store volatile i32 %v5, ptr addrspace(1) poison
- store volatile i32 %v6, ptr addrspace(1) poison
- store volatile i32 %v7, ptr addrspace(1) poison
- store volatile i32 %v8, ptr addrspace(1) poison
- store volatile i32 %v9, ptr addrspace(1) poison
- store volatile i32 %v10, ptr addrspace(1) poison
- ret void
-}
-
-; GCN-LABEL: {{^}}max_10_vgprs_spill_v32:
-; GCN-DAG: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD0
-; GCN-DAG: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD1
-; GCN: buffer_store_dword v{{[0-9]}},
-; GFX908-DAG: v_accvgpr_write_b32 a0, v{{[0-9]}}
-; GFX908-DAG: v_accvgpr_write_b32 a9, v{{[0-9]}}
-; GCN-NOT: a10
-
-; GFX908: NumVgprs: 10
-; GFX900: ScratchSize: 100
-; GFX908: ScratchSize: 68
-; GFX908: VGPRBlocks: 2
-; GFX908: NumVGPRsForWavesPerEU: 10
-define amdgpu_kernel void @max_10_vgprs_spill_v32(ptr addrspace(1) %p) #0 {
- %tid = call i32 @llvm.amdgcn.workitem.id.x()
- %gep = getelementptr inbounds <32 x float>, ptr addrspace(1) %p, i32 %tid
- %v = load volatile <32 x float>, ptr addrspace(1) %gep
- store volatile <32 x float> %v, ptr addrspace(1) poison
- ret void
-}
; GCN-LABEL: {{^}}max_256_vgprs_spill_9x32:
; GFX900-DAG: s_mov_b32 s{{[0-9]+}}, SCRATCH_RSRC_DWORD0
@@ -219,6 +147,4 @@ st:
declare i32 @llvm.amdgcn.workitem.id.x()
-attributes #0 = { nounwind "amdgpu-num-vgpr"="10" }
attributes #1 = { "amdgpu-flat-work-group-size"="1,256" }
-attributes #2 = { nounwind "amdgpu-num-vgpr"="11" }
diff --git a/llvm/test/CodeGen/AMDGPU/splitkit.mir b/llvm/test/CodeGen/AMDGPU/splitkit.mir
index be5fdada9c41d..6af6399b057af 100644
--- a/llvm/test/CodeGen/AMDGPU/splitkit.mir
+++ b/llvm/test/CodeGen/AMDGPU/splitkit.mir
@@ -1,11 +1,9 @@
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 2
-# RUN: llc -o - %s -mtriple=amdgpu8.03-- -verify-machineinstrs -run-pass=greedy,virtregrewriter | FileCheck %s
+# RUN: llc -o - %s -mtriple=amdgpu8.03-- -verify-machineinstrs -run-pass=greedy,virtregrewriter -amdgpu-stress-sgpr=12 | FileCheck %s
--- |
- define amdgpu_kernel void @func0() #0 { ret void }
- define amdgpu_kernel void @func1() #0 { ret void }
- define amdgpu_kernel void @splitHoist() #0 { ret void }
-
- attributes #0 = { "amdgpu-num-sgpr"="12" }
+ define amdgpu_kernel void @func0() { ret void }
+ define amdgpu_kernel void @func1() { ret void }
+ define amdgpu_kernel void @splitHoist() { ret void }
...
---
# Make sure we only get a single spill+reload even if liverange splitting
diff --git a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
index c70327aaea993..ddfb19b42c349 100644
--- a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
+++ b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
@@ -1,4 +1,4 @@
-# RUN: llc -mtriple=amdgpu9.00--amdhsa -run-pass si-optimize-exec-masking -verify-machineinstrs -o - %s | FileCheck %s
+# RUN: llc -mtriple=amdgpu9.00--amdhsa -amdgpu-stress-sgpr=16 -run-pass si-optimize-exec-masking -verify-machineinstrs -o - %s | FileCheck %s
--- |
define amdgpu_kernel void @undefined_physreg_sgpr_spill() #0 {
unreachable
@@ -8,7 +8,7 @@
unreachable
}
- attributes #0 = { nounwind "amdgpu-num-sgpr"="16" }
+ attributes #0 = { nounwind }
...
---
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress11.ll b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress11.ll
new file mode 100644
index 0000000000000..53cf5053cf333
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress11.ll
@@ -0,0 +1,214 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 4
+; RUN: llc -mtriple=amdgpu12.00 -amdgpu-stress-vgpr=11 -o - %s | FileCheck -check-prefix=CHECK %s
+
+define amdgpu_cs void @max_11_vgprs_branch(ptr addrspace(1) %p, i32 %tmp) nounwind {
+; CHECK-LABEL: max_11_vgprs_branch:
+; CHECK: ; %bb.0: ; %.entry
+; CHECK-NEXT: global_load_b32 v3, v[0:1], off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: s_mov_b32 s0, exec_lo
+; CHECK-NEXT: v_ashrrev_i32_e32 v4, 31, v3
+; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)
+; CHECK-NEXT: v_lshlrev_b64_e32 v[3:4], 2, v[3:4]
+; CHECK-NEXT: v_add_co_u32 v0, vcc_lo, v0, v3
+; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1)
+; CHECK-NEXT: v_add_co_ci_u32_e64 v1, null, v1, v4, vcc_lo
+; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:336 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v3, off offset:8 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:448 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v3, off offset:12 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:576 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v3, off ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:720 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v3, off offset:4 ; 4-byte Folded Spill
+; CHECK-NEXT: v_cmpx_eq_u32_e32 0, v2
+; CHECK-NEXT: s_xor_b32 s0, exec_lo, s0
+; CHECK-NEXT: s_cbranch_execz .LBB0_2
+; CHECK-NEXT: ; %bb.1: ; %.false
+; CHECK-NEXT: global_load_b32 v10, v[0:1], off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:16 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:20 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:24 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:160 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:28 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:240 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v0, off offset:32 ; 4-byte Folded Spill
+; CHECK-NEXT: ;;#ASMSTART
+; CHECK-NEXT: ;;#ASMEND
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v10, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:16 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:20 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:24 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:28 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:32 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: ; implicit-def: $vgpr0
+; CHECK-NEXT: ; kill: killed $vgpr0
+; CHECK-NEXT: ; implicit-def: $vgpr0
+; CHECK-NEXT: ; kill: killed $vgpr0
+; CHECK-NEXT: ; implicit-def: $vgpr0_vgpr1
+; CHECK-NEXT: .LBB0_2: ; %Flow
+; CHECK-NEXT: s_and_not1_saveexec_b32 s0, s0
+; CHECK-NEXT: s_cbranch_execz .LBB0_4
+; CHECK-NEXT: ; %bb.3: ; %.true
+; CHECK-NEXT: global_load_b32 v10, v[0:1], off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:16 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:20 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:24 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:160 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:28 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:240 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v0, off offset:32 ; 4-byte Folded Spill
+; CHECK-NEXT: ;;#ASMSTART
+; CHECK-NEXT: ;;#ASMEND
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v10, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:16 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:20 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:24 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:28 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:32 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: .LBB0_4: ; %.exit
+; CHECK-NEXT: s_or_b32 exec_lo, exec_lo, s0
+; CHECK-NEXT: scratch_load_b32 v0, off, off th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:4 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: s_endpgm
+.entry:
+ %tid = load volatile i32, ptr addrspace(1) poison
+ %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
+ %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
+ %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
+ %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
+ %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
+ %p6 = getelementptr inbounds i32, ptr addrspace(1) %p5, i32 20
+ %p7 = getelementptr inbounds i32, ptr addrspace(1) %p6, i32 24
+ %p8 = getelementptr inbounds i32, ptr addrspace(1) %p7, i32 28
+ %p9 = getelementptr inbounds i32, ptr addrspace(1) %p8, i32 32
+ %p10 = getelementptr inbounds i32, ptr addrspace(1) %p9, i32 36
+ %v7 = load volatile i32, ptr addrspace(1) %p7
+ %v8 = load volatile i32, ptr addrspace(1) %p8
+ %v9 = load volatile i32, ptr addrspace(1) %p9
+ %v10 = load volatile i32, ptr addrspace(1) %p10
+ %cmp = icmp ne i32 %tmp, 0
+ br i1 %cmp, label %.true, label %.false
+
+.true:
+ %v1_t = load volatile i32, ptr addrspace(1) %p1
+ %v2_t = load volatile i32, ptr addrspace(1) %p2
+ %v3_t = load volatile i32, ptr addrspace(1) %p3
+ %v4_t = load volatile i32, ptr addrspace(1) %p4
+ %v5_t = load volatile i32, ptr addrspace(1) %p5
+ %v6_t = load volatile i32, ptr addrspace(1) %p6
+ call void asm sideeffect "", "~{v[0:9]}" ()
+ store volatile i32 %v1_t, ptr addrspace(1) poison
+ store volatile i32 %v2_t, ptr addrspace(1) poison
+ store volatile i32 %v3_t, ptr addrspace(1) poison
+ store volatile i32 %v4_t, ptr addrspace(1) poison
+ store volatile i32 %v5_t, ptr addrspace(1) poison
+ store volatile i32 %v6_t, ptr addrspace(1) poison
+ store volatile i32 %v7, ptr addrspace(1) poison
+ store volatile i32 %v8, ptr addrspace(1) poison
+
+ br label %.exit
+
+.false:
+ %v1_f = load volatile i32, ptr addrspace(1) %p1
+ %v2_f = load volatile i32, ptr addrspace(1) %p2
+ %v3_f = load volatile i32, ptr addrspace(1) %p3
+ %v4_f = load volatile i32, ptr addrspace(1) %p4
+ %v5_f = load volatile i32, ptr addrspace(1) %p5
+ %v6_f = load volatile i32, ptr addrspace(1) %p6
+ call void asm sideeffect "", "~{v[0:9]}" ()
+ store volatile i32 %v1_f, ptr addrspace(1) poison
+ store volatile i32 %v2_f, ptr addrspace(1) poison
+ store volatile i32 %v3_f, ptr addrspace(1) poison
+ store volatile i32 %v4_f, ptr addrspace(1) poison
+ store volatile i32 %v5_f, ptr addrspace(1) poison
+ store volatile i32 %v6_f, ptr addrspace(1) poison
+ store volatile i32 %v7, ptr addrspace(1) poison
+ store volatile i32 %v8, ptr addrspace(1) poison
+
+ br label %.exit
+
+.exit:
+ store volatile i32 %v9, ptr addrspace(1) poison
+ store volatile i32 %v10, ptr addrspace(1) poison
+ ret void
+}
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress6.ll b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress6.ll
new file mode 100644
index 0000000000000..72aa08ee41ac0
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load-stress6.ll
@@ -0,0 +1,69 @@
+; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 4
+; RUN: llc -mtriple=amdgpu12.00 -amdgpu-stress-vgpr=6 -o - %s | FileCheck -check-prefix=CHECK %s
+
+define amdgpu_cs void @max_6_vgprs(ptr addrspace(1) %p) nounwind {
+; CHECK-LABEL: max_6_vgprs:
+; CHECK: ; %bb.0:
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: v_ashrrev_i32_e32 v3, 31, v2
+; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)
+; CHECK-NEXT: v_lshlrev_b64_e32 v[2:3], 2, v[2:3]
+; CHECK-NEXT: v_add_co_u32 v0, vcc_lo, v0, v2
+; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1)
+; CHECK-NEXT: v_add_co_ci_u32_e64 v1, null, v1, v3, vcc_lo
+; CHECK-NEXT: global_load_b32 v5, v[0:1], off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:4 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v2, off offset:8 ; 4-byte Folded Spill
+; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:160 scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: scratch_store_b32 off, v0, off offset:12 ; 4-byte Folded Spill
+; CHECK-NEXT: ;;#ASMSTART
+; CHECK-NEXT: ;;#ASMEND
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v5, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:4 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
+; CHECK-NEXT: s_wait_loadcnt 0x0
+; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
+; CHECK-NEXT: s_wait_storecnt 0x0
+; CHECK-NEXT: s_endpgm
+ %tid = load volatile i32, ptr addrspace(1) poison
+ %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
+ %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
+ %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
+ %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
+ %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
+ %v1 = load volatile i32, ptr addrspace(1) %p1
+ %v2 = load volatile i32, ptr addrspace(1) %p2
+ %v3 = load volatile i32, ptr addrspace(1) %p3
+ %v4 = load volatile i32, ptr addrspace(1) %p4
+ %v5 = load volatile i32, ptr addrspace(1) %p5
+ call void asm sideeffect "", "~{v[0:4]}" ()
+ store volatile i32 %v1, ptr addrspace(1) poison
+ store volatile i32 %v2, ptr addrspace(1) poison
+ store volatile i32 %v3, ptr addrspace(1) poison
+ store volatile i32 %v4, ptr addrspace(1) poison
+ store volatile i32 %v5, ptr addrspace(1) poison
+ ret void
+}
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
index 735b8e0f74089..281aca0a59fd8 100644
--- a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.ll
@@ -1,286 +1,6 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 4
; RUN: llc -mtriple=amdgpu12.00 -o - %s | FileCheck -check-prefix=CHECK %s
-define amdgpu_cs void @max_6_vgprs(ptr addrspace(1) %p) "amdgpu-num-vgpr"="6" nounwind {
-; CHECK-LABEL: max_6_vgprs:
-; CHECK: ; %bb.0:
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: v_ashrrev_i32_e32 v3, 31, v2
-; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)
-; CHECK-NEXT: v_lshlrev_b64_e32 v[2:3], 2, v[2:3]
-; CHECK-NEXT: v_add_co_u32 v0, vcc_lo, v0, v2
-; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1)
-; CHECK-NEXT: v_add_co_ci_u32_e64 v1, null, v1, v3, vcc_lo
-; CHECK-NEXT: global_load_b32 v5, v[0:1], off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:4 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:8 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:160 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v0, off offset:12 ; 4-byte Folded Spill
-; CHECK-NEXT: ;;#ASMSTART
-; CHECK-NEXT: ;;#ASMEND
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v5, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:4 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: s_endpgm
- %tid = load volatile i32, ptr addrspace(1) poison
- %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
- %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
- %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
- %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
- %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
- %v1 = load volatile i32, ptr addrspace(1) %p1
- %v2 = load volatile i32, ptr addrspace(1) %p2
- %v3 = load volatile i32, ptr addrspace(1) %p3
- %v4 = load volatile i32, ptr addrspace(1) %p4
- %v5 = load volatile i32, ptr addrspace(1) %p5
- call void asm sideeffect "", "~{v[0:4]}" ()
- store volatile i32 %v1, ptr addrspace(1) poison
- store volatile i32 %v2, ptr addrspace(1) poison
- store volatile i32 %v3, ptr addrspace(1) poison
- store volatile i32 %v4, ptr addrspace(1) poison
- store volatile i32 %v5, ptr addrspace(1) poison
- ret void
-}
-
-define amdgpu_cs void @max_11_vgprs_branch(ptr addrspace(1) %p, i32 %tmp) "amdgpu-num-vgpr"="11" nounwind {
-; CHECK-LABEL: max_11_vgprs_branch:
-; CHECK: ; %bb.0: ; %.entry
-; CHECK-NEXT: global_load_b32 v3, v[0:1], off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: s_mov_b32 s0, exec_lo
-; CHECK-NEXT: v_ashrrev_i32_e32 v4, 31, v3
-; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1) | instskip(NEXT) | instid1(VALU_DEP_1)
-; CHECK-NEXT: v_lshlrev_b64_e32 v[3:4], 2, v[3:4]
-; CHECK-NEXT: v_add_co_u32 v0, vcc_lo, v0, v3
-; CHECK-NEXT: s_delay_alu instid0(VALU_DEP_1)
-; CHECK-NEXT: v_add_co_ci_u32_e64 v1, null, v1, v4, vcc_lo
-; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:336 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v3, off offset:8 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:448 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v3, off offset:12 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:576 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v3, off ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v3, v[0:1], off offset:720 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v3, off offset:4 ; 4-byte Folded Spill
-; CHECK-NEXT: v_cmpx_eq_u32_e32 0, v2
-; CHECK-NEXT: s_xor_b32 s0, exec_lo, s0
-; CHECK-NEXT: s_cbranch_execz .LBB1_2
-; CHECK-NEXT: ; %bb.1: ; %.false
-; CHECK-NEXT: global_load_b32 v10, v[0:1], off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:16 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:20 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:24 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:160 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:28 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:240 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v0, off offset:32 ; 4-byte Folded Spill
-; CHECK-NEXT: ;;#ASMSTART
-; CHECK-NEXT: ;;#ASMEND
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v10, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:16 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:20 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:24 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:28 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:32 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: ; implicit-def: $vgpr0
-; CHECK-NEXT: ; kill: killed $vgpr0
-; CHECK-NEXT: ; implicit-def: $vgpr0
-; CHECK-NEXT: ; kill: killed $vgpr0
-; CHECK-NEXT: ; implicit-def: $vgpr0_vgpr1
-; CHECK-NEXT: .LBB1_2: ; %Flow
-; CHECK-NEXT: s_and_not1_saveexec_b32 s0, s0
-; CHECK-NEXT: s_cbranch_execz .LBB1_4
-; CHECK-NEXT: ; %bb.3: ; %.true
-; CHECK-NEXT: global_load_b32 v10, v[0:1], off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:16 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:16 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:48 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:20 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:96 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:24 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v2, v[0:1], off offset:160 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v2, off offset:28 ; 4-byte Folded Spill
-; CHECK-NEXT: global_load_b32 v0, v[0:1], off offset:240 scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: scratch_store_b32 off, v0, off offset:32 ; 4-byte Folded Spill
-; CHECK-NEXT: ;;#ASMSTART
-; CHECK-NEXT: ;;#ASMEND
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v10, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:16 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:20 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:24 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:28 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:32 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:8 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:12 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: .LBB1_4: ; %.exit
-; CHECK-NEXT: s_or_b32 exec_lo, exec_lo, s0
-; CHECK-NEXT: scratch_load_b32 v0, off, off th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: scratch_load_b32 v0, off, off offset:4 th:TH_LOAD_LU ; 4-byte Folded Reload
-; CHECK-NEXT: s_wait_loadcnt 0x0
-; CHECK-NEXT: global_store_b32 v[0:1], v0, off scope:SCOPE_SYS
-; CHECK-NEXT: s_wait_storecnt 0x0
-; CHECK-NEXT: s_endpgm
-.entry:
- %tid = load volatile i32, ptr addrspace(1) poison
- %p1 = getelementptr inbounds i32, ptr addrspace(1) %p, i32 %tid
- %p2 = getelementptr inbounds i32, ptr addrspace(1) %p1, i32 4
- %p3 = getelementptr inbounds i32, ptr addrspace(1) %p2, i32 8
- %p4 = getelementptr inbounds i32, ptr addrspace(1) %p3, i32 12
- %p5 = getelementptr inbounds i32, ptr addrspace(1) %p4, i32 16
- %p6 = getelementptr inbounds i32, ptr addrspace(1) %p5, i32 20
- %p7 = getelementptr inbounds i32, ptr addrspace(1) %p6, i32 24
- %p8 = getelementptr inbounds i32, ptr addrspace(1) %p7, i32 28
- %p9 = getelementptr inbounds i32, ptr addrspace(1) %p8, i32 32
- %p10 = getelementptr inbounds i32, ptr addrspace(1) %p9, i32 36
- %v7 = load volatile i32, ptr addrspace(1) %p7
- %v8 = load volatile i32, ptr addrspace(1) %p8
- %v9 = load volatile i32, ptr addrspace(1) %p9
- %v10 = load volatile i32, ptr addrspace(1) %p10
- %cmp = icmp ne i32 %tmp, 0
- br i1 %cmp, label %.true, label %.false
-
-.true:
- %v1_t = load volatile i32, ptr addrspace(1) %p1
- %v2_t = load volatile i32, ptr addrspace(1) %p2
- %v3_t = load volatile i32, ptr addrspace(1) %p3
- %v4_t = load volatile i32, ptr addrspace(1) %p4
- %v5_t = load volatile i32, ptr addrspace(1) %p5
- %v6_t = load volatile i32, ptr addrspace(1) %p6
- call void asm sideeffect "", "~{v[0:9]}" ()
- store volatile i32 %v1_t, ptr addrspace(1) poison
- store volatile i32 %v2_t, ptr addrspace(1) poison
- store volatile i32 %v3_t, ptr addrspace(1) poison
- store volatile i32 %v4_t, ptr addrspace(1) poison
- store volatile i32 %v5_t, ptr addrspace(1) poison
- store volatile i32 %v6_t, ptr addrspace(1) poison
- store volatile i32 %v7, ptr addrspace(1) poison
- store volatile i32 %v8, ptr addrspace(1) poison
-
- br label %.exit
-
-.false:
- %v1_f = load volatile i32, ptr addrspace(1) %p1
- %v2_f = load volatile i32, ptr addrspace(1) %p2
- %v3_f = load volatile i32, ptr addrspace(1) %p3
- %v4_f = load volatile i32, ptr addrspace(1) %p4
- %v5_f = load volatile i32, ptr addrspace(1) %p5
- %v6_f = load volatile i32, ptr addrspace(1) %p6
- call void asm sideeffect "", "~{v[0:9]}" ()
- store volatile i32 %v1_f, ptr addrspace(1) poison
- store volatile i32 %v2_f, ptr addrspace(1) poison
- store volatile i32 %v3_f, ptr addrspace(1) poison
- store volatile i32 %v4_f, ptr addrspace(1) poison
- store volatile i32 %v5_f, ptr addrspace(1) poison
- store volatile i32 %v6_f, ptr addrspace(1) poison
- store volatile i32 %v7, ptr addrspace(1) poison
- store volatile i32 %v8, ptr addrspace(1) poison
-
- br label %.exit
-
-.exit:
- store volatile i32 %v9, ptr addrspace(1) poison
- store volatile i32 %v10, ptr addrspace(1) poison
- ret void
-}
-
-
declare i32 @foo() nounwind
declare <8 x half> @bar(<32 x i64>) nounwind
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
index 323d898b5b2d9..6a9974df5f257 100644
--- a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
@@ -1,15 +1,15 @@
# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 3
-# RUN: llc -mtriple=amdgpu12.00-amd-amdhsa -o - %s -run-pass=greedy -run-pass=amdgpu-mark-last-scratch-load -verify-machineinstrs | FileCheck -check-prefix=CHECK %s
-# RUN: llc -mtriple=amdgpu12.00-amd-amdhsa -o - %s -passes=greedy,amdgpu-mark-last-scratch-load | FileCheck -check-prefix=CHECK %s
+# RUN: llc -mtriple=amdgpu12.00-amd-amdhsa -o - -amdgpu-stress-vgpr=12 %s -run-pass=greedy -run-pass=amdgpu-mark-last-scratch-load -verify-machineinstrs | FileCheck -check-prefix=CHECK %s
+# RUN: llc -mtriple=amdgpu12.00-amd-amdhsa -o - -amdgpu-stress-vgpr=12 %s -passes=greedy,amdgpu-mark-last-scratch-load | FileCheck -check-prefix=CHECK %s
--- |
- define amdgpu_cs void @test_spill_12x32() "amdgpu-num-vgpr"="12" {
+ define amdgpu_cs void @test_spill_12x32() {
ret void
}
- define amdgpu_cs void @test_spill_384() "amdgpu-num-vgpr"="12" {
+ define amdgpu_cs void @test_spill_384() {
ret void
}
- define amdgpu_ps void @test_loop_12() "amdgpu-num-vgpr"="12" {
+ define amdgpu_ps void @test_loop_12() {
ret void
}
...
diff --git a/llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll b/llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll
index 82516c1e01268..65df0585e7ae5 100644
--- a/llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll
+++ b/llvm/test/CodeGen/AMDGPU/whole-wave-register-copy.ll
@@ -1,5 +1,5 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa < %s | FileCheck -check-prefix=GFX90A %s
+; RUN: llc -mtriple=amdgpu9.0a-amd-amdhsa -amdgpu-stress-vgpr=41 -amdgpu-stress-sgpr=34 -amdgpu-stress-agpr=41 < %s | FileCheck -check-prefix=GFX90A %s
; The test forces a high vector register pressure and there won't be sufficient VGPRs to be allocated
; for writelane/readlane SGPR spill instructions. Regalloc would split the vector register liverange
@@ -68,4 +68,4 @@ define void @vector_reg_liverange_split() #0 {
declare void @foo()
-attributes #0 = { "amdgpu-num-vgpr"="41" "amdgpu-num-sgpr"="34" nounwind }
+attributes #0 = { nounwind }
diff --git a/llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll b/llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll
index 064438977142f..9f1558af7bf9b 100644
--- a/llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll
+++ b/llvm/test/CodeGen/AMDGPU/whole-wave-register-spill.ll
@@ -1,6 +1,6 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -o - %s | FileCheck -check-prefix=GCN %s
-; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -O0 -o - %s | FileCheck -check-prefix=GCN-O0 %s
+; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -o - -amdgpu-stress-vgpr=41 -amdgpu-stress-sgpr=34 %s | FileCheck -check-prefix=GCN %s
+; RUN: llc -mtriple=amdgpu9.06-amd-amdhsa -O0 -o - -amdgpu-stress-vgpr=41 -amdgpu-stress-sgpr=34 %s | FileCheck -check-prefix=GCN-O0 %s
; Test whole-wave register spilling.
@@ -133,4 +133,4 @@ define void @test() #0 {
declare void @ext_func();
-attributes #0 = { nounwind "amdgpu-num-vgpr"="41" "amdgpu-num-sgpr"="34"}
+attributes #0 = { nounwind}
>From d6f88af9ec5434cdebc27c76eb28c265d2ed15b2 Mon Sep 17 00:00:00 2001
From: Vlad <Vladislav.Romanov at amd.com>
Date: Tue, 4 Aug 2026 07:37:44 -0500
Subject: [PATCH 2/2] formatting
---
llvm/lib/Target/AMDGPU/GCNSubtarget.cpp | 22 +++++++++++-----------
1 file changed, 11 insertions(+), 11 deletions(-)
diff --git a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
index c47565ef9b1b7..0d9aaccc0a8a1 100644
--- a/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
+++ b/llvm/lib/Target/AMDGPU/GCNSubtarget.cpp
@@ -53,17 +53,17 @@ static cl::opt<unsigned>
cl::desc("Number of addresses from which to enable MIMG NSA."),
cl::init(2), cl::Hidden);
-static cl::opt<unsigned> StressVGPRLimit(
- "amdgpu-stress-vgpr", cl::Hidden, cl::init(0),
- cl::desc("Limit VGPRs to N arch registers"));
+static cl::opt<unsigned>
+ StressVGPRLimit("amdgpu-stress-vgpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit VGPRs to N arch registers"));
-static cl::opt<unsigned> StressAGPRLimit(
- "amdgpu-stress-agpr", cl::Hidden, cl::init(0),
- cl::desc("Limit AGPRs to N registers"));
+static cl::opt<unsigned>
+ StressAGPRLimit("amdgpu-stress-agpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit AGPRs to N registers"));
-static cl::opt<unsigned> StressSGPRLimit(
- "amdgpu-stress-sgpr", cl::Hidden, cl::init(0),
- cl::desc("Limit SGPRs to N registers"));
+static cl::opt<unsigned>
+ StressSGPRLimit("amdgpu-stress-sgpr", cl::Hidden, cl::init(0),
+ cl::desc("Limit SGPRs to N registers"));
GCNSubtarget::~GCNSubtarget() = default;
@@ -612,8 +612,8 @@ GCNSubtarget::getMaxNumVectorRegs(const Function &F) const {
DynamicVGPRBlockSize = getDynamicVGPRBlockSize();
std::pair<unsigned, unsigned> Waves = getWavesPerEU(F);
- const unsigned MaxVectorRegs = getMaxNumVGPRs(Waves.first,
- DynamicVGPRBlockSize);
+ const unsigned MaxVectorRegs =
+ getMaxNumVGPRs(Waves.first, DynamicVGPRBlockSize);
unsigned MaxNumVGPRs = MaxVectorRegs;
unsigned MaxNumAGPRs = 0;
More information about the llvm-commits
mailing list