[llvm] [AMDGPU] Add backward compatibility layer for kernarg preloading (PR #119167)
Austin Kerbow via llvm-commits
llvm-commits at lists.llvm.org
Sun Dec 8 20:59:16 PST 2024
https://github.com/kerbowa created https://github.com/llvm/llvm-project/pull/119167
Add a prologue to the kernel entry to handle cases where code designed for kernarg preloading is executed on hardware equipped with incompatible firmware. If hardware has compatible firmware the 256 bytes at the start of the kernel entry will be skipped. This skipping is done automatically by hardware that supports the feature.
A pass is added which is intended to be run at the very end of the pipeline to avoid any optimizations that would assume the prologue is a real predecessor block to the actual code start. In reality we have two possible entry points for the function. 1. The optimized path that supports kernarg preloading which begins at an offset of 256 bytes. 2. The backwards compatible entry point which starts at offset 0.
>From 2860450eb25c91cd474049b5d42dcaf52147bc39 Mon Sep 17 00:00:00 2001
From: Austin Kerbow <Austin.Kerbow at amd.com>
Date: Thu, 5 Dec 2024 14:59:14 -0800
Subject: [PATCH] [AMDGPU] Add backward compatibility layer for kernarg
preloading
Add a prologue to the kernel entry to handle cases where code designed
for kernarg preloading is executed on hardware equipped with
incompatible firmware. If hardware has compatible firmware the 256 bytes
at the start of the kernel entry will be skipped. This skipping is done
automatically by hardware that supports the feature.
A pass is added which is intended to be run at the very end of the
pipeline to avoid any optimizations that would assume the prologue is a
real predecessor block to the actual code start. In reality we have two
possible entry points for the function. 1. The optimized path that
supports kernarg preloading which begins at an offset of 256 bytes. 2.
The backwards compatible entry point which starts at offset 0.
---
llvm/docs/AMDGPUUsage.rst | 5 +-
llvm/lib/Target/AMDGPU/AMDGPU.h | 4 +
.../Target/AMDGPU/AMDGPUArgumentUsageInfo.h | 2 +
llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp | 6 -
.../AMDGPU/AMDGPUPreloadKernargHeader.cpp | 229 +
.../AMDGPU/AMDGPUPreloadKernargHeader.h | 25 +
.../lib/Target/AMDGPU/AMDGPUTargetMachine.cpp | 2 +
llvm/lib/Target/AMDGPU/CMakeLists.txt | 1 +
.../MCTargetDesc/AMDGPUTargetStreamer.cpp | 23 -
.../MCTargetDesc/AMDGPUTargetStreamer.h | 14 -
.../Target/AMDGPU/SIMachineFunctionInfo.cpp | 2 +
llvm/test/CodeGen/AMDGPU/llc-pipeline.ll | 5 +
.../AMDGPU/preload-implicit-kernargs.ll | 2624 ++++++++-
.../CodeGen/AMDGPU/preload-kernarg-header.ll | 23 -
llvm/test/CodeGen/AMDGPU/preload-kernargs.ll | 4712 ++++++++++++++++-
15 files changed, 7277 insertions(+), 400 deletions(-)
create mode 100644 llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.cpp
create mode 100644 llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.h
delete mode 100644 llvm/test/CodeGen/AMDGPU/preload-kernarg-header.ll
diff --git a/llvm/docs/AMDGPUUsage.rst b/llvm/docs/AMDGPUUsage.rst
index 5c6034753eb4af..40b393224f15dd 100644
--- a/llvm/docs/AMDGPUUsage.rst
+++ b/llvm/docs/AMDGPUUsage.rst
@@ -5914,10 +5914,7 @@ additional 256 bytes to the kernel_code_entry_byte_offset. This addition
facilitates the incorporation of a prologue to the kernel entry to handle cases
where code designed for kernarg preloading is executed on hardware equipped with
incompatible firmware. If hardware has compatible firmware the 256 bytes at the
-start of the kernel entry will be skipped. Additionally, the compiler backend
-may insert a trap instruction at the start of the kernel prologue to manage
-situations where kernarg preloading is attempted on hardware with incompatible
-firmware.
+start of the kernel entry will be skipped.
With code object V5 and later, hidden kernel arguments that are normally
accessed through the Implicit Argument Ptr, may be preloaded into User SGPRs.
diff --git a/llvm/lib/Target/AMDGPU/AMDGPU.h b/llvm/lib/Target/AMDGPU/AMDGPU.h
index b9769a1baf4d17..94de55923b4a3c 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPU.h
+++ b/llvm/lib/Target/AMDGPU/AMDGPU.h
@@ -64,6 +64,7 @@ createAMDGPULowerModuleLDSLegacyPass(const AMDGPUTargetMachine *TM = nullptr);
ModulePass *createAMDGPULowerBufferFatPointersPass();
FunctionPass *createSIModeRegisterPass();
FunctionPass *createGCNPreRAOptimizationsPass();
+FunctionPass *createAMDGPUPreloadKernargHeaderLegacyPass();
struct AMDGPUSimplifyLibCallsPass : PassInfoMixin<AMDGPUSimplifyLibCallsPass> {
AMDGPUSimplifyLibCallsPass() {}
@@ -230,6 +231,9 @@ extern char &AMDGPUPerfHintAnalysisLegacyID;
void initializeGCNRegPressurePrinterPass(PassRegistry &);
extern char &GCNRegPressurePrinterID;
+void initializeAMDGPUPreloadKernargHeaderLegacyPass(PassRegistry &);
+extern char &AMDGPUPreloadKernargHeaderLegacyID;
+
// Passes common to R600 and SI
FunctionPass *createAMDGPUPromoteAlloca();
void initializeAMDGPUPromoteAllocaPass(PassRegistry&);
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUArgumentUsageInfo.h b/llvm/lib/Target/AMDGPU/AMDGPUArgumentUsageInfo.h
index 06b2f181c276cd..859625e9b538a3 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUArgumentUsageInfo.h
+++ b/llvm/lib/Target/AMDGPU/AMDGPUArgumentUsageInfo.h
@@ -9,6 +9,7 @@
#ifndef LLVM_LIB_TARGET_AMDGPU_AMDGPUARGUMENTUSAGEINFO_H
#define LLVM_LIB_TARGET_AMDGPU_AMDGPUARGUMENTUSAGEINFO_H
+#include "MCTargetDesc/AMDGPUMCTargetDesc.h"
#include "llvm/ADT/DenseMap.h"
#include "llvm/CodeGen/Register.h"
#include "llvm/Pass.h"
@@ -161,6 +162,7 @@ struct AMDGPUFunctionArgInfo {
// Map the index of preloaded kernel arguments to its descriptor.
SmallDenseMap<int, KernArgPreloadDescriptor> PreloadKernArgs{};
+ Register FirstKernArgPreloadReg = AMDGPU::NoRegister;
std::tuple<const ArgDescriptor *, const TargetRegisterClass *, LLT>
getPreloadedValue(PreloadedValue Value) const;
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp b/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp
index 90c341ac0819cc..737b2f740d6f77 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUAsmPrinter.cpp
@@ -207,12 +207,6 @@ void AMDGPUAsmPrinter::emitFunctionBodyStart() {
if (STM.isAmdHsaOS())
HSAMetadataStream->emitKernel(*MF, CurrentProgramInfo);
-
- if (MFI.getNumKernargPreloadedSGPRs() > 0) {
- assert(AMDGPU::hasKernargPreload(STM));
- getTargetStreamer()->EmitKernargPreloadHeader(*getGlobalSTI(),
- STM.isAmdHsaOS());
- }
}
void AMDGPUAsmPrinter::emitFunctionBodyEnd() {
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.cpp b/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.cpp
new file mode 100644
index 00000000000000..756665a7e4d7a2
--- /dev/null
+++ b/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.cpp
@@ -0,0 +1,229 @@
+//===- AMDGPUPreloadKernargHeader.cpp - Preload Kernarg Header ------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+/// \file This pass handles the creation of the backwards compatability layer
+/// for kernarg prealoding. Code may be compiled with the feature enabled, while
+/// the kernel is executed on hardware without firmware support.
+///
+/// To avoid the need for recompilation, we insert a block at the beginning of
+/// the kernel that is responsible for loading the kernel arguments into SGPRs
+/// using s_load instructions which setup the registers exactly as they would be
+/// by firmware if the code were executed on a system that supported kernarg
+/// preladoing.
+///
+/// This essentially allows for two entry points for the kernel. Firmware that
+/// supports the feature will automatically jump past the first 256 bytes of the
+/// program, skipping the backwards compatibility layer and directly beginning
+/// execution on the fast code path.
+///
+/// This pass should be run as late as possible, to avoid any optimization that
+/// may assume that padding is dead code or that the prologue added here is a
+/// true predecessor of the kernel entry block.
+//===----------------------------------------------------------------------===//
+
+#include "AMDGPUPreloadKernargHeader.h"
+#include "AMDGPU.h"
+#include "GCNSubtarget.h"
+#include "SIMachineFunctionInfo.h"
+#include "llvm/CodeGen/MachineFunctionPass.h"
+#include "llvm/TargetParser/TargetParser.h"
+
+using namespace llvm;
+
+#define DEBUG_TYPE "amdgpu-preload-kernarg-header"
+
+namespace {
+
+struct LoadConfig {
+ unsigned Size;
+ const TargetRegisterClass *RegClass;
+ unsigned Opcode;
+ Register LoadReg;
+
+ // Constructor for the static config array
+ LoadConfig(unsigned S, const TargetRegisterClass *RC, unsigned Op)
+ : Size(S), RegClass(RC), Opcode(Op), LoadReg(AMDGPU::NoRegister) {}
+
+ // Constructor for the return value
+ LoadConfig(unsigned S, const TargetRegisterClass *RC, unsigned Op,
+ Register Reg)
+ : Size(S), RegClass(RC), Opcode(Op), LoadReg(Reg) {}
+};
+
+class AMDGPUPreloadKernargHeader {
+public:
+ AMDGPUPreloadKernargHeader(MachineFunction &MF);
+
+ bool run();
+
+private:
+ MachineFunction &MF;
+ const GCNSubtarget &ST;
+ const SIMachineFunctionInfo &MFI;
+ const SIInstrInfo &TII;
+ const TargetRegisterInfo &TRI;
+
+ // Create a new block before the entry point to the kernel. Firmware that
+ // supports preloading kernel arguments will automatically jump past this
+ // block to the alternative kernel entry point.
+ void createBackCompatBlock();
+
+ // Add instructions to load kernel arguments into SGPRs, returns the number of
+ // s_load instructions added.
+ unsigned addBackCompatLoads(MachineBasicBlock *BackCompatMBB,
+ Register KernargSegmentPtr,
+ unsigned NumKernargPreloadSGPRs);
+};
+
+class AMDGPUPreloadKernargHeaderLegacy : public MachineFunctionPass {
+public:
+ static char ID;
+
+ AMDGPUPreloadKernargHeaderLegacy() : MachineFunctionPass(ID) {}
+
+ StringRef getPassName() const override {
+ return "AMDGPU Preload Kernarg Header";
+ }
+
+ bool runOnMachineFunction(MachineFunction &MF) override;
+};
+
+} // end anonymous namespace
+
+char AMDGPUPreloadKernargHeaderLegacy::ID = 0;
+
+INITIALIZE_PASS(AMDGPUPreloadKernargHeaderLegacy, DEBUG_TYPE,
+ "AMDGPU Preload Kernarg Header", false, false)
+
+char &llvm::AMDGPUPreloadKernargHeaderLegacyID =
+ AMDGPUPreloadKernargHeaderLegacy::ID;
+
+FunctionPass *llvm::createAMDGPUPreloadKernargHeaderLegacyPass() {
+ return new AMDGPUPreloadKernargHeaderLegacy();
+}
+
+bool AMDGPUPreloadKernargHeaderLegacy::runOnMachineFunction(
+ MachineFunction &MF) {
+ return AMDGPUPreloadKernargHeader(MF).run();
+}
+
+AMDGPUPreloadKernargHeader::AMDGPUPreloadKernargHeader(MachineFunction &MF)
+ : MF(MF), ST(MF.getSubtarget<GCNSubtarget>()),
+ MFI(*MF.getInfo<SIMachineFunctionInfo>()), TII(*ST.getInstrInfo()),
+ TRI(*ST.getRegisterInfo()) {}
+
+bool AMDGPUPreloadKernargHeader::run() {
+ if (!ST.hasKernargPreload())
+ return false;
+
+ unsigned NumPreloadSGPRs = MFI.getNumKernargPreloadedSGPRs();
+ if (NumPreloadSGPRs <= 0)
+ return false;
+
+ if (MF.begin() == MF.end())
+ return false;
+
+ createBackCompatBlock();
+
+ return true;
+}
+
+void AMDGPUPreloadKernargHeader::createBackCompatBlock() {
+ auto KernelEntryMBB = MF.begin();
+ MachineBasicBlock *BackCompatMBB = MF.CreateMachineBasicBlock();
+ MF.insert(KernelEntryMBB, BackCompatMBB);
+ BackCompatMBB->addSuccessor(&*KernelEntryMBB);
+
+ assert(MFI.getUserSGPRInfo().hasKernargSegmentPtr());
+ Register KernargSegmentPtr = MFI.getArgInfo().KernargSegmentPtr.getRegister();
+ BackCompatMBB->addLiveIn(KernargSegmentPtr);
+
+ unsigned NumKernargPreloadSGPRs = MFI.getNumKernargPreloadedSGPRs();
+ unsigned NumInstrs = 0;
+
+ // Load kernel arguments to SGPRs
+ NumInstrs += addBackCompatLoads(BackCompatMBB, KernargSegmentPtr,
+ NumKernargPreloadSGPRs);
+
+ AMDGPU::IsaVersion IV = AMDGPU::getIsaVersion(ST.getCPU());
+ unsigned Waitcnt =
+ AMDGPU::encodeWaitcnt(IV, getVmcntBitMask(IV), getExpcntBitMask(IV), 0);
+
+ // Wait for loads to complete
+ BuildMI(BackCompatMBB, DebugLoc(), TII.get(AMDGPU::S_WAITCNT))
+ .addImm(Waitcnt);
+ NumInstrs++;
+
+ // Set PC to the actual kernel entry point. Add padding to fill out the rest
+ // of the backcompat block. The total number of bytes must be 256.
+ for (unsigned I = 0; I < 64 - NumInstrs; ++I) {
+ BuildMI(BackCompatMBB, DebugLoc(), TII.get(AMDGPU::S_BRANCH))
+ .addMBB(&*KernelEntryMBB);
+ }
+}
+
+// Find the largest possible load size that fits with SGRP alignment
+static LoadConfig getLoadParameters(const TargetRegisterInfo &TRI,
+ Register KernargPreloadSGPR,
+ unsigned NumKernargPreloadSGPRs) {
+ static const LoadConfig Configs[] = {
+ {8, &AMDGPU::SReg_256RegClass, AMDGPU::S_LOAD_DWORDX8_IMM},
+ {4, &AMDGPU::SReg_128RegClass, AMDGPU::S_LOAD_DWORDX4_IMM},
+ {2, &AMDGPU::SReg_64RegClass, AMDGPU::S_LOAD_DWORDX2_IMM},
+ {1, &AMDGPU::SReg_32RegClass, AMDGPU::S_LOAD_DWORD_IMM}};
+
+ // Find the largest possible load size
+ for (const auto &Config : Configs) {
+ if (NumKernargPreloadSGPRs >= Config.Size) {
+ Register LoadReg = TRI.getMatchingSuperReg(KernargPreloadSGPR,
+ AMDGPU::sub0, Config.RegClass);
+ if (LoadReg != AMDGPU::NoRegister)
+ return LoadConfig(Config.Size, Config.RegClass, Config.Opcode, LoadReg);
+ }
+ }
+
+ // Fallback to a single register
+ return LoadConfig(1, &AMDGPU::SReg_32RegClass, AMDGPU::S_LOAD_DWORD_IMM,
+ KernargPreloadSGPR);
+}
+
+unsigned AMDGPUPreloadKernargHeader::addBackCompatLoads(
+ MachineBasicBlock *BackCompatMBB, Register KernargSegmentPtr,
+ unsigned NumKernargPreloadSGPRs) {
+ Register KernargPreloadSGPR = MFI.getArgInfo().FirstKernArgPreloadReg;
+ unsigned Offset = 0;
+ unsigned NumLoads = 0;
+
+ // Fill all user SGPRs used for kernarg preloading with sequential data from
+ // the kernarg segment
+ while (NumKernargPreloadSGPRs > 0) {
+ LoadConfig Config =
+ getLoadParameters(TRI, KernargPreloadSGPR, NumKernargPreloadSGPRs);
+
+ BuildMI(BackCompatMBB, DebugLoc(), TII.get(Config.Opcode), Config.LoadReg)
+ .addReg(KernargSegmentPtr)
+ .addImm(Offset)
+ .addImm(0);
+
+ Offset += 4 * Config.Size;
+ KernargPreloadSGPR = KernargPreloadSGPR.asMCReg() + Config.Size;
+ NumKernargPreloadSGPRs -= Config.Size;
+ NumLoads++;
+ }
+
+ return NumLoads;
+}
+
+PreservedAnalyses
+AMDGPUPreloadKernargHeaderPass::run(MachineFunction &MF,
+ MachineFunctionAnalysisManager &) {
+ if (!AMDGPUPreloadKernargHeader(MF).run())
+ return PreservedAnalyses::all();
+
+ return PreservedAnalyses::none();
+}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.h b/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.h
new file mode 100644
index 00000000000000..b0a3d065d562e0
--- /dev/null
+++ b/llvm/lib/Target/AMDGPU/AMDGPUPreloadKernargHeader.h
@@ -0,0 +1,25 @@
+//===- AMDGPUPreloadKernargHeader.h ----------------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_LIB_TARGET_AMDGPU_PRELOAD_KERNARG_HEADER_H
+#define LLVM_LIB_TARGET_AMDGPU_PRELOAD_KERNARG_HEADER_H
+
+#include "llvm/CodeGen/MachinePassManager.h"
+
+namespace llvm {
+
+class AMDGPUPreloadKernargHeaderPass
+ : public PassInfoMixin<AMDGPUPreloadKernargHeaderPass> {
+public:
+ PreservedAnalyses run(MachineFunction &MF,
+ MachineFunctionAnalysisManager &AM);
+};
+
+} // end namespace llvm
+
+#endif // LLVM_LIB_TARGET_AMDGPU_PRELOAD_KERNARG_HEADER_H
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp b/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
index 34ad99dd980f27..854f5898303951 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUTargetMachine.cpp
@@ -535,6 +535,7 @@ extern "C" LLVM_EXTERNAL_VISIBILITY void LLVMInitializeAMDGPUTarget() {
initializeGCNPreRALongBranchRegPass(*PR);
initializeGCNRewritePartialRegUsesPass(*PR);
initializeGCNRegPressurePrinterPass(*PR);
+ initializeAMDGPUPreloadKernargHeaderLegacyPass(*PR);
}
static std::unique_ptr<TargetLoweringObjectFile> createTLOF(const Triple &TT) {
@@ -1658,6 +1659,7 @@ void GCNPassConfig::addPreEmitPass() {
addPass(&AMDGPUInsertDelayAluID);
addPass(&BranchRelaxationPassID);
+ addPass(createAMDGPUPreloadKernargHeaderLegacyPass());
}
TargetPassConfig *GCNTargetMachine::createPassConfig(PassManagerBase &PM) {
diff --git a/llvm/lib/Target/AMDGPU/CMakeLists.txt b/llvm/lib/Target/AMDGPU/CMakeLists.txt
index 68d141e338a882..87112ef4729a8a 100644
--- a/llvm/lib/Target/AMDGPU/CMakeLists.txt
+++ b/llvm/lib/Target/AMDGPU/CMakeLists.txt
@@ -88,6 +88,7 @@ add_llvm_target(AMDGPUCodeGen
AMDGPUPerfHintAnalysis.cpp
AMDGPUPostLegalizerCombiner.cpp
AMDGPUPreLegalizerCombiner.cpp
+ AMDGPUPreloadKernargHeader.cpp
AMDGPUPrintfRuntimeBinding.cpp
AMDGPUPromoteAlloca.cpp
AMDGPUPromoteKernelArguments.cpp
diff --git a/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp b/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp
index ffde4d33f1341a..eccd77d6c00f0b 100644
--- a/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp
+++ b/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.cpp
@@ -338,15 +338,6 @@ bool AMDGPUTargetAsmStreamer::EmitHSAMetadata(
return true;
}
-bool AMDGPUTargetAsmStreamer::EmitKernargPreloadHeader(
- const MCSubtargetInfo &STI, bool TrapEnabled) {
- OS << (TrapEnabled ? "\ts_trap 2" : "\ts_endpgm")
- << " ; Kernarg preload header. Trap with incompatible firmware that "
- "doesn't support preloading kernel arguments.\n";
- OS << "\t.fill 63, 4, 0xbf800000 ; s_nop 0\n";
- return true;
-}
-
bool AMDGPUTargetAsmStreamer::EmitCodeEnd(const MCSubtargetInfo &STI) {
const uint32_t Encoded_s_code_end = 0xbf9f0000;
const uint32_t Encoded_s_nop = 0xbf800000;
@@ -935,20 +926,6 @@ bool AMDGPUTargetELFStreamer::EmitHSAMetadata(msgpack::Document &HSAMetadataDoc,
return true;
}
-bool AMDGPUTargetELFStreamer::EmitKernargPreloadHeader(
- const MCSubtargetInfo &STI, bool TrapEnabled) {
- const uint32_t Encoded_s_nop = 0xbf800000;
- const uint32_t Encoded_s_trap = 0xbf920002;
- const uint32_t Encoded_s_endpgm = 0xbf810000;
- const uint32_t TrapInstr = TrapEnabled ? Encoded_s_trap : Encoded_s_endpgm;
- MCStreamer &OS = getStreamer();
- OS.emitInt32(TrapInstr);
- for (int i = 0; i < 63; ++i) {
- OS.emitInt32(Encoded_s_nop);
- }
- return true;
-}
-
bool AMDGPUTargetELFStreamer::EmitCodeEnd(const MCSubtargetInfo &STI) {
const uint32_t Encoded_s_code_end = 0xbf9f0000;
const uint32_t Encoded_s_nop = 0xbf800000;
diff --git a/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h b/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h
index 6a91ad06de5d12..9c490208505846 100644
--- a/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h
+++ b/llvm/lib/Target/AMDGPU/MCTargetDesc/AMDGPUTargetStreamer.h
@@ -96,12 +96,6 @@ class AMDGPUTargetStreamer : public MCTargetStreamer {
/// \returns True on success, false on failure.
virtual bool EmitCodeEnd(const MCSubtargetInfo &STI) { return true; }
- /// \returns True on success, false on failure.
- virtual bool EmitKernargPreloadHeader(const MCSubtargetInfo &STI,
- bool TrapEnabled) {
- return true;
- }
-
virtual void
EmitAmdhsaKernelDescriptor(const MCSubtargetInfo &STI, StringRef KernelName,
const AMDGPU::MCKernelDescriptor &KernelDescriptor,
@@ -168,10 +162,6 @@ class AMDGPUTargetAsmStreamer final : public AMDGPUTargetStreamer {
/// \returns True on success, false on failure.
bool EmitCodeEnd(const MCSubtargetInfo &STI) override;
- /// \returns True on success, false on failure.
- bool EmitKernargPreloadHeader(const MCSubtargetInfo &STI,
- bool TrapEnabled) override;
-
void
EmitAmdhsaKernelDescriptor(const MCSubtargetInfo &STI, StringRef KernelName,
const AMDGPU::MCKernelDescriptor &KernelDescriptor,
@@ -225,10 +215,6 @@ class AMDGPUTargetELFStreamer final : public AMDGPUTargetStreamer {
/// \returns True on success, false on failure.
bool EmitCodeEnd(const MCSubtargetInfo &STI) override;
- /// \returns True on success, false on failure.
- bool EmitKernargPreloadHeader(const MCSubtargetInfo &STI,
- bool TrapEnabled) override;
-
void
EmitAmdhsaKernelDescriptor(const MCSubtargetInfo &STI, StringRef KernelName,
const AMDGPU::MCKernelDescriptor &KernelDescriptor,
diff --git a/llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp b/llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp
index 1e43d2727a00da..a3402668237a58 100644
--- a/llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIMachineFunctionInfo.cpp
@@ -259,6 +259,8 @@ SmallVectorImpl<MCRegister> *SIMachineFunctionInfo::addPreloadedKernArg(
// If the available register tuples are aligned with the kernarg to be
// preloaded use that register, otherwise we need to use a set of SGPRs and
// merge them.
+ if (ArgInfo.FirstKernArgPreloadReg == AMDGPU::NoRegister)
+ ArgInfo.FirstKernArgPreloadReg = getNextUserSGPR();
Register PreloadReg =
TRI.getMatchingSuperReg(getNextUserSGPR(), AMDGPU::sub0, RC);
if (PreloadReg &&
diff --git a/llvm/test/CodeGen/AMDGPU/llc-pipeline.ll b/llvm/test/CodeGen/AMDGPU/llc-pipeline.ll
index e77f4f69e265bb..950967aebd44a8 100644
--- a/llvm/test/CodeGen/AMDGPU/llc-pipeline.ll
+++ b/llvm/test/CodeGen/AMDGPU/llc-pipeline.ll
@@ -144,6 +144,7 @@
; GCN-O0-NEXT: SI Final Branch Preparation
; GCN-O0-NEXT: Post RA hazard recognizer
; GCN-O0-NEXT: Branch relaxation pass
+; GCN-O0-NEXT: AMDGPU Preload Kernarg Header
; GCN-O0-NEXT: Register Usage Information Collector Pass
; GCN-O0-NEXT: Remove Loads Into Fake Uses
; GCN-O0-NEXT: Live DEBUG_VALUE analysis
@@ -424,6 +425,7 @@
; GCN-O1-NEXT: Post RA hazard recognizer
; GCN-O1-NEXT: AMDGPU Insert Delay ALU
; GCN-O1-NEXT: Branch relaxation pass
+; GCN-O1-NEXT: AMDGPU Preload Kernarg Header
; GCN-O1-NEXT: Register Usage Information Collector Pass
; GCN-O1-NEXT: Remove Loads Into Fake Uses
; GCN-O1-NEXT: Live DEBUG_VALUE analysis
@@ -732,6 +734,7 @@
; GCN-O1-OPTS-NEXT: Post RA hazard recognizer
; GCN-O1-OPTS-NEXT: AMDGPU Insert Delay ALU
; GCN-O1-OPTS-NEXT: Branch relaxation pass
+; GCN-O1-OPTS-NEXT: AMDGPU Preload Kernarg Header
; GCN-O1-OPTS-NEXT: Register Usage Information Collector Pass
; GCN-O1-OPTS-NEXT: Remove Loads Into Fake Uses
; GCN-O1-OPTS-NEXT: Live DEBUG_VALUE analysis
@@ -1046,6 +1049,7 @@
; GCN-O2-NEXT: Post RA hazard recognizer
; GCN-O2-NEXT: AMDGPU Insert Delay ALU
; GCN-O2-NEXT: Branch relaxation pass
+; GCN-O2-NEXT: AMDGPU Preload Kernarg Header
; GCN-O2-NEXT: Register Usage Information Collector Pass
; GCN-O2-NEXT: Remove Loads Into Fake Uses
; GCN-O2-NEXT: Live DEBUG_VALUE analysis
@@ -1373,6 +1377,7 @@
; GCN-O3-NEXT: Post RA hazard recognizer
; GCN-O3-NEXT: AMDGPU Insert Delay ALU
; GCN-O3-NEXT: Branch relaxation pass
+; GCN-O3-NEXT: AMDGPU Preload Kernarg Header
; GCN-O3-NEXT: Register Usage Information Collector Pass
; GCN-O3-NEXT: Remove Loads Into Fake Uses
; GCN-O3-NEXT: Live DEBUG_VALUE analysis
diff --git a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
index 0c6d8dce193da6..fe4d1e67c0fe71 100644
--- a/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
+++ b/llvm/test/CodeGen/AMDGPU/preload-implicit-kernargs.ll
@@ -4,18 +4,144 @@
define amdgpu_kernel void @preload_block_count_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_block_count_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: .LBB0_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_block_count_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: .LBB0_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -28,18 +154,144 @@ define amdgpu_kernel void @preload_block_count_x(ptr addrspace(1) inreg %out) #0
define amdgpu_kernel void @preload_unused_arg_block_count_x(ptr addrspace(1) inreg %out, i32 inreg) #0 {
; GFX940-LABEL: preload_unused_arg_block_count_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s6, s[0:1], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: .LBB1_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s6
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_unused_arg_block_count_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s10, s[4:5], 0x10
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: .LBB1_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s10
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -52,9 +304,72 @@ define amdgpu_kernel void @preload_unused_arg_block_count_x(ptr addrspace(1) inr
define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) inreg %out, i256 inreg) {
; GFX940-LABEL: no_free_sgprs_block_count_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: .LBB2_0:
; GFX940-NEXT: s_load_dword s0, s[4:5], 0x28
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -63,9 +378,72 @@ define amdgpu_kernel void @no_free_sgprs_block_count_x(ptr addrspace(1) inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: no_free_sgprs_block_count_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[8:9], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: .LBB2_0:
; GFX90a-NEXT: s_load_dword s0, s[8:9], 0x28
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -135,9 +513,72 @@ define amdgpu_kernel void @mixed_inreg_block_count_x(ptr addrspace(1) %out, i32
define amdgpu_kernel void @incorrect_type_i64_block_count_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: incorrect_type_i64_block_count_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: .LBB5_0:
; GFX940-NEXT: s_load_dwordx2 s[0:1], s[0:1], 0x8
; GFX940-NEXT: v_mov_b32_e32 v2, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -146,9 +587,72 @@ define amdgpu_kernel void @incorrect_type_i64_block_count_x(ptr addrspace(1) inr
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: incorrect_type_i64_block_count_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: .LBB5_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x8
; GFX90a-NEXT: v_mov_b32_e32 v2, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -163,9 +667,72 @@ define amdgpu_kernel void @incorrect_type_i64_block_count_x(ptr addrspace(1) inr
define amdgpu_kernel void @incorrect_type_i16_block_count_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: incorrect_type_i16_block_count_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: .LBB6_0:
; GFX940-NEXT: s_load_dword s0, s[0:1], 0x8
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -174,9 +741,72 @@ define amdgpu_kernel void @incorrect_type_i16_block_count_x(ptr addrspace(1) inr
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: incorrect_type_i16_block_count_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: .LBB6_0:
; GFX90a-NEXT: s_load_dword s0, s[4:5], 0x8
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -191,18 +821,144 @@ define amdgpu_kernel void @incorrect_type_i16_block_count_x(ptr addrspace(1) inr
define amdgpu_kernel void @preload_block_count_y(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_block_count_y:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: .LBB7_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s5
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_block_count_y:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: .LBB7_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s9
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -216,9 +972,72 @@ define amdgpu_kernel void @preload_block_count_y(ptr addrspace(1) inreg %out) #0
define amdgpu_kernel void @random_incorrect_offset(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: random_incorrect_offset:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: .LBB8_0:
; GFX940-NEXT: s_mov_b32 s4, 8
; GFX940-NEXT: s_load_dword s0, s[0:1], s4 offset:0x2
; GFX940-NEXT: v_mov_b32_e32 v0, 0
@@ -228,9 +1047,72 @@ define amdgpu_kernel void @random_incorrect_offset(ptr addrspace(1) inreg %out)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: random_incorrect_offset:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: .LBB8_0:
; GFX90a-NEXT: s_mov_b32 s0, 8
; GFX90a-NEXT: s_load_dword s0, s[4:5], s0 offset:0x2
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
@@ -247,18 +1129,144 @@ define amdgpu_kernel void @random_incorrect_offset(ptr addrspace(1) inreg %out)
define amdgpu_kernel void @preload_block_count_z(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_block_count_z:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s6, s[0:1], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: .LBB9_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s6
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_block_count_z:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s10, s[4:5], 0x10
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: .LBB9_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s10
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -272,9 +1280,72 @@ define amdgpu_kernel void @preload_block_count_z(ptr addrspace(1) inreg %out) #0
define amdgpu_kernel void @preload_block_count_x_imparg_align_ptr_i8(ptr addrspace(1) inreg %out, i8 inreg %val) #0 {
; GFX940-LABEL: preload_block_count_x_imparg_align_ptr_i8:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s6, s[0:1], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: .LBB10_0:
; GFX940-NEXT: s_and_b32 s0, s4, 0xff
; GFX940-NEXT: s_add_i32 s0, s6, s0
; GFX940-NEXT: v_mov_b32_e32 v0, 0
@@ -283,9 +1354,72 @@ define amdgpu_kernel void @preload_block_count_x_imparg_align_ptr_i8(ptr addrspa
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_block_count_x_imparg_align_ptr_i8:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s10, s[4:5], 0x10
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: .LBB10_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 0xff
; GFX90a-NEXT: s_add_i32 s0, s10, s0
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
@@ -302,9 +1436,72 @@ define amdgpu_kernel void @preload_block_count_x_imparg_align_ptr_i8(ptr addrspa
define amdgpu_kernel void @preload_block_count_xyz(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_block_count_xyz:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s6, s[0:1], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: .LBB11_0:
; GFX940-NEXT: v_mov_b32_e32 v3, 0
; GFX940-NEXT: v_mov_b32_e32 v0, s4
; GFX940-NEXT: v_mov_b32_e32 v1, s5
@@ -313,9 +1510,72 @@ define amdgpu_kernel void @preload_block_count_xyz(ptr addrspace(1) inreg %out)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_block_count_xyz:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s10, s[4:5], 0x10
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: .LBB11_0:
; GFX90a-NEXT: v_mov_b32_e32 v3, 0
; GFX90a-NEXT: v_mov_b32_e32 v0, s8
; GFX90a-NEXT: v_mov_b32_e32 v1, s9
@@ -338,9 +1598,72 @@ define amdgpu_kernel void @preload_block_count_xyz(ptr addrspace(1) inreg %out)
define amdgpu_kernel void @preload_workgroup_size_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_workgroup_size_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: .LBB12_0:
; GFX940-NEXT: s_and_b32 s0, s7, 0xffff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -348,9 +1671,72 @@ define amdgpu_kernel void @preload_workgroup_size_x(ptr addrspace(1) inreg %out)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_workgroup_size_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: .LBB12_0:
; GFX90a-NEXT: s_and_b32 s0, s11, 0xffff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -366,9 +1752,72 @@ define amdgpu_kernel void @preload_workgroup_size_x(ptr addrspace(1) inreg %out)
define amdgpu_kernel void @preload_workgroup_size_y(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_workgroup_size_y:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: .LBB13_0:
; GFX940-NEXT: s_lshr_b32 s0, s7, 16
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -376,9 +1825,72 @@ define amdgpu_kernel void @preload_workgroup_size_y(ptr addrspace(1) inreg %out)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_workgroup_size_y:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: .LBB13_0:
; GFX90a-NEXT: s_lshr_b32 s0, s11, 16
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -394,9 +1906,72 @@ define amdgpu_kernel void @preload_workgroup_size_y(ptr addrspace(1) inreg %out)
define amdgpu_kernel void @preload_workgroup_size_z(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_workgroup_size_z:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s8, s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: .LBB14_0:
; GFX940-NEXT: s_and_b32 s0, s8, 0xffff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -404,9 +1979,72 @@ define amdgpu_kernel void @preload_workgroup_size_z(ptr addrspace(1) inreg %out)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_workgroup_size_z:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s12, s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: .LBB14_0:
; GFX90a-NEXT: s_and_b32 s0, s12, 0xffff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -422,9 +2060,72 @@ define amdgpu_kernel void @preload_workgroup_size_z(ptr addrspace(1) inreg %out)
define amdgpu_kernel void @preload_workgroup_size_xyz(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_workgroup_size_xyz:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s8, s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: .LBB15_0:
; GFX940-NEXT: s_lshr_b32 s0, s7, 16
; GFX940-NEXT: s_and_b32 s1, s7, 0xffff
; GFX940-NEXT: s_and_b32 s4, s8, 0xffff
@@ -436,9 +2137,72 @@ define amdgpu_kernel void @preload_workgroup_size_xyz(ptr addrspace(1) inreg %ou
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_workgroup_size_xyz:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s12, s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: .LBB15_0:
; GFX90a-NEXT: s_lshr_b32 s0, s11, 16
; GFX90a-NEXT: s_and_b32 s1, s11, 0xffff
; GFX90a-NEXT: s_and_b32 s2, s12, 0xffff
@@ -467,9 +2231,72 @@ define amdgpu_kernel void @preload_workgroup_size_xyz(ptr addrspace(1) inreg %ou
define amdgpu_kernel void @preload_remainder_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preload_remainder_x:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s8, s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: .LBB16_0:
; GFX940-NEXT: s_lshr_b32 s0, s8, 16
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -477,9 +2304,72 @@ define amdgpu_kernel void @preload_remainder_x(ptr addrspace(1) inreg %out) #0 {
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preload_remainder_x:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s12, s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: .LBB16_0:
; GFX90a-NEXT: s_lshr_b32 s0, s12, 16
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -495,9 +2385,72 @@ define amdgpu_kernel void @preload_remainder_x(ptr addrspace(1) inreg %out) #0 {
define amdgpu_kernel void @preloadremainder_y(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preloadremainder_y:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: .LBB17_0:
; GFX940-NEXT: s_and_b32 s0, s9, 0xffff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -505,9 +2458,72 @@ define amdgpu_kernel void @preloadremainder_y(ptr addrspace(1) inreg %out) #0 {
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preloadremainder_y:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: .LBB17_0:
; GFX90a-NEXT: s_and_b32 s0, s13, 0xffff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -523,9 +2539,72 @@ define amdgpu_kernel void @preloadremainder_y(ptr addrspace(1) inreg %out) #0 {
define amdgpu_kernel void @preloadremainder_z(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preloadremainder_z:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: .LBB18_0:
; GFX940-NEXT: s_lshr_b32 s0, s9, 16
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -533,9 +2612,72 @@ define amdgpu_kernel void @preloadremainder_z(ptr addrspace(1) inreg %out) #0 {
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preloadremainder_z:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: .LBB18_0:
; GFX90a-NEXT: s_lshr_b32 s0, s13, 16
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -551,9 +2693,72 @@ define amdgpu_kernel void @preloadremainder_z(ptr addrspace(1) inreg %out) #0 {
define amdgpu_kernel void @preloadremainder_xyz(ptr addrspace(1) inreg %out) #0 {
; GFX940-LABEL: preloadremainder_xyz:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: .LBB19_0:
; GFX940-NEXT: s_lshr_b32 s0, s9, 16
; GFX940-NEXT: s_lshr_b32 s1, s8, 16
; GFX940-NEXT: s_and_b32 s4, s9, 0xffff
@@ -565,9 +2770,72 @@ define amdgpu_kernel void @preloadremainder_xyz(ptr addrspace(1) inreg %out) #0
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: preloadremainder_xyz:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: .LBB19_0:
; GFX90a-NEXT: s_lshr_b32 s0, s13, 16
; GFX90a-NEXT: s_lshr_b32 s1, s12, 16
; GFX90a-NEXT: s_and_b32 s2, s13, 0xffff
@@ -596,19 +2864,145 @@ define amdgpu_kernel void @preloadremainder_xyz(ptr addrspace(1) inreg %out) #0
define amdgpu_kernel void @no_free_sgprs_preloadremainder_z(ptr addrspace(1) inreg %out) {
; GFX940-LABEL: no_free_sgprs_preloadremainder_z:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
-; GFX940-NEXT: s_lshr_b32 s0, s15, 16
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x10
+; GFX940-NEXT: s_load_dword s14, s[4:5], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: .LBB20_0:
+; GFX940-NEXT: s_load_dword s0, s[4:5], 0x1c
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
; GFX940-NEXT: global_store_dword v0, v1, s[8:9] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: no_free_sgprs_preloadremainder_z:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[8:9], 0x0
+; GFX90a-NEXT: s_load_dword s14, s[8:9], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: .LBB20_0:
; GFX90a-NEXT: s_load_dword s0, s[8:9], 0x1c
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
diff --git a/llvm/test/CodeGen/AMDGPU/preload-kernarg-header.ll b/llvm/test/CodeGen/AMDGPU/preload-kernarg-header.ll
deleted file mode 100644
index 5a03381447d0eb..00000000000000
--- a/llvm/test/CodeGen/AMDGPU/preload-kernarg-header.ll
+++ /dev/null
@@ -1,23 +0,0 @@
-; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx940 -amdgpu-kernarg-preload-count=1 -asm-verbose=0 < %s | FileCheck -check-prefixes=GCN,HSA,ASM %s
-; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx940 -amdgpu-kernarg-preload-count=1 -filetype=obj < %s | llvm-objdump --arch=amdgcn --mcpu=gfx940 --disassemble - | FileCheck -check-prefixes=GCN,HSA,OBJ %s
-; RUN: llc -mtriple=amdgcn -mcpu=gfx940 -amdgpu-kernarg-preload-count=1 -filetype=obj < %s | llvm-objdump --arch=amdgcn --mcpu=gfx940 --disassemble - | FileCheck -check-prefixes=GCN,NON-HSA,OBJ %s
-; RUN: llc -mtriple=amdgcn-amd-amdhsa -mcpu=gfx940 -amdgpu-kernarg-preload-count=1 -asm-verbose=0 < %s | llvm-mc -triple amdgcn-amd-amdhsa -mcpu=gfx940 -filetype=obj | llvm-objdump --arch=amdgcn --mcpu=gfx940 --disassemble - | FileCheck -check-prefixes=GCN,HSA,OBJ %s
-
-; GCN: preload_kernarg_header
-; HSA: s_trap 2
-; NON-HSA: s_endpgm
-; ASM: .fill 63, 4, 0xbf800000 ; s_nop 0
-; OBJ-COUNT-63: s_nop 0
-define amdgpu_kernel void @preload_kernarg_header(ptr inreg %arg) {
- store ptr %arg, ptr %arg
- ret void
-}
-
-; GCN: non_kernel_function
-; GCN-NOT: s_trap 2
-; GCN-NOT: s_nop 0
-; GCN: flat_store
-define void @non_kernel_function(ptr %arg) {
- store ptr %arg, ptr %arg
- ret void
-}
diff --git a/llvm/test/CodeGen/AMDGPU/preload-kernargs.ll b/llvm/test/CodeGen/AMDGPU/preload-kernargs.ll
index 2aaffd7121ae97..08c278d771a625 100644
--- a/llvm/test/CodeGen/AMDGPU/preload-kernargs.ll
+++ b/llvm/test/CodeGen/AMDGPU/preload-kernargs.ll
@@ -5,9 +5,72 @@
define amdgpu_kernel void @ptr1_i8(ptr addrspace(1) inreg %out, i8 inreg %arg0) #0 {
; GFX940-LABEL: ptr1_i8:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: s_branch .LBB0_0
+; GFX940-NEXT: .LBB0_0:
; GFX940-NEXT: s_and_b32 s0, s4, 0xff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -15,9 +78,72 @@ define amdgpu_kernel void @ptr1_i8(ptr addrspace(1) inreg %out, i8 inreg %arg0)
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i8:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: s_branch .LBB0_0
+; GFX90a-NEXT: .LBB0_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 0xff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -30,9 +156,72 @@ define amdgpu_kernel void @ptr1_i8(ptr addrspace(1) inreg %out, i8 inreg %arg0)
define amdgpu_kernel void @ptr1_i8_zext_arg(ptr addrspace(1) inreg %out, i8 zeroext inreg %arg0) #0 {
; GFX940-LABEL: ptr1_i8_zext_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: s_branch .LBB1_0
+; GFX940-NEXT: .LBB1_0:
; GFX940-NEXT: s_and_b32 s0, s4, 0xff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -40,9 +229,72 @@ define amdgpu_kernel void @ptr1_i8_zext_arg(ptr addrspace(1) inreg %out, i8 zero
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i8_zext_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: s_branch .LBB1_0
+; GFX90a-NEXT: .LBB1_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 0xff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -55,9 +307,72 @@ define amdgpu_kernel void @ptr1_i8_zext_arg(ptr addrspace(1) inreg %out, i8 zero
define amdgpu_kernel void @ptr1_i16_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %arg0) #0 {
; GFX940-LABEL: ptr1_i16_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: s_branch .LBB2_0
+; GFX940-NEXT: .LBB2_0:
; GFX940-NEXT: s_and_b32 s0, s4, 0xffff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -65,9 +380,72 @@ define amdgpu_kernel void @ptr1_i16_preload_arg(ptr addrspace(1) inreg %out, i16
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i16_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: s_branch .LBB2_0
+; GFX90a-NEXT: .LBB2_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 0xffff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -80,18 +458,144 @@ define amdgpu_kernel void @ptr1_i16_preload_arg(ptr addrspace(1) inreg %out, i16
define amdgpu_kernel void @ptr1_i32_preload_arg(ptr addrspace(1) inreg %out, i32 inreg %arg0) #0 {
; GFX940-LABEL: ptr1_i32_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: s_branch .LBB3_0
+; GFX940-NEXT: .LBB3_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i32_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: s_branch .LBB3_0
+; GFX90a-NEXT: .LBB3_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -103,9 +607,72 @@ define amdgpu_kernel void @ptr1_i32_preload_arg(ptr addrspace(1) inreg %out, i32
define amdgpu_kernel void @i32_ptr1_i32_preload_arg(i32 inreg %arg0, ptr addrspace(1) inreg %out, i32 inreg %arg1) #0 {
; GFX940-LABEL: i32_ptr1_i32_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_load_dword s6, s[0:1], 0x10
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: s_branch .LBB4_0
+; GFX940-NEXT: .LBB4_0:
; GFX940-NEXT: s_add_i32 s0, s2, s6
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -113,9 +680,72 @@ define amdgpu_kernel void @i32_ptr1_i32_preload_arg(i32 inreg %arg0, ptr addrspa
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i32_ptr1_i32_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dword s10, s[4:5], 0x10
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: s_branch .LBB4_0
+; GFX90a-NEXT: .LBB4_0:
; GFX90a-NEXT: s_add_i32 s0, s6, s10
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -128,9 +758,72 @@ define amdgpu_kernel void @i32_ptr1_i32_preload_arg(i32 inreg %arg0, ptr addrspa
define amdgpu_kernel void @ptr1_i16_i16_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %arg0, i16 inreg %arg1) #0 {
; GFX940-LABEL: ptr1_i16_i16_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: s_branch .LBB5_0
+; GFX940-NEXT: .LBB5_0:
; GFX940-NEXT: s_lshr_b32 s0, s4, 16
; GFX940-NEXT: s_and_b32 s1, s4, 0xffff
; GFX940-NEXT: s_add_i32 s0, s1, s0
@@ -140,9 +833,72 @@ define amdgpu_kernel void @ptr1_i16_i16_preload_arg(ptr addrspace(1) inreg %out,
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i16_i16_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: s_branch .LBB5_0
+; GFX90a-NEXT: .LBB5_0:
; GFX90a-NEXT: s_lshr_b32 s0, s8, 16
; GFX90a-NEXT: s_and_b32 s1, s8, 0xffff
; GFX90a-NEXT: s_add_i32 s0, s1, s0
@@ -159,18 +915,144 @@ define amdgpu_kernel void @ptr1_i16_i16_preload_arg(ptr addrspace(1) inreg %out,
define amdgpu_kernel void @ptr1_v2i8_preload_arg(ptr addrspace(1) inreg %out, <2 x i8> inreg %in) #0 {
; GFX940-LABEL: ptr1_v2i8_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: s_branch .LBB6_0
+; GFX940-NEXT: .LBB6_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_short v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_v2i8_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: s_branch .LBB6_0
+; GFX90a-NEXT: .LBB6_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_short v0, v1, s[6:7]
@@ -182,9 +1064,72 @@ define amdgpu_kernel void @ptr1_v2i8_preload_arg(ptr addrspace(1) inreg %out, <2
define amdgpu_kernel void @byref_preload_arg(ptr addrspace(1) inreg %out, ptr addrspace(4) byref(i32) align(256) %in.byref, i32 %after.offset) #0 {
; GFX940-LABEL: byref_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: s_branch .LBB7_0
+; GFX940-NEXT: .LBB7_0:
; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x100
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -197,9 +1142,72 @@ define amdgpu_kernel void @byref_preload_arg(ptr addrspace(1) inreg %out, ptr ad
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: byref_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: s_branch .LBB7_0
+; GFX90a-NEXT: .LBB7_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x100
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -220,9 +1228,72 @@ define amdgpu_kernel void @byref_preload_arg(ptr addrspace(1) inreg %out, ptr ad
define amdgpu_kernel void @byref_staggered_preload_arg(ptr addrspace(1) inreg %out, ptr addrspace(4) byref(i32) align(256) %in.byref, i32 inreg %after.offset) #0 {
; GFX940-LABEL: byref_staggered_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: s_branch .LBB8_0
+; GFX940-NEXT: .LBB8_0:
; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x100
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -235,9 +1306,72 @@ define amdgpu_kernel void @byref_staggered_preload_arg(ptr addrspace(1) inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: byref_staggered_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: s_branch .LBB8_0
+; GFX90a-NEXT: .LBB8_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x100
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -257,9 +1391,72 @@ define amdgpu_kernel void @byref_staggered_preload_arg(ptr addrspace(1) inreg %o
define amdgpu_kernel void @v8i32_arg(ptr addrspace(1) nocapture inreg %out, <8 x i32> inreg %in) #0 {
; GFX940-LABEL: v8i32_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: s_branch .LBB9_0
+; GFX940-NEXT: .LBB9_0:
; GFX940-NEXT: s_load_dwordx8 s[4:11], s[0:1], 0x20
; GFX940-NEXT: v_mov_b32_e32 v4, 0
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
@@ -277,9 +1474,72 @@ define amdgpu_kernel void @v8i32_arg(ptr addrspace(1) nocapture inreg %out, <8 x
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v8i32_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: s_branch .LBB9_0
+; GFX90a-NEXT: .LBB9_0:
; GFX90a-NEXT: s_load_dwordx8 s[8:15], s[4:5], 0x20
; GFX90a-NEXT: v_mov_b32_e32 v4, 0
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
@@ -301,9 +1561,72 @@ define amdgpu_kernel void @v8i32_arg(ptr addrspace(1) nocapture inreg %out, <8 x
define amdgpu_kernel void @v3i16_preload_arg(ptr addrspace(1) nocapture inreg %out, <3 x i16> inreg %in) #0 {
; GFX940-LABEL: v3i16_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: s_branch .LBB10_0
+; GFX940-NEXT: .LBB10_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s5
; GFX940-NEXT: global_store_short v0, v1, s[2:3] offset:4 sc0 sc1
@@ -312,9 +1635,72 @@ define amdgpu_kernel void @v3i16_preload_arg(ptr addrspace(1) nocapture inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v3i16_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: s_branch .LBB10_0
+; GFX90a-NEXT: .LBB10_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s9
; GFX90a-NEXT: global_store_short v0, v1, s[6:7] offset:4
@@ -327,9 +1713,72 @@ define amdgpu_kernel void @v3i16_preload_arg(ptr addrspace(1) nocapture inreg %o
define amdgpu_kernel void @v3i32_preload_arg(ptr addrspace(1) nocapture inreg %out, <3 x i32> inreg %in) #0 {
; GFX940-LABEL: v3i32_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: s_branch .LBB11_0
+; GFX940-NEXT: .LBB11_0:
; GFX940-NEXT: v_mov_b32_e32 v0, s6
; GFX940-NEXT: v_mov_b32_e32 v1, s7
; GFX940-NEXT: v_mov_b32_e32 v2, s8
@@ -338,9 +1787,72 @@ define amdgpu_kernel void @v3i32_preload_arg(ptr addrspace(1) nocapture inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v3i32_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: s_branch .LBB11_0
+; GFX90a-NEXT: .LBB11_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, s10
; GFX90a-NEXT: v_mov_b32_e32 v1, s11
; GFX90a-NEXT: v_mov_b32_e32 v2, s12
@@ -353,9 +1865,72 @@ define amdgpu_kernel void @v3i32_preload_arg(ptr addrspace(1) nocapture inreg %o
define amdgpu_kernel void @v3f32_preload_arg(ptr addrspace(1) nocapture inreg %out, <3 x float> inreg %in) #0 {
; GFX940-LABEL: v3f32_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: s_branch .LBB12_0
+; GFX940-NEXT: .LBB12_0:
; GFX940-NEXT: v_mov_b32_e32 v3, 0
; GFX940-NEXT: v_mov_b32_e32 v0, s6
; GFX940-NEXT: v_mov_b32_e32 v1, s7
@@ -364,9 +1939,72 @@ define amdgpu_kernel void @v3f32_preload_arg(ptr addrspace(1) nocapture inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v3f32_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: s_branch .LBB12_0
+; GFX90a-NEXT: .LBB12_0:
; GFX90a-NEXT: v_mov_b32_e32 v3, 0
; GFX90a-NEXT: v_mov_b32_e32 v0, s10
; GFX90a-NEXT: v_mov_b32_e32 v1, s11
@@ -379,9 +2017,72 @@ define amdgpu_kernel void @v3f32_preload_arg(ptr addrspace(1) nocapture inreg %o
define amdgpu_kernel void @v5i8_preload_arg(ptr addrspace(1) nocapture inreg %out, <5 x i8> inreg %in) #0 {
; GFX940-LABEL: v5i8_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: s_branch .LBB13_0
+; GFX940-NEXT: .LBB13_0:
; GFX940-NEXT: s_lshr_b32 s1, s4, 24
; GFX940-NEXT: s_and_b32 s0, s4, 0xffff
; GFX940-NEXT: s_lshl_b32 s1, s1, 8
@@ -397,9 +2098,72 @@ define amdgpu_kernel void @v5i8_preload_arg(ptr addrspace(1) nocapture inreg %ou
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v5i8_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: s_branch .LBB13_0
+; GFX90a-NEXT: .LBB13_0:
; GFX90a-NEXT: s_lshr_b32 s1, s8, 24
; GFX90a-NEXT: s_lshl_b32 s1, s1, 8
; GFX90a-NEXT: s_bfe_u32 s2, s8, 0x80010
@@ -419,9 +2183,72 @@ define amdgpu_kernel void @v5i8_preload_arg(ptr addrspace(1) nocapture inreg %ou
define amdgpu_kernel void @v5f64_arg(ptr addrspace(1) nocapture inreg %out, <5 x double> inreg %in) #0 {
; GFX940-LABEL: v5f64_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: s_branch .LBB14_0
+; GFX940-NEXT: .LBB14_0:
; GFX940-NEXT: s_load_dwordx2 s[12:13], s[0:1], 0x60
; GFX940-NEXT: s_load_dwordx8 s[4:11], s[0:1], 0x40
; GFX940-NEXT: v_mov_b32_e32 v4, 0
@@ -442,9 +2269,72 @@ define amdgpu_kernel void @v5f64_arg(ptr addrspace(1) nocapture inreg %out, <5 x
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v5f64_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: s_branch .LBB14_0
+; GFX90a-NEXT: .LBB14_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x60
; GFX90a-NEXT: s_load_dwordx8 s[8:15], s[4:5], 0x40
; GFX90a-NEXT: v_mov_b32_e32 v4, 0
@@ -469,9 +2359,72 @@ define amdgpu_kernel void @v5f64_arg(ptr addrspace(1) nocapture inreg %out, <5 x
define amdgpu_kernel void @v8i8_preload_arg(ptr addrspace(1) inreg %out, <8 x i8> inreg %in) #0 {
; GFX940-LABEL: v8i8_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: s_branch .LBB15_0
+; GFX940-NEXT: .LBB15_0:
; GFX940-NEXT: s_lshr_b32 s1, s5, 24
; GFX940-NEXT: s_and_b32 s0, s5, 0xffff
; GFX940-NEXT: s_lshl_b32 s1, s1, 8
@@ -493,9 +2446,72 @@ define amdgpu_kernel void @v8i8_preload_arg(ptr addrspace(1) inreg %out, <8 x i8
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v8i8_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: s_branch .LBB15_0
+; GFX90a-NEXT: .LBB15_0:
; GFX90a-NEXT: s_lshr_b32 s1, s9, 24
; GFX90a-NEXT: s_lshl_b32 s1, s1, 8
; GFX90a-NEXT: s_bfe_u32 s2, s9, 0x80010
@@ -521,18 +2537,144 @@ define amdgpu_kernel void @v8i8_preload_arg(ptr addrspace(1) inreg %out, <8 x i8
define amdgpu_kernel void @i64_kernel_preload_arg(ptr addrspace(1) inreg %out, i64 inreg %a) #0 {
; GFX940-LABEL: i64_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: s_branch .LBB16_0
+; GFX940-NEXT: .LBB16_0:
; GFX940-NEXT: v_mov_b32_e32 v2, 0
; GFX940-NEXT: v_mov_b64_e32 v[0:1], s[4:5]
; GFX940-NEXT: global_store_dwordx2 v2, v[0:1], s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i64_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: s_branch .LBB16_0
+; GFX90a-NEXT: .LBB16_0:
; GFX90a-NEXT: v_mov_b32_e32 v2, 0
; GFX90a-NEXT: v_pk_mov_b32 v[0:1], s[8:9], s[8:9] op_sel:[0,1]
; GFX90a-NEXT: global_store_dwordx2 v2, v[0:1], s[6:7]
@@ -543,18 +2685,144 @@ define amdgpu_kernel void @i64_kernel_preload_arg(ptr addrspace(1) inreg %out, i
define amdgpu_kernel void @f64_kernel_preload_arg(ptr addrspace(1) inreg %out, double inreg %in) #0 {
; GFX940-LABEL: f64_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: s_branch .LBB17_0
+; GFX940-NEXT: .LBB17_0:
; GFX940-NEXT: v_mov_b32_e32 v2, 0
; GFX940-NEXT: v_mov_b64_e32 v[0:1], s[4:5]
; GFX940-NEXT: global_store_dwordx2 v2, v[0:1], s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: f64_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: s_branch .LBB17_0
+; GFX90a-NEXT: .LBB17_0:
; GFX90a-NEXT: v_mov_b32_e32 v2, 0
; GFX90a-NEXT: v_pk_mov_b32 v[0:1], s[8:9], s[8:9] op_sel:[0,1]
; GFX90a-NEXT: global_store_dwordx2 v2, v[0:1], s[6:7]
@@ -565,18 +2833,144 @@ define amdgpu_kernel void @f64_kernel_preload_arg(ptr addrspace(1) inreg %out, d
define amdgpu_kernel void @half_kernel_preload_arg(ptr addrspace(1) inreg %out, half inreg %in) #0 {
; GFX940-LABEL: half_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: s_branch .LBB18_0
+; GFX940-NEXT: .LBB18_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_short v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: half_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: s_branch .LBB18_0
+; GFX90a-NEXT: .LBB18_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_short v0, v1, s[6:7]
@@ -587,18 +2981,144 @@ define amdgpu_kernel void @half_kernel_preload_arg(ptr addrspace(1) inreg %out,
define amdgpu_kernel void @bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out, bfloat inreg %in) #0 {
; GFX940-LABEL: bfloat_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: s_branch .LBB19_0
+; GFX940-NEXT: .LBB19_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_short v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: bfloat_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: s_branch .LBB19_0
+; GFX90a-NEXT: .LBB19_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_short v0, v1, s[6:7]
@@ -609,18 +3129,144 @@ define amdgpu_kernel void @bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out
define amdgpu_kernel void @v2bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out, <2 x bfloat> inreg %in) #0 {
; GFX940-LABEL: v2bfloat_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: s_branch .LBB20_0
+; GFX940-NEXT: .LBB20_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_dword v0, v1, s[2:3] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v2bfloat_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: s_branch .LBB20_0
+; GFX90a-NEXT: .LBB20_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_dword v0, v1, s[6:7]
@@ -631,9 +3277,72 @@ define amdgpu_kernel void @v2bfloat_kernel_preload_arg(ptr addrspace(1) inreg %o
define amdgpu_kernel void @v3bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out, <3 x bfloat> inreg %in) #0 {
; GFX940-LABEL: v3bfloat_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: s_branch .LBB21_0
+; GFX940-NEXT: .LBB21_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s5
; GFX940-NEXT: global_store_short v0, v1, s[2:3] offset:4 sc0 sc1
@@ -642,9 +3351,72 @@ define amdgpu_kernel void @v3bfloat_kernel_preload_arg(ptr addrspace(1) inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v3bfloat_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: s_branch .LBB21_0
+; GFX90a-NEXT: .LBB21_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s9
; GFX90a-NEXT: global_store_short v0, v1, s[6:7] offset:4
@@ -657,9 +3429,72 @@ define amdgpu_kernel void @v3bfloat_kernel_preload_arg(ptr addrspace(1) inreg %o
define amdgpu_kernel void @v6bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out, <6 x bfloat> inreg %in) #0 {
; GFX940-LABEL: v6bfloat_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: s_branch .LBB22_0
+; GFX940-NEXT: .LBB22_0:
; GFX940-NEXT: v_mov_b32_e32 v0, s6
; GFX940-NEXT: v_mov_b32_e32 v1, s7
; GFX940-NEXT: v_mov_b32_e32 v2, s8
@@ -668,9 +3503,72 @@ define amdgpu_kernel void @v6bfloat_kernel_preload_arg(ptr addrspace(1) inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v6bfloat_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: s_branch .LBB22_0
+; GFX90a-NEXT: .LBB22_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, s10
; GFX90a-NEXT: v_mov_b32_e32 v1, s11
; GFX90a-NEXT: v_mov_b32_e32 v2, s12
@@ -683,9 +3581,72 @@ define amdgpu_kernel void @v6bfloat_kernel_preload_arg(ptr addrspace(1) inreg %o
define amdgpu_kernel void @half_v7bfloat_kernel_preload_arg(ptr addrspace(1) inreg %out, half inreg %in, <7 x bfloat> inreg %in2, ptr addrspace(1) inreg %out2) #0 {
; GFX940-LABEL: half_v7bfloat_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx8 s[4:11], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: s_branch .LBB23_0
+; GFX940-NEXT: .LBB23_0:
; GFX940-NEXT: v_mov_b32_e32 v3, 0
; GFX940-NEXT: v_mov_b32_e32 v0, s4
; GFX940-NEXT: global_store_short v3, v0, s[2:3] sc0 sc1
@@ -698,9 +3659,72 @@ define amdgpu_kernel void @half_v7bfloat_kernel_preload_arg(ptr addrspace(1) inr
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: half_v7bfloat_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: s_branch .LBB23_0
+; GFX90a-NEXT: .LBB23_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x20
; GFX90a-NEXT: v_mov_b32_e32 v3, 0
; GFX90a-NEXT: v_mov_b32_e32 v0, s8
@@ -720,9 +3744,72 @@ define amdgpu_kernel void @half_v7bfloat_kernel_preload_arg(ptr addrspace(1) inr
define amdgpu_kernel void @i1_kernel_preload_arg(ptr addrspace(1) inreg %out, i1 inreg %in) #0 {
; GFX940-LABEL: i1_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dword s4, s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: s_branch .LBB24_0
+; GFX940-NEXT: .LBB24_0:
; GFX940-NEXT: s_and_b32 s0, s4, 1
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -730,9 +3817,72 @@ define amdgpu_kernel void @i1_kernel_preload_arg(ptr addrspace(1) inreg %out, i1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i1_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dword s8, s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: s_branch .LBB24_0
+; GFX90a-NEXT: .LBB24_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 1
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
@@ -744,9 +3894,72 @@ define amdgpu_kernel void @i1_kernel_preload_arg(ptr addrspace(1) inreg %out, i1
define amdgpu_kernel void @fp128_kernel_preload_arg(ptr addrspace(1) inreg %out, fp128 inreg %in) #0 {
; GFX940-LABEL: fp128_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: s_branch .LBB25_0
+; GFX940-NEXT: .LBB25_0:
; GFX940-NEXT: v_mov_b32_e32 v4, 0
; GFX940-NEXT: v_mov_b32_e32 v0, s6
; GFX940-NEXT: v_mov_b32_e32 v1, s7
@@ -756,9 +3969,72 @@ define amdgpu_kernel void @fp128_kernel_preload_arg(ptr addrspace(1) inreg %out,
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: fp128_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: s_branch .LBB25_0
+; GFX90a-NEXT: .LBB25_0:
; GFX90a-NEXT: v_mov_b32_e32 v4, 0
; GFX90a-NEXT: v_mov_b32_e32 v0, s10
; GFX90a-NEXT: v_mov_b32_e32 v1, s11
@@ -772,9 +4048,72 @@ define amdgpu_kernel void @fp128_kernel_preload_arg(ptr addrspace(1) inreg %out,
define amdgpu_kernel void @v7i8_kernel_preload_arg(ptr addrspace(1) inreg %out, <7 x i8> inreg %in) #0 {
; GFX940-LABEL: v7i8_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: s_branch .LBB26_0
+; GFX940-NEXT: .LBB26_0:
; GFX940-NEXT: s_lshr_b32 s1, s4, 24
; GFX940-NEXT: s_and_b32 s0, s4, 0xffff
; GFX940-NEXT: s_lshl_b32 s1, s1, 8
@@ -791,9 +4130,72 @@ define amdgpu_kernel void @v7i8_kernel_preload_arg(ptr addrspace(1) inreg %out,
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v7i8_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: s_branch .LBB26_0
+; GFX90a-NEXT: .LBB26_0:
; GFX90a-NEXT: s_lshr_b32 s1, s8, 24
; GFX90a-NEXT: s_lshl_b32 s1, s1, 8
; GFX90a-NEXT: s_bfe_u32 s2, s8, 0x80010
@@ -814,9 +4216,72 @@ define amdgpu_kernel void @v7i8_kernel_preload_arg(ptr addrspace(1) inreg %out,
define amdgpu_kernel void @v7half_kernel_preload_arg(ptr addrspace(1) inreg %out, <7 x half> inreg %in) #0 {
; GFX940-LABEL: v7half_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_load_dwordx2 s[8:9], s[0:1], 0x18
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: s_branch .LBB27_0
+; GFX940-NEXT: .LBB27_0:
; GFX940-NEXT: v_mov_b32_e32 v3, 0
; GFX940-NEXT: v_mov_b32_e32 v0, s9
; GFX940-NEXT: global_store_short v3, v0, s[2:3] offset:12 sc0 sc1
@@ -827,9 +4292,72 @@ define amdgpu_kernel void @v7half_kernel_preload_arg(ptr addrspace(1) inreg %out
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: v7half_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: s_branch .LBB27_0
+; GFX90a-NEXT: .LBB27_0:
; GFX90a-NEXT: v_mov_b32_e32 v3, 0
; GFX90a-NEXT: v_mov_b32_e32 v0, s13
; GFX90a-NEXT: global_store_short v3, v0, s[6:7] offset:12
@@ -842,29 +4370,153 @@ define amdgpu_kernel void @v7half_kernel_preload_arg(ptr addrspace(1) inreg %out
ret void
}
-define amdgpu_kernel void @i16_i32_kernel_preload_arg(ptr addrspace(1) %out, i16 inreg %in, i32 inreg %in2, ptr addrspace(1) inreg %out2) #0 {
+define amdgpu_kernel void @i16_i32_kernel_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %in, i32 inreg %in2, ptr addrspace(1) inreg %out2) #0 {
; GFX940-LABEL: i16_i32_kernel_preload_arg:
-; GFX940: ; %bb.0:
-; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x0
-; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x10
-; GFX940-NEXT: v_mov_b32_e32 v0, 0
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
; GFX940-NEXT: s_waitcnt lgkmcnt(0)
-; GFX940-NEXT: v_mov_b32_e32 v1, s6
-; GFX940-NEXT: v_mov_b32_e32 v2, s7
-; GFX940-NEXT: global_store_short v0, v1, s[4:5] sc0 sc1
-; GFX940-NEXT: global_store_dword v0, v2, s[2:3] sc0 sc1
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: s_branch .LBB28_0
+; GFX940-NEXT: .LBB28_0:
+; GFX940-NEXT: v_mov_b32_e32 v0, 0
+; GFX940-NEXT: v_mov_b32_e32 v1, s4
+; GFX940-NEXT: global_store_short v0, v1, s[2:3] sc0 sc1
+; GFX940-NEXT: v_mov_b32_e32 v1, s5
+; GFX940-NEXT: global_store_dword v0, v1, s[6:7] sc0 sc1
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i16_i32_kernel_preload_arg:
-; GFX90a: ; %bb.0:
-; GFX90a-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x0
-; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x10
-; GFX90a-NEXT: v_mov_b32_e32 v0, 0
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
-; GFX90a-NEXT: v_mov_b32_e32 v1, s2
-; GFX90a-NEXT: v_mov_b32_e32 v2, s3
-; GFX90a-NEXT: global_store_short v0, v1, s[0:1]
-; GFX90a-NEXT: global_store_dword v0, v2, s[6:7]
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: s_branch .LBB28_0
+; GFX90a-NEXT: .LBB28_0:
+; GFX90a-NEXT: v_mov_b32_e32 v0, 0
+; GFX90a-NEXT: v_mov_b32_e32 v1, s8
+; GFX90a-NEXT: global_store_short v0, v1, s[6:7]
+; GFX90a-NEXT: v_mov_b32_e32 v1, s9
+; GFX90a-NEXT: global_store_dword v0, v1, s[10:11]
; GFX90a-NEXT: s_endpgm
store i16 %in, ptr addrspace(1) %out
store i32 %in2, ptr addrspace(1) %out2
@@ -873,9 +4525,72 @@ define amdgpu_kernel void @i16_i32_kernel_preload_arg(ptr addrspace(1) %out, i16
define amdgpu_kernel void @i16_v3i32_kernel_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %in, <3 x i32> inreg %in2, ptr addrspace(1) inreg %out2) #0 {
; GFX940-LABEL: i16_v3i32_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx8 s[4:11], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: s_branch .LBB29_0
+; GFX940-NEXT: .LBB29_0:
; GFX940-NEXT: v_mov_b32_e32 v3, 0
; GFX940-NEXT: v_mov_b32_e32 v4, s4
; GFX940-NEXT: v_mov_b32_e32 v0, s6
@@ -886,9 +4601,72 @@ define amdgpu_kernel void @i16_v3i32_kernel_preload_arg(ptr addrspace(1) inreg %
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i16_v3i32_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_load_dwordx2 s[12:13], s[4:5], 0x18
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: s_branch .LBB29_0
+; GFX90a-NEXT: .LBB29_0:
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x20
; GFX90a-NEXT: v_mov_b32_e32 v3, 0
; GFX90a-NEXT: v_mov_b32_e32 v4, s8
@@ -906,9 +4684,72 @@ define amdgpu_kernel void @i16_v3i32_kernel_preload_arg(ptr addrspace(1) inreg %
define amdgpu_kernel void @i16_i16_kernel_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %in, i16 inreg %in2, ptr addrspace(1) inreg %out2) #0 {
; GFX940-LABEL: i16_i16_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: s_branch .LBB30_0
+; GFX940-NEXT: .LBB30_0:
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s4
; GFX940-NEXT: global_store_short v0, v1, s[2:3] sc0 sc1
@@ -916,9 +4757,72 @@ define amdgpu_kernel void @i16_i16_kernel_preload_arg(ptr addrspace(1) inreg %ou
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i16_i16_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: s_branch .LBB30_0
+; GFX90a-NEXT: .LBB30_0:
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s8
; GFX90a-NEXT: global_store_short v0, v1, s[6:7]
@@ -931,9 +4835,72 @@ define amdgpu_kernel void @i16_i16_kernel_preload_arg(ptr addrspace(1) inreg %ou
define amdgpu_kernel void @i16_v2i8_kernel_preload_arg(ptr addrspace(1) inreg %out, i16 inreg %in, <2 x i8> inreg %in2, ptr addrspace(1) inreg %out2) #0 {
; GFX940-LABEL: i16_v2i8_kernel_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx4 s[4:7], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: s_branch .LBB31_0
+; GFX940-NEXT: .LBB31_0:
; GFX940-NEXT: s_lshr_b32 s0, s4, 24
; GFX940-NEXT: s_lshl_b32 s0, s0, 8
; GFX940-NEXT: s_bfe_u32 s1, s4, 0x80010
@@ -946,9 +4913,72 @@ define amdgpu_kernel void @i16_v2i8_kernel_preload_arg(ptr addrspace(1) inreg %o
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i16_v2i8_kernel_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx4 s[8:11], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: s_branch .LBB31_0
+; GFX90a-NEXT: .LBB31_0:
; GFX90a-NEXT: s_lshr_b32 s0, s8, 24
; GFX90a-NEXT: s_lshl_b32 s0, s0, 8
; GFX90a-NEXT: s_bfe_u32 s1, s8, 0x80010
@@ -968,9 +4998,72 @@ define amdgpu_kernel void @i16_v2i8_kernel_preload_arg(ptr addrspace(1) inreg %o
define amdgpu_kernel void @i32_ptr1_i32_staggered_preload_arg(i32 inreg %arg0, ptr addrspace(1) %out, i32 inreg %arg1) #0 {
; GFX940-LABEL: i32_ptr1_i32_staggered_preload_arg:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dword s2, s[0:1], 0x0
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: s_branch .LBB32_0
+; GFX940-NEXT: .LBB32_0:
; GFX940-NEXT: s_load_dword s3, s[0:1], 0x10
; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
; GFX940-NEXT: v_mov_b32_e32 v0, 0
@@ -981,9 +5074,72 @@ define amdgpu_kernel void @i32_ptr1_i32_staggered_preload_arg(i32 inreg %arg0, p
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: i32_ptr1_i32_staggered_preload_arg:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dword s6, s[4:5], 0x0
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: s_branch .LBB32_0
+; GFX90a-NEXT: .LBB32_0:
; GFX90a-NEXT: s_load_dword s2, s[4:5], 0x10
; GFX90a-NEXT: s_load_dwordx2 s[0:1], s[4:5], 0x8
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
@@ -999,9 +5155,72 @@ define amdgpu_kernel void @i32_ptr1_i32_staggered_preload_arg(i32 inreg %arg0, p
define amdgpu_kernel void @ptr1_i8_trailing_unused(ptr addrspace(1) inreg %out, i8 inreg %arg0, i32 inreg %unused) #0 {
; GFX940-LABEL: ptr1_i8_trailing_unused:
-; GFX940: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX940-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX940-NEXT: ; %bb.0:
+; GFX940: ; %bb.1:
+; GFX940-NEXT: s_load_dwordx2 s[2:3], s[0:1], 0x0
+; GFX940-NEXT: s_load_dwordx2 s[4:5], s[0:1], 0x8
+; GFX940-NEXT: s_waitcnt lgkmcnt(0)
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: s_branch .LBB33_0
+; GFX940-NEXT: .LBB33_0:
; GFX940-NEXT: s_and_b32 s0, s4, 0xff
; GFX940-NEXT: v_mov_b32_e32 v0, 0
; GFX940-NEXT: v_mov_b32_e32 v1, s0
@@ -1009,9 +5228,72 @@ define amdgpu_kernel void @ptr1_i8_trailing_unused(ptr addrspace(1) inreg %out,
; GFX940-NEXT: s_endpgm
;
; GFX90a-LABEL: ptr1_i8_trailing_unused:
-; GFX90a: s_trap 2 ; Kernarg preload header. Trap with incompatible firmware that doesn't support preloading kernel arguments.
-; GFX90a-NEXT: .fill 63, 4, 0xbf800000 ; s_nop 0
-; GFX90a-NEXT: ; %bb.0:
+; GFX90a: ; %bb.1:
+; GFX90a-NEXT: s_load_dwordx2 s[6:7], s[4:5], 0x0
+; GFX90a-NEXT: s_load_dwordx2 s[8:9], s[4:5], 0x8
+; GFX90a-NEXT: s_waitcnt lgkmcnt(0)
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: s_branch .LBB33_0
+; GFX90a-NEXT: .LBB33_0:
; GFX90a-NEXT: s_and_b32 s0, s8, 0xff
; GFX90a-NEXT: v_mov_b32_e32 v0, 0
; GFX90a-NEXT: v_mov_b32_e32 v1, s0
More information about the llvm-commits
mailing list