[llvm] [CodeGen][AMDGPU] Replace isBasicBlockPrologue with a MachineInstr::BBProlog flag (PR #225433)
Gabriel Baraldi via llvm-commits
llvm-commits at lists.llvm.org
Fri Sep 25 11:00:09 PDT 2026
https://github.com/gbaraldi updated https://github.com/llvm/llvm-project/pull/225433
>From 320909d182f5f522aeea14e88c204b849a5bf4f5 Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:16 +0000
Subject: [PATCH 1/3] [CodeGen][AMDGPU] Replace isBasicBlockPrologue with a
MachineInstr::BBProlog flag
TargetInstrInfo::isBasicBlockPrologue asked the target to infer, per
instruction and per register being allocated, whether an instruction at
the top of a block belongs to the block prolog. AMDGPU implemented this
by pattern matching everything the register allocator might have
inserted ahead of the exec restore (SGPR spills, WWM spills, implicit
defs, LRSplit copies) plus any EXEC write, and by sending all SGPR
insertions to MBB.begin(). Every new kind of RA-inserted instruction had
to be taught to the check; rematerialized instructions were not, which
let the vgpr-regalloc pipeline place copies and spills above the exec
restore of an if/else Flow block (#222368).
Represent the prolog explicitly instead:
* Add MachineInstr::BBProlog (MIR: bb-prolog). An instruction carrying it
must stay ahead of everything later inserted at the block entry.
* MachineBasicBlock::SkipPHIsAndLabels / SkipPHIsLabelsAndDebug skip
flagged instructions. The Register parameter is gone, as are the
TargetInstrInfo hook and SIInstrInfo::canAddToBBProlog. MachineSink,
RegAllocFast and SIFixSGPRCopies test the flag directly.
* Code that inserts in front of an existing instruction calls
MachineBasicBlock::inheritBBProlog, which marks the new instructions
when the next non-debug instruction is a prolog instruction: spills
(including hoisted ones), reloads, split copies, rematerialization and
memory operand folding in the register allocators, PHI elimination, and
AMDGPU's SGPR spill lowering.
* The MachineVerifier checks that, until register allocation is done,
prolog instructions form a block prefix, only preceded by PHIs, labels,
debug instructions and pseudo probes. A missed insertion site fails
verification.
* AMDGPU sets the flag where it creates prolog instructions at the block
start: SI_END_CF at instruction selection, the exec restores emitted
by SILowerControlFlow, and the exec writes SIWholeQuadMode emits at
block entry.
The sgpr-regalloc pipeline no longer inserts at MBB.begin(); its copies,
remats and reloads now follow the exec restore like everything else.
Scalar instructions do not depend on EXEC, so this only reorders code.
Test updates follow in separate commits.
Fixes #222368
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply at anthropic.com>
---
llvm/include/llvm/CodeGen/MachineBasicBlock.h | 21 ++--
llvm/include/llvm/CodeGen/MachineInstr.h | 4 +-
llvm/include/llvm/CodeGen/TargetInstrInfo.h | 9 --
.../CodeGen/FixupStatepointCallerSaved.cpp | 3 +-
llvm/lib/CodeGen/InlineSpiller.cpp | 6 +-
llvm/lib/CodeGen/LiveRangeEdit.cpp | 2 +
llvm/lib/CodeGen/MIRParser/MILexer.cpp | 1 +
llvm/lib/CodeGen/MIRParser/MILexer.h | 1 +
llvm/lib/CodeGen/MIRParser/MIParser.cpp | 5 +-
llvm/lib/CodeGen/MIRPrinter.cpp | 2 +
llvm/lib/CodeGen/MachineBasicBlock.cpp | 23 ++--
llvm/lib/CodeGen/MachineInstr.cpp | 2 +
llvm/lib/CodeGen/MachineSink.cpp | 4 +-
llvm/lib/CodeGen/MachineVerifier.cpp | 16 +++
llvm/lib/CodeGen/PHIElimination.cpp | 2 +
llvm/lib/CodeGen/RegAllocFast.cpp | 20 ++--
llvm/lib/CodeGen/RegAllocGreedy.cpp | 3 +-
llvm/lib/CodeGen/SplitKit.cpp | 5 +-
llvm/lib/CodeGen/TargetInstrInfo.cpp | 6 ++
.../AMDGPU/AMDGPUInstructionSelector.cpp | 7 +-
llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp | 11 +-
llvm/lib/Target/AMDGPU/SIISelLowering.cpp | 6 ++
llvm/lib/Target/AMDGPU/SIInstrInfo.cpp | 45 --------
llvm/lib/Target/AMDGPU/SIInstrInfo.h | 13 ++-
llvm/lib/Target/AMDGPU/SIInstructions.td | 1 +
llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp | 5 +
llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp | 2 +
llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp | 6 ++
llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp | 16 ++-
.../AMDGPU/regalloc-hoist-spill-bb-prolog.mir | 44 ++++++++
.../AMDGPU/sgpr-remat-in-bb-prolog.mir | 82 ++++++++++++++
.../CodeGen/MIR/AMDGPU/bb-prolog-flag.mir | 17 +++
.../test/CodeGen/MIR/AMDGPU/lr-split-flag.mir | 3 +-
.../test/MachineVerifier/AMDGPU/bb-prolog.mir | 43 ++++++++
.../CodeGen/MachineBasicBlockTest.cpp | 101 ++++++++++++++++++
llvm/utils/UpdateTestChecks/mir.py | 2 +-
36 files changed, 434 insertions(+), 105 deletions(-)
create mode 100644 llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-bb-prolog.mir
create mode 100644 llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
create mode 100644 llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
create mode 100644 llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
diff --git a/llvm/include/llvm/CodeGen/MachineBasicBlock.h b/llvm/include/llvm/CodeGen/MachineBasicBlock.h
index 215562968158f..0247cca6472f8 100644
--- a/llvm/include/llvm/CodeGen/MachineBasicBlock.h
+++ b/llvm/include/llvm/CodeGen/MachineBasicBlock.h
@@ -903,19 +903,24 @@ class MachineBasicBlock
return const_cast<MachineBasicBlock *>(this)->getFirstNonPHI();
}
- /// Return the first instruction in MBB after I that is not a PHI or a label.
- /// This is the correct point to insert lowered copies at the beginning of a
- /// basic block that must be before any debugging information.
+ /// Return the first instruction in MBB after I that is not a PHI, a label or
+ /// a block prolog instruction. This is the correct point to insert lowered
+ /// copies at the beginning of a basic block that must be before any debugging
+ /// information.
LLVM_ABI iterator SkipPHIsAndLabels(iterator I);
- /// Return the first instruction in MBB after I that is not a PHI, label or
- /// debug. This is the correct point to insert copies at the beginning of a
- /// basic block. \p Reg is the register being used by a spill or defined for a
- /// restore/split during register allocation.
+ /// Return the first instruction in MBB after I that is not a PHI, label,
+ /// debug or block prolog instruction. This is the correct point to insert
+ /// copies at the beginning of a basic block.
LLVM_ABI iterator SkipPHIsLabelsAndDebug(iterator I,
- Register Reg = Register(),
bool SkipPseudoOp = true);
+ /// Instructions [\p Begin, \p End) were inserted in front of \p End. If that
+ /// is inside the block prolog (the next instruction that is not a label,
+ /// debug instruction or pseudo probe is a prolog instruction), mark them as
+ /// prolog instructions too.
+ LLVM_ABI void inheritBBProlog(iterator Begin, iterator End);
+
/// Returns an iterator to the first terminator instruction of this basic
/// block. If a terminator does not exist, it returns end().
LLVM_ABI iterator getFirstTerminator();
diff --git a/llvm/include/llvm/CodeGen/MachineInstr.h b/llvm/include/llvm/CodeGen/MachineInstr.h
index a18ee9bde66b7..561db4c652d10 100644
--- a/llvm/include/llvm/CodeGen/MachineInstr.h
+++ b/llvm/include/llvm/CodeGen/MachineInstr.h
@@ -129,8 +129,10 @@ class MachineInstr
InBounds = 1 << 22, // Pointer arithmetic remains inbounds.
// Implies NoUSWrap.
LRSplit = 1 << 23, // Instruction for live range split.
- NonNull = 1 << 24 // Address space cast source is not the null
+ NonNull = 1 << 24, // Address space cast source is not the null
// value of the source address space.
+ BBProlog = 1 << 25 // Part of the basic block prolog: must stay
+ // ahead of everything inserted at block entry.
};
static constexpr uint32_t getPoisonGeneratingFlags() {
diff --git a/llvm/include/llvm/CodeGen/TargetInstrInfo.h b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
index b013511d33313..94db7f162967f 100644
--- a/llvm/include/llvm/CodeGen/TargetInstrInfo.h
+++ b/llvm/include/llvm/CodeGen/TargetInstrInfo.h
@@ -2190,15 +2190,6 @@ class LLVM_ABI TargetInstrInfo : public MCInstrInfo {
return Inst.isReturn() && Inst.isCall();
}
- /// True if the instruction is bound to the top of its basic block and no
- /// other instructions shall be inserted before it. This can be implemented
- /// to prevent register allocator to insert spills for \p Reg before such
- /// instructions.
- virtual bool isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg = Register()) const {
- return false;
- }
-
/// Allows targets to use appropriate copy instruction while spilitting live
/// range of a register in register allocation.
virtual unsigned getLiveRangeSplitOpcode(Register Reg,
diff --git a/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp b/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
index 8d2f9bfb6c5cb..2be3def1b09d0 100644
--- a/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
+++ b/llvm/lib/CodeGen/FixupStatepointCallerSaved.cpp
@@ -455,8 +455,7 @@ class StatepointState {
<< RegToSlotIdx[Reg] << " after statepoint\n");
if (EHPad && RC.tryRecordReload(Reg, RegToSlotIdx[Reg], EHPad)) {
- auto EHPadInsertPoint =
- EHPad->SkipPHIsLabelsAndDebug(EHPad->begin(), Reg);
+ auto EHPadInsertPoint = EHPad->SkipPHIsLabelsAndDebug(EHPad->begin());
insertReloadBefore(Reg, EHPadInsertPoint, EHPad);
LLVM_DEBUG(dbgs() << "...also reload at EHPad "
<< printMBBReference(*EHPad) << "\n");
diff --git a/llvm/lib/CodeGen/InlineSpiller.cpp b/llvm/lib/CodeGen/InlineSpiller.cpp
index f3682a2e24808..d35f75981ec59 100644
--- a/llvm/lib/CodeGen/InlineSpiller.cpp
+++ b/llvm/lib/CodeGen/InlineSpiller.cpp
@@ -457,7 +457,7 @@ bool InlineSpiller::hoistSpillInsideBB(LiveInterval &SpillLI,
MachineBasicBlock *MBB = DefMBB;
MachineBasicBlock::iterator MII;
if (SrcVNI->isPHIDef())
- MII = MBB->SkipPHIsLabelsAndDebug(MBB->begin(), SrcReg);
+ MII = MBB->SkipPHIsLabelsAndDebug(MBB->begin());
else {
MachineInstr *DefMI = LIS.getInstructionFromIndex(SrcVNI->def);
assert(DefMI && "Defining instruction disappeared");
@@ -500,6 +500,7 @@ bool InlineSpiller::hoistSpillInsideBB(LiveInterval &SpillLI,
// Insert spill without kill flag immediately after def.
TII.storeRegToStackSlot(*MBB, MII, SrcReg, false, StackSlot,
MRI.getRegClass(SrcReg), Register());
+ MBB->inheritBBProlog(MIS.begin(), MII);
LIS.InsertMachineInstrRangeInMaps(MIS.begin(), MII);
for (const MachineInstr &MI : make_range(MIS.begin(), MII))
getVDefInterval(MI, LIS);
@@ -1252,6 +1253,7 @@ void InlineSpiller::insertReload(Register NewVReg,
MachineInstrSpan MIS(MI, &MBB);
TII.loadRegFromStackSlot(MBB, MI, NewVReg, StackSlot,
MRI.getRegClass(NewVReg), Register());
+ MBB.inheritBBProlog(MIS.begin(), MI);
LIS.InsertMachineInstrRangeInMaps(MIS.begin(), MI);
@@ -1297,6 +1299,7 @@ void InlineSpiller::insertSpill(Register NewVReg, bool isKill,
.addReg(NewVReg, getKillRegState(isKill));
MachineBasicBlock::iterator Spill = std::next(MI);
+ MBB.inheritBBProlog(Spill, MIS.end());
LIS.InsertMachineInstrRangeInMaps(Spill, MIS.end());
for (const MachineInstr &MI : make_range(Spill, MIS.end()))
getVDefInterval(MI, LIS);
@@ -1868,6 +1871,7 @@ void HoistSpillHelper::hoistAllSpills() {
MachineInstrSpan MIS(MII, BB);
TII.storeRegToStackSlot(*BB, MII, LiveReg, false, Slot,
MRI.getRegClass(LiveReg), Register());
+ BB->inheritBBProlog(MIS.begin(), MII);
LIS.InsertMachineInstrRangeInMaps(MIS.begin(), MII);
for (const MachineInstr &MI : make_range(MIS.begin(), MII))
getVDefInterval(MI, LIS);
diff --git a/llvm/lib/CodeGen/LiveRangeEdit.cpp b/llvm/lib/CodeGen/LiveRangeEdit.cpp
index 1d6b835f16468..f1c24d4e94255 100644
--- a/llvm/lib/CodeGen/LiveRangeEdit.cpp
+++ b/llvm/lib/CodeGen/LiveRangeEdit.cpp
@@ -91,6 +91,8 @@ SlotIndex LiveRangeEdit::rematerializeAt(
// to false anyway in case the isDead flag of RM.OrigMI's dest register
// is true.
(*--MI).clearRegisterDeads(DestReg);
+ MI->clearFlag(MachineInstr::BBProlog);
+ MBB.inheritBBProlog(MI, std::next(MI));
Rematted.insert(RM.ParentVNI);
++NumReMaterialization;
diff --git a/llvm/lib/CodeGen/MIRParser/MILexer.cpp b/llvm/lib/CodeGen/MIRParser/MILexer.cpp
index 0851c42c7777f..b4cb50cbabe7c 100644
--- a/llvm/lib/CodeGen/MIRParser/MILexer.cpp
+++ b/llvm/lib/CodeGen/MIRParser/MILexer.cpp
@@ -303,6 +303,7 @@ static MIToken::TokenKind getIdentifierKind(StringRef Identifier) {
.Case("noconvergent", MIToken::kw_noconvergent)
.Case("mmra", MIToken::kw_mmra)
.Case("lr-split", MIToken::kw_lr_split)
+ .Case("bb-prolog", MIToken::kw_bb_prolog)
.Default(MIToken::Identifier);
}
diff --git a/llvm/lib/CodeGen/MIRParser/MILexer.h b/llvm/lib/CodeGen/MIRParser/MILexer.h
index 0e28ee3635a87..eb9b728f9ed99 100644
--- a/llvm/lib/CodeGen/MIRParser/MILexer.h
+++ b/llvm/lib/CodeGen/MIRParser/MILexer.h
@@ -155,6 +155,7 @@ struct MIToken {
kw_noconvergent,
kw_mmra,
kw_lr_split,
+ kw_bb_prolog,
// Metadata types.
kw_distinct,
diff --git a/llvm/lib/CodeGen/MIRParser/MIParser.cpp b/llvm/lib/CodeGen/MIRParser/MIParser.cpp
index 7b45d028efdd7..9440227683c4b 100644
--- a/llvm/lib/CodeGen/MIRParser/MIParser.cpp
+++ b/llvm/lib/CodeGen/MIRParser/MIParser.cpp
@@ -1390,7 +1390,8 @@ bool MIParser::parseInstruction(unsigned &OpCode, unsigned &Flags) {
Token.is(MIToken::kw_samesign) ||
Token.is(MIToken::kw_inbounds) ||
Token.is(MIToken::kw_nonnull) ||
- Token.is(MIToken::kw_lr_split)) {
+ Token.is(MIToken::kw_lr_split) ||
+ Token.is(MIToken::kw_bb_prolog)) {
// clang-format on
// Mine frame and fast math flags
if (Token.is(MIToken::kw_frame_setup))
@@ -1437,6 +1438,8 @@ bool MIParser::parseInstruction(unsigned &OpCode, unsigned &Flags) {
Flags |= MachineInstr::NonNull;
if (Token.is(MIToken::kw_lr_split))
Flags |= MachineInstr::LRSplit;
+ if (Token.is(MIToken::kw_bb_prolog))
+ Flags |= MachineInstr::BBProlog;
lex();
}
diff --git a/llvm/lib/CodeGen/MIRPrinter.cpp b/llvm/lib/CodeGen/MIRPrinter.cpp
index 7ac0c40dfd93f..ef6c7e6360eba 100644
--- a/llvm/lib/CodeGen/MIRPrinter.cpp
+++ b/llvm/lib/CodeGen/MIRPrinter.cpp
@@ -908,6 +908,8 @@ static void printMI(raw_ostream &OS, MFPrintState &State,
OS << "lr-split ";
if (MI.getFlag(MachineInstr::NonNull))
OS << "nonnull ";
+ if (MI.getFlag(MachineInstr::BBProlog))
+ OS << "bb-prolog ";
// NOTE: Please add new MIFlags also to the MI_FLAGS_STR in
// llvm/utils/UpdateTestChecks/mir.py.
diff --git a/llvm/lib/CodeGen/MachineBasicBlock.cpp b/llvm/lib/CodeGen/MachineBasicBlock.cpp
index 39c286f7d3510..ab7eeb4c7091b 100644
--- a/llvm/lib/CodeGen/MachineBasicBlock.cpp
+++ b/llvm/lib/CodeGen/MachineBasicBlock.cpp
@@ -210,13 +210,22 @@ MachineBasicBlock::iterator MachineBasicBlock::getFirstNonPHI() {
return I;
}
+void MachineBasicBlock::inheritBBProlog(iterator Begin, iterator End) {
+ iterator Next = End;
+ while (Next != end() && (Next->isDebugOrPseudoInstr() || Next->isPosition()))
+ ++Next;
+ if (Next == end() || !Next->getFlag(MachineInstr::BBProlog))
+ return;
+ for (MachineInstr &MI : make_range(Begin, End))
+ if (!MI.isDebugOrPseudoInstr() && !MI.isPosition())
+ MI.setFlag(MachineInstr::BBProlog);
+}
+
MachineBasicBlock::iterator
MachineBasicBlock::SkipPHIsAndLabels(MachineBasicBlock::iterator I) {
- const TargetInstrInfo *TII = getParent()->getSubtarget().getInstrInfo();
-
iterator E = end();
- while (I != E && (I->isPHI() || I->isPosition() ||
- TII->isBasicBlockPrologue(*I)))
+ while (I != E &&
+ (I->isPHI() || I->isPosition() || I->getFlag(MachineInstr::BBProlog)))
++I;
// FIXME: This needs to change if we wish to bundle labels
// inside the bundle.
@@ -227,13 +236,11 @@ MachineBasicBlock::SkipPHIsAndLabels(MachineBasicBlock::iterator I) {
MachineBasicBlock::iterator
MachineBasicBlock::SkipPHIsLabelsAndDebug(MachineBasicBlock::iterator I,
- Register Reg, bool SkipPseudoOp) {
- const TargetInstrInfo *TII = getParent()->getSubtarget().getInstrInfo();
-
+ bool SkipPseudoOp) {
iterator E = end();
while (I != E && (I->isPHI() || I->isPosition() || I->isDebugInstr() ||
(SkipPseudoOp && I->isPseudoProbe()) ||
- TII->isBasicBlockPrologue(*I, Reg)))
+ I->getFlag(MachineInstr::BBProlog)))
++I;
// FIXME: This needs to change if we wish to bundle labels / dbg_values
// inside the bundle.
diff --git a/llvm/lib/CodeGen/MachineInstr.cpp b/llvm/lib/CodeGen/MachineInstr.cpp
index 4978967c7cb11..ce7dac9fcbe91 100644
--- a/llvm/lib/CodeGen/MachineInstr.cpp
+++ b/llvm/lib/CodeGen/MachineInstr.cpp
@@ -1920,6 +1920,8 @@ void MachineInstr::print(raw_ostream &OS, ModuleSlotTracker &MST,
OS << "lr-split ";
if (getFlag(MachineInstr::NonNull))
OS << "nonnull ";
+ if (getFlag(MachineInstr::BBProlog))
+ OS << "bb-prolog ";
// Print the opcode name.
if (TII)
diff --git a/llvm/lib/CodeGen/MachineSink.cpp b/llvm/lib/CodeGen/MachineSink.cpp
index cbd2598a47210..834effea43e01 100644
--- a/llvm/lib/CodeGen/MachineSink.cpp
+++ b/llvm/lib/CodeGen/MachineSink.cpp
@@ -343,7 +343,7 @@ static bool blockPrologueInterferes(const MachineBasicBlock *BB,
for (MachineBasicBlock::const_iterator PI = BB->getFirstNonPHI(); PI != End;
++PI) {
// Only check target defined prologue instructions
- if (!TII->isBasicBlockPrologue(*PI))
+ if (!PI->getFlag(MachineInstr::BBProlog))
continue;
for (auto &MO : MI.operands()) {
if (!MO.isReg())
@@ -1776,7 +1776,7 @@ bool MachineSinking::aggressivelySinkIntoCycle(
continue;
}
// We cannot sink before the prologue
- if (MI->isPosition() || TII->isBasicBlockPrologue(*MI)) {
+ if (MI->isPosition() || MI->getFlag(MachineInstr::BBProlog)) {
LLVM_DEBUG(dbgs() << "AggressiveCycleSink: Use is BasicBlock prologue, "
"can't sink.\n");
continue;
diff --git a/llvm/lib/CodeGen/MachineVerifier.cpp b/llvm/lib/CodeGen/MachineVerifier.cpp
index cae5012b41c33..9cecf0709a849 100644
--- a/llvm/lib/CodeGen/MachineVerifier.cpp
+++ b/llvm/lib/CodeGen/MachineVerifier.cpp
@@ -159,6 +159,7 @@ struct MachineVerifier {
const MachineInstr *FirstNonPHI = nullptr;
const MachineInstr *FirstTerminator = nullptr;
+ const MachineInstr *FirstNonBBProlog = nullptr;
BlockSet FunctionBlocks;
BitVector regsReserved;
@@ -742,6 +743,7 @@ void
MachineVerifier::visitMachineBasicBlockBefore(const MachineBasicBlock *MBB) {
FirstTerminator = nullptr;
FirstNonPHI = nullptr;
+ FirstNonBBProlog = nullptr;
if (MRI->tracksLiveness() && hasPHIs(*MF)) {
// If this block has allocatable physical registers live-in, check that
@@ -2359,6 +2361,20 @@ void MachineVerifier::visitMachineInstrBefore(const MachineInstr *MI) {
} else if (FirstNonPHI == nullptr)
FirstNonPHI = MI;
+ // Until register allocation is done, block prolog instructions form a block
+ // prefix. Only PHIs, labels, debug instructions and pseudo probes may precede
+ // them.
+ if (!MF->getProperties().hasNoVRegs() && !MI->isPHI() && !MI->isPosition() &&
+ !MI->isDebugOrPseudoInstr()) {
+ if (!MI->getFlag(MachineInstr::BBProlog)) {
+ if (!FirstNonBBProlog)
+ FirstNonBBProlog = MI;
+ } else if (FirstNonBBProlog) {
+ report("Block prolog instruction after non-prolog instruction", MI);
+ OS << "First non-prolog instruction: " << *FirstNonBBProlog;
+ }
+ }
+
// Check the tied operands.
if (MI->isInlineAsm())
verifyInlineAsm(MI);
diff --git a/llvm/lib/CodeGen/PHIElimination.cpp b/llvm/lib/CodeGen/PHIElimination.cpp
index de69ca8d91898..c1b257ab35d16 100644
--- a/llvm/lib/CodeGen/PHIElimination.cpp
+++ b/llvm/lib/CodeGen/PHIElimination.cpp
@@ -423,6 +423,8 @@ void PHIEliminationImpl::LowerPHINode(MachineBasicBlock &MBB,
PHICopy = TII->createPHIDestinationCopy(
MBB, AfterPHIsIt, MPhi->getDebugLoc(), IncomingReg, DestReg);
}
+ MBB.inheritBBProlog(PHICopy->getIterator(),
+ std::next(PHICopy->getIterator()));
if (MPhi->peekDebugInstrNum() && IncomingReg) {
// If referred to by debug-info, store where this PHI was.
diff --git a/llvm/lib/CodeGen/RegAllocFast.cpp b/llvm/lib/CodeGen/RegAllocFast.cpp
index 078b3f3056a7e..73d6e384c192c 100644
--- a/llvm/lib/CodeGen/RegAllocFast.cpp
+++ b/llvm/lib/CodeGen/RegAllocFast.cpp
@@ -616,7 +616,9 @@ void RegAllocFastImpl::spill(MachineBasicBlock::iterator Before,
LLVM_DEBUG(dbgs() << " to stack slot #" << FI << '\n');
const TargetRegisterClass &RC = *MRI->getRegClass(VirtReg);
+ MachineInstrSpan MIS(Before, MBB);
TII->storeRegToStackSlot(*MBB, Before, AssignedReg, Kill, FI, &RC, VirtReg);
+ MBB->inheritBBProlog(MIS.begin(), Before);
++NumStores;
MachineBasicBlock::iterator FirstTerm = MBB->getFirstTerminator();
@@ -673,7 +675,9 @@ void RegAllocFastImpl::reload(MachineBasicBlock::iterator Before,
<< printReg(PhysReg, TRI) << '\n');
int FI = getStackSpaceFor(VirtReg);
const TargetRegisterClass &RC = *MRI->getRegClass(VirtReg);
+ MachineInstrSpan MIS(Before, MBB);
TII->loadRegFromStackSlot(*MBB, Before, PhysReg, FI, &RC, VirtReg);
+ MBB->inheritBBProlog(MIS.begin(), Before);
++NumLoads;
}
@@ -691,7 +695,7 @@ MachineBasicBlock::iterator RegAllocFastImpl::getMBBBeginInsertionPoint(
}
// Skip prologues and inlineasm_br spills to place reloads afterwards.
- if (!TII->isBasicBlockPrologue(*I) && !mayBeSpillFromInlineAsmBr(*I))
+ if (!I->getFlag(MachineInstr::BBProlog) && !mayBeSpillFromInlineAsmBr(*I))
break;
// However if a prolog instruction reads a register that needs to be
@@ -1104,9 +1108,10 @@ bool RegAllocFastImpl::defineLiveThroughVirtReg(MachineInstr &MI,
std::next((MachineBasicBlock::iterator)MI.getIterator());
LLVM_DEBUG(dbgs() << "Copy " << printReg(LRI->PhysReg, TRI) << " to "
<< printReg(PrevReg, TRI) << '\n');
- BuildMI(*MBB, InsertBefore, MI.getDebugLoc(),
- TII->get(TargetOpcode::COPY), PrevReg)
- .addReg(LRI->PhysReg, llvm::RegState::Kill);
+ MachineInstr *Copy = BuildMI(*MBB, InsertBefore, MI.getDebugLoc(),
+ TII->get(TargetOpcode::COPY), PrevReg)
+ .addReg(LRI->PhysReg, llvm::RegState::Kill);
+ MBB->inheritBBProlog(Copy->getIterator(), InsertBefore);
}
MachineOperand &MO = MI.getOperand(OpNum);
if (MO.getSubReg() && !MO.isUndef()) {
@@ -1171,8 +1176,11 @@ bool RegAllocFastImpl::defineVirtReg(MachineInstr &MI, unsigned OpNum,
for (MachineOperand &MO : MI.operands()) {
if (MO.isMBB()) {
MachineBasicBlock *Succ = MO.getMBB();
- TII->storeRegToStackSlot(*Succ, Succ->begin(), PhysReg, Kill, FI,
- &RC, VirtReg);
+ MachineBasicBlock::iterator SuccBegin = Succ->begin();
+ MachineInstrSpan MIS(SuccBegin, Succ);
+ TII->storeRegToStackSlot(*Succ, SuccBegin, PhysReg, Kill, FI, &RC,
+ VirtReg);
+ Succ->inheritBBProlog(MIS.begin(), SuccBegin);
++NumStores;
Succ->addLiveIn(PhysReg);
}
diff --git a/llvm/lib/CodeGen/RegAllocGreedy.cpp b/llvm/lib/CodeGen/RegAllocGreedy.cpp
index af4fc60fe5ec3..b7b41fba7df01 100644
--- a/llvm/lib/CodeGen/RegAllocGreedy.cpp
+++ b/llvm/lib/CodeGen/RegAllocGreedy.cpp
@@ -844,8 +844,7 @@ bool RAGreedy::addThroughConstraints(InterferenceCache::Cursor Intf,
return false;
// Interference for the live-in value.
- Register Reg = SA->getParent().reg();
- auto InsertPt = MBB->SkipPHIsLabelsAndDebug(MBB->begin(), Reg);
+ auto InsertPt = MBB->SkipPHIsLabelsAndDebug(MBB->begin());
SlotIndex InsertIdx = InsertPt == MBB->end()
? Indexes->getMBBEndIdx(MBB)
: LIS->getInstructionIndex(*InsertPt);
diff --git a/llvm/lib/CodeGen/SplitKit.cpp b/llvm/lib/CodeGen/SplitKit.cpp
index 404cfe43289d6..76fe405e7116c 100644
--- a/llvm/lib/CodeGen/SplitKit.cpp
+++ b/llvm/lib/CodeGen/SplitKit.cpp
@@ -672,6 +672,7 @@ VNInfo *SplitEditor::defFromParent(unsigned RegIdx, const VNInfo *ParentVNI,
}
SlotIndex Def;
+ MachineInstrSpan MIS(I, &MBB);
if (LaneMask.none()) {
const MCInstrDesc &Desc = TII.get(TargetOpcode::IMPLICIT_DEF);
MachineInstr *ImplicitDef = BuildMI(MBB, I, DebugLoc(), Desc, Reg);
@@ -681,6 +682,7 @@ VNInfo *SplitEditor::defFromParent(unsigned RegIdx, const VNInfo *ParentVNI,
++NumCopies;
Def = buildCopy(Edit->getReg(), Reg, LaneMask, MBB, I, Late, RegIdx);
}
+ MBB.inheritBBProlog(MIS.begin(), I);
// Define the value in Reg.
return defValue(RegIdx, ParentVNI, Def, false);
@@ -851,9 +853,8 @@ SlotIndex SplitEditor::leaveIntvAtTop(MachineBasicBlock &MBB) {
}
unsigned RegIdx = 0;
- Register Reg = LIS.getInterval(Edit->get(RegIdx)).reg();
VNInfo *VNI = defFromParent(RegIdx, ParentVNI, Start, MBB,
- MBB.SkipPHIsLabelsAndDebug(MBB.begin(), Reg));
+ MBB.SkipPHIsLabelsAndDebug(MBB.begin()));
RegAssign.insert(Start, VNI->def, OpenIdx);
LLVM_DEBUG(dump());
return VNI->def;
diff --git a/llvm/lib/CodeGen/TargetInstrInfo.cpp b/llvm/lib/CodeGen/TargetInstrInfo.cpp
index 92fc628e888e5..0a0d9261036b8 100644
--- a/llvm/lib/CodeGen/TargetInstrInfo.cpp
+++ b/llvm/lib/CodeGen/TargetInstrInfo.cpp
@@ -788,6 +788,10 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
// call instructions. We need copy it form old instruction.
NewMI->cloneInstrSymbols(MF, MI);
+ // NewMI replaces MI.
+ if (MI.getFlag(MachineInstr::BBProlog))
+ NewMI->setFlag(MachineInstr::BBProlog);
+
return NewMI;
}
@@ -801,6 +805,7 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
const MachineOperand &MO = MI.getOperand(1 - Ops[0]);
MachineBasicBlock::iterator Pos = MI;
+ MachineInstrSpan MIS(Pos, MBB);
if (Flags == MachineMemOperand::MOStore) {
if (MO.isUndef()) {
// If this is an undef copy, we do not need to bother we inserting spill
@@ -812,6 +817,7 @@ MachineInstr *TargetInstrInfo::foldMemoryOperand(MachineInstr &MI,
}
} else
loadRegFromStackSlot(*MBB, Pos, MO.getReg(), FI, RC, Register());
+ MBB->inheritBBProlog(MIS.begin(), Pos);
return &*--Pos;
}
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp b/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
index f06837ff13869..10094c07dc72b 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUInstructionSelector.cpp
@@ -1817,8 +1817,11 @@ bool AMDGPUInstructionSelector::selectEndCfIntrinsic(MachineInstr &MI) const {
// FIXME: Manually selecting to avoid dealing with the SReg_1 trick
// SelectionDAG uses for wave32 vs wave64.
MachineBasicBlock *BB = MI.getParent();
- BuildMI(*BB, &MI, MI.getDebugLoc(), TII.get(AMDGPU::SI_END_CF))
- .add(MI.getOperand(1));
+ MachineInstr *EndCf =
+ BuildMI(*BB, &MI, MI.getDebugLoc(), TII.get(AMDGPU::SI_END_CF))
+ .add(MI.getOperand(1));
+ // Exec restore at the top of the join block.
+ SIInstrInfo::setBBPrologIfAtBlockStart(*EndCf);
Register Reg = MI.getOperand(1).getReg();
MI.eraseFromParent();
diff --git a/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp b/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
index 964ddde0fbb01..0f9be4c20b591 100644
--- a/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
+++ b/llvm/lib/Target/AMDGPU/SIFixSGPRCopies.cpp
@@ -443,10 +443,9 @@ static bool isReachable(const MachineInstr *From,
}
// Return the first non-prologue instruction in the block.
-static MachineBasicBlock::iterator
-getFirstNonPrologue(MachineBasicBlock *MBB, const TargetInstrInfo *TII) {
+static MachineBasicBlock::iterator getFirstNonPrologue(MachineBasicBlock *MBB) {
MachineBasicBlock::iterator I = MBB->getFirstNonPHI();
- while (I != MBB->end() && TII->isBasicBlockPrologue(*I))
+ while (I != MBB->end() && I->getFlag(MachineInstr::BBProlog))
++I;
return I;
@@ -558,7 +557,7 @@ static bool hoistAndMergeSGPRInits(unsigned Reg,
continue;
}
- MachineBasicBlock::iterator I = getFirstNonPrologue(MBB, TII);
+ MachineBasicBlock::iterator I = getFirstNonPrologue(MBB);
if (!interferes(MI1, I) && !interferes(MI2, I)) {
LLVM_DEBUG(dbgs()
<< "Erasing from "
@@ -597,11 +596,11 @@ static bool hoistAndMergeSGPRInits(unsigned Reg,
auto &Defs = Init.second;
for (auto *MI : Defs) {
auto *MBB = MI->getParent();
- MachineInstr &BoundaryMI = *getFirstNonPrologue(MBB, TII);
+ MachineInstr &BoundaryMI = *getFirstNonPrologue(MBB);
MachineBasicBlock::reverse_iterator B(BoundaryMI);
// Check if B should actually be a boundary. If not set the previous
// instruction as the boundary instead.
- if (!TII->isBasicBlockPrologue(*B))
+ if (!B->getFlag(MachineInstr::BBProlog))
B++;
auto R = std::next(MI->getReverseIterator());
diff --git a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
index 25fcaa0ac8c4b..1b2d1342d8d4f 100644
--- a/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/SIISelLowering.cpp
@@ -19842,6 +19842,12 @@ void SITargetLowering::AdjustInstrPostInstrSelection(MachineInstr &MI,
MachineFunction *MF = MI.getMF();
MachineRegisterInfo &MRI = MF->getRegInfo();
+ if (MI.getOpcode() == AMDGPU::SI_END_CF) {
+ // Exec restore at the top of the join block.
+ SIInstrInfo::setBBPrologIfAtBlockStart(MI);
+ return;
+ }
+
if (TII->isVOP3(MI.getOpcode())) {
// Make sure constant bus requirements are respected.
TII->legalizeOperandsVOP3(MRI, MI);
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
index 0346ebc254361..f4dadf66d2f78 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
@@ -10491,51 +10491,6 @@ unsigned SIInstrInfo::getLiveRangeSplitOpcode(Register SrcReg,
return AMDGPU::COPY;
}
-bool SIInstrInfo::canAddToBBProlog(const MachineInstr &MI) const {
- uint32_t Opcode = MI.getOpcode();
- // Check if it is SGPR spill or wwm-register spill Opcode.
- if (isSGPRSpill(Opcode) || isWWMRegSpillOpcode(Opcode))
- return true;
-
- const MachineFunction *MF = MI.getMF();
- const MachineRegisterInfo &MRI = MF->getRegInfo();
- const SIMachineFunctionInfo *MFI = MF->getInfo<SIMachineFunctionInfo>();
-
- // See if this is Liverange split instruction inserted for SGPR or
- // wwm-register. The implicit def inserted for wwm-registers should also be
- // included as they can appear at the bb begin.
- bool IsLRSplitInst = MI.getFlag(MachineInstr::LRSplit);
- if (!IsLRSplitInst && Opcode != AMDGPU::IMPLICIT_DEF)
- return false;
-
- Register Reg = MI.getOperand(0).getReg();
- if (RI.isSGPRClass(RI.getRegClassForReg(MRI, Reg)))
- return IsLRSplitInst;
-
- return MFI->isWWMReg(Reg);
-}
-
-bool SIInstrInfo::isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg) const {
- // We need to handle instructions which may be inserted during register
- // allocation to handle the prolog. The initial prolog instruction may have
- // been separated from the start of the block by spills and copies inserted
- // needed by the prolog. However, the insertions for scalar registers can
- // always be placed at the BB top as they are independent of the exec mask
- // value.
- bool IsNullOrVectorRegister = true;
- if (Reg) {
- const MachineFunction *MF = MI.getMF();
- const MachineRegisterInfo &MRI = MF->getRegInfo();
- IsNullOrVectorRegister = !RI.isSGPRClass(RI.getRegClassForReg(MRI, Reg));
- }
-
- return IsNullOrVectorRegister &&
- (canAddToBBProlog(MI) ||
- (!MI.isTerminator() && MI.getOpcode() != AMDGPU::COPY &&
- MI.modifiesRegister(AMDGPU::EXEC, &RI)));
-}
-
MachineInstrBuilder
SIInstrInfo::getAddNoCarry(MachineBasicBlock &MBB,
MachineBasicBlock::iterator I,
diff --git a/llvm/lib/Target/AMDGPU/SIInstrInfo.h b/llvm/lib/Target/AMDGPU/SIInstrInfo.h
index 48eba6b2a567d..fa3cb9a8a2cc1 100644
--- a/llvm/lib/Target/AMDGPU/SIInstrInfo.h
+++ b/llvm/lib/Target/AMDGPU/SIInstrInfo.h
@@ -924,6 +924,14 @@ class SIInstrInfo final : public AMDGPUGenInstrInfo {
static bool isSpill(const MachineInstr &MI) { return isSpill(MI.getDesc()); }
+ /// Mark \p MI as a block prolog instruction if it is at the block start.
+ static void setBBPrologIfAtBlockStart(MachineInstr &MI) {
+ MachineBasicBlock &MBB = *MI.getParent();
+ if (!MI.isTerminator() &&
+ MBB.SkipPHIsLabelsAndDebug(MBB.begin()) == MI.getIterator())
+ MI.setFlag(MachineInstr::BBProlog);
+ }
+
static bool isWWMRegSpillOpcode(uint32_t Opcode) {
return Opcode == AMDGPU::SI_SPILL_WWM_V32_SAVE ||
Opcode == AMDGPU::SI_SPILL_WWM_AV32_SAVE ||
@@ -1684,11 +1692,6 @@ class SIInstrInfo final : public AMDGPUGenInstrInfo {
unsigned getLiveRangeSplitOpcode(Register Reg,
const MachineFunction &MF) const override;
- bool isBasicBlockPrologue(const MachineInstr &MI,
- Register Reg = Register()) const override;
-
- bool canAddToBBProlog(const MachineInstr &MI) const;
-
MachineInstr *createPHIDestinationCopy(MachineBasicBlock &MBB,
MachineBasicBlock::iterator InsPt,
const DebugLoc &DL, Register Src,
diff --git a/llvm/lib/Target/AMDGPU/SIInstructions.td b/llvm/lib/Target/AMDGPU/SIInstructions.td
index 3cf20ffb1fcf4..6a24ecc5d23cc 100644
--- a/llvm/lib/Target/AMDGPU/SIInstructions.td
+++ b/llvm/lib/Target/AMDGPU/SIInstructions.td
@@ -630,6 +630,7 @@ def SI_LOOP : CFPseudoInstSI <
def SI_END_CF : CFPseudoInstSI <
(outs), (ins SReg_1:$saved), [], 1, 1> {
let Size = 4;
+ let hasPostISelHook = 1; // Sets MachineInstr::BBProlog.
let isAsCheapAsAMove = 1;
let isReMaterializable = 1;
let hasSideEffects = 1;
diff --git a/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp b/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
index 8d7bd32c866c4..4e841a7934c22 100644
--- a/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
+++ b/llvm/lib/Target/AMDGPU/SILowerControlFlow.cpp
@@ -324,6 +324,7 @@ void SILowerControlFlow::emitElse(MachineInstr &MI) {
BuildMI(MBB, Start, DL, TII->get(LMC.OrSaveExecOpc), SaveReg)
.add(MI.getOperand(1)); // Saved EXEC
setImpSCCDefDead(*OrSaveExec, /*IsDead=*/true);
+ OrSaveExec->setFlag(MachineInstr::BBProlog);
if (LV)
LV->replaceKillInstruction(SrcReg, MI, *OrSaveExec);
@@ -530,6 +531,10 @@ MachineBasicBlock *SILowerControlFlow::emitEndCf(MachineInstr &MI) {
.addReg(LMC.ExecReg)
.add(MI.getOperand(0));
copySCCDefDead(*NewMI, MI.getOperand(2));
+ if (NeedBlockSplit)
+ NewMI->clearFlag(MachineInstr::BBProlog);
+ else
+ NewMI->setFlag(MachineInstr::BBProlog);
if (LV) {
LV->replaceKillInstruction(DataReg, MI, *NewMI);
diff --git a/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp b/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
index d0a57e163d29a..fd7809e1fb76e 100644
--- a/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
+++ b/llvm/lib/Target/AMDGPU/SILowerSGPRSpills.cpp
@@ -132,6 +132,7 @@ static void insertCSRSaves(const GCNSubtarget &ST, MachineBasicBlock &SaveBlock,
bool Success = TFI->spillCalleeSavedRegisters(SaveBlock, I, CSI, TRI);
assert(Success && "spillCalleeSavedRegisters should always succeed");
(void)Success;
+ SaveBlock.inheritBBProlog(MIS.begin(), I);
// TFI doesn't update Indexes and LIS, so we have to do it separately.
if (Indexes)
@@ -581,6 +582,7 @@ bool SILowerSGPRSpills::run(MachineFunction &MF) {
MachineBasicBlock &Block = *IP.MBB;
DebugLoc DL = Block.findDebugLoc(IP.It);
auto MIB = BuildMI(Block, IP.It, DL, TII->get(AMDGPU::IMPLICIT_DEF), Reg);
+ Block.inheritBBProlog(MIB->getIterator(), IP.It);
// Add WWM flag to the virtual register.
FuncInfo->setFlag(Reg, AMDGPU::VirtRegFlag::WWM_REG);
diff --git a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
index 1aa423e1da647..18dd3eb9f82ee 100644
--- a/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/SIRegisterInfo.cpp
@@ -2172,6 +2172,7 @@ bool SIRegisterInfo::spillSGPR(MachineBasicBlock::iterator MI, int Index,
"undef spill should have been deleted earlier");
SGPRSpillBuilder SB(*this, *ST.getInstrInfo(), isWave32, MI, Index, RS);
+ MachineInstrSpan MIS(MI, MI->getParent());
ArrayRef<SpilledReg> VGPRSpills =
SpillToPhysVGPRLane ? SB.MFI.getSGPRSpillToPhysicalVGPRLanes(Index)
@@ -2316,6 +2317,8 @@ bool SIRegisterInfo::spillSGPR(MachineBasicBlock::iterator MI, int Index,
SB.restore();
}
+ // The expansion replaces MI.
+ MI->getParent()->inheritBBProlog(MIS.begin(), MI);
MI->eraseFromParent();
SB.MFI.addToSpilledSGPRs(SB.NumSubRegs);
@@ -2330,6 +2333,7 @@ bool SIRegisterInfo::restoreSGPR(MachineBasicBlock::iterator MI, int Index,
LiveIntervals *LIS, bool OnlyToVGPR,
bool SpillToPhysVGPRLane) const {
SGPRSpillBuilder SB(*this, *ST.getInstrInfo(), isWave32, MI, Index, RS);
+ MachineInstrSpan MIS(MI, MI->getParent());
ArrayRef<SpilledReg> VGPRSpills =
SpillToPhysVGPRLane ? SB.MFI.getSGPRSpillToPhysicalVGPRLanes(Index)
@@ -2397,6 +2401,8 @@ bool SIRegisterInfo::restoreSGPR(MachineBasicBlock::iterator MI, int Index,
SB.restore();
}
+ // The expansion replaces MI.
+ MI->getParent()->inheritBBProlog(MIS.begin(), MI);
MI->eraseFromParent();
if (LIS)
diff --git a/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp b/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
index 3c6e11acf89f9..883867d622ff2 100644
--- a/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
+++ b/llvm/lib/Target/AMDGPU/SIWholeQuadMode.cpp
@@ -1230,6 +1230,7 @@ void SIWholeQuadMode::toExact(MachineBasicBlock &MBB,
.addReg(LiveMaskReg);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
LIS->removeAllRegUnitsForPhysReg(AMDGPU::EXEC);
StateTransition[MI] = StateExact;
@@ -1249,6 +1250,7 @@ void SIWholeQuadMode::toWQM(MachineBasicBlock &MBB,
.addReg(LMC.ExecReg);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = StateWQM;
}
@@ -1270,6 +1272,7 @@ void SIWholeQuadMode::toStrictMode(MachineBasicBlock &MBB,
MI = BuildMI(MBB, Before, DL, TII->get(AMDGPU::ENTER_STRICT_WQM), SaveOrig)
.addImm(-1);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = StrictStateNeeded;
}
@@ -1295,6 +1298,7 @@ void SIWholeQuadMode::fromStrictMode(MachineBasicBlock &MBB,
BuildMI(MBB, Before, DL, TII->get(AMDGPU::EXIT_STRICT_WQM), LMC.ExecReg)
.addReg(SavedOrig);
}
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
StateTransition[MI] = NonStrictState;
}
@@ -1615,7 +1619,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
MachineInstr *SaveExec = BuildMI(*MBB, MBB->begin(), MI.getDebugLoc(),
TII->get(LMC.OrSaveExecOpc), EntryExec)
.addImm(-1)
- .setOperandDead(3);
+ .setOperandDead(3)
+ .setMIFlag(MachineInstr::BBProlog);
// Replace all uses of MI's destination reg with EntryExec.
MRI->replaceRegWith(MI.getOperand(0).getReg(), EntryExec);
@@ -1637,7 +1642,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
// This should be before all vector instructions.
MachineInstr *InitMI = BuildMI(*MBB, MBB->begin(), MI.getDebugLoc(),
TII->get(LMC.MovOpc), LMC.ExecReg)
- .addImm(MI.getOperand(0).getImm());
+ .addImm(MI.getOperand(0).getImm())
+ .setMIFlag(MachineInstr::BBProlog);
if (LIS) {
LIS->RemoveMachineInstrFromMaps(MI);
LIS->InsertMachineInstrInMaps(*InitMI);
@@ -1670,6 +1676,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
// If first instruction is definition then move pointer after it.
FirstMI = &*std::next(FirstMI->getIterator());
}
+ // The input copy feeds the exec setup below.
+ DefInstr->setFlag(MachineInstr::BBProlog);
}
}
@@ -1690,6 +1698,8 @@ void SIWholeQuadMode::lowerInitExec(MachineInstr &MI) {
.addImm(WavefrontSize);
auto CmovMI =
BuildMI(*MBB, FirstMI, DL, TII->get(LMC.CMovOpc), LMC.ExecReg).addImm(-1);
+ for (MachineInstr *I : {&*BfeMI, &*BfmMI, &*CmpMI, &*CmovMI})
+ I->setFlag(MachineInstr::BBProlog);
if (!LIS) {
MI.eraseFromParent();
@@ -1762,6 +1772,7 @@ bool SIWholeQuadMode::run(MachineFunction &MF) {
MachineInstr *MI =
BuildMI(Entry, EntryMI, DebugLoc(), TII->get(AMDGPU::COPY), LiveMaskReg)
.addReg(LMC.ExecReg);
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
Changed = true;
}
@@ -1795,6 +1806,7 @@ bool SIWholeQuadMode::run(MachineFunction &MF) {
auto MI =
BuildMI(Entry, EntryMI, DebugLoc(), TII->get(LMC.WQMOpc), LMC.ExecReg)
.addReg(LMC.ExecReg);
+ SIInstrInfo::setBBPrologIfAtBlockStart(*MI);
LIS->InsertMachineInstrInMaps(*MI);
lowerKillInstrs(true);
Changed = true;
diff --git a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-bb-prolog.mir b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-bb-prolog.mir
new file mode 100644
index 0000000000000..f978bec424641
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-bb-prolog.mir
@@ -0,0 +1,44 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 6
+# RUN: llc -mtriple=amdgpu9.50-amd-amdhsa -passes='greedy<wwm>' -stress-regalloc=3 -verify-regalloc -verify-machineinstrs -o - %s | FileCheck %s
+
+# InlineSpiller hoists the spill of %2 to right after its definition, which is
+# inside the block prolog. The spill must join the prolog.
+
+---
+name: hoist_spill_in_bb_prolog
+tracksRegLiveness: true
+machineFunctionInfo:
+ isEntryFunction: true
+ scratchRSrcReg: '$sgpr96_sgpr97_sgpr98_sgpr99'
+ stackPtrOffsetReg: '$sgpr32'
+registers:
+ - { id: 0, class: vgpr_32, flags: [ WWM_REG ], split-from: '%2' }
+ - { id: 1, class: vgpr_32, flags: [ WWM_REG ], split-from: '%2' }
+ - { id: 2, class: vgpr_32, flags: [ WWM_REG ] }
+body: |
+ bb.0:
+ liveins: $sgpr0_sgpr1
+ ; CHECK-LABEL: name: hoist_spill_in_bb_prolog
+ ; CHECK: liveins: $sgpr0_sgpr1
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: [[KILL:%[0-9]+]]:av_32 = bb-prolog KILL
+ ; CHECK-NEXT: bb-prolog SI_SPILL_WWM_AV32_SAVE [[KILL]], %stack.0, $sgpr32, 0, implicit $exec :: (store (s32) into %stack.0, addrspace 5)
+ ; CHECK-NEXT: [[WWM_COPY:%[0-9]+]]:vgpr_32 = bb-prolog WWM_COPY [[KILL]]
+ ; CHECK-NEXT: bb-prolog KILL [[WWM_COPY]]
+ ; CHECK-NEXT: bb-prolog KILL [[WWM_COPY]]
+ ; CHECK-NEXT: $exec = bb-prolog S_AND_B64 $exec, $sgpr0_sgpr1, implicit-def $scc
+ ; CHECK-NEXT: KILL implicit-def $vgpr0, implicit-def $vgpr1, implicit-def $vgpr2
+ ; CHECK-NEXT: [[SI_SPILL_WWM_AV32_RESTORE:%[0-9]+]]:av_32 = SI_SPILL_WWM_AV32_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.0, addrspace 5)
+ ; CHECK-NEXT: [[SI_SPILL_WWM_V32_RESTORE:%[0-9]+]]:vgpr_32 = SI_SPILL_WWM_V32_RESTORE %stack.0, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.0, addrspace 5)
+ ; CHECK-NEXT: KILL [[SI_SPILL_WWM_V32_RESTORE]], [[SI_SPILL_WWM_AV32_RESTORE]]
+ ; CHECK-NEXT: S_ENDPGM 0
+ %2:vgpr_32 = bb-prolog KILL
+ %0:vgpr_32 = bb-prolog WWM_COPY %2
+ bb-prolog KILL %0
+ bb-prolog KILL %0
+ %1:vgpr_32 = lr-split bb-prolog WWM_COPY %0:vgpr_32
+ $exec = bb-prolog S_AND_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ KILL implicit-def $vgpr0, implicit-def $vgpr1, implicit-def $vgpr2
+ KILL %1:vgpr_32, %2:vgpr_32
+ S_ENDPGM 0
+...
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir b/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
new file mode 100644
index 0000000000000..e76379cf98f54
--- /dev/null
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-remat-in-bb-prolog.mir
@@ -0,0 +1,82 @@
+# RUN: llc -mtriple=amdgpu9.42-amd-amdhsa -start-before=greedy,2 -stop-after=greedy,2 -stress-regalloc=3 -o - %s | FileCheck %s
+
+# The sgpr-regalloc pipeline rematerialized an SMEM load into the Flow block,
+# ahead of the exec restore of the if/else region. The exec restore carries the
+# bb-prolog flag, so the vgpr-regalloc pipeline must place the spill it inserts
+# in bb.3 after the S_OR_SAVEEXEC_B64. Otherwise the spill would only be
+# executed for the lanes of the first branch arm. The rematerialized load is not
+# part of the prolog: the spill goes ahead of it.
+
+# CHECK-LABEL: name: sgpr_remat_in_bb_prolog
+# CHECK: bb.3:
+# CHECK: bb-prolog S_OR_SAVEEXEC_B64
+# CHECK-NEXT: SI_SPILL_AV64_SAVE
+# CHECK-NEXT: S_LOAD_DWORDX8_IMM_ec
+# CHECK-NEXT: V_LSHL_ADD_U64_e64
+
+--- |
+ define amdgpu_kernel void @sgpr_remat_in_bb_prolog() { ret void }
+...
+---
+name: sgpr_remat_in_bb_prolog
+tracksRegLiveness: true
+machineFunctionInfo:
+ stackPtrOffsetReg: '$sgpr32'
+body: |
+ bb.0:
+ successors: %bb.1(0x40000000), %bb.2(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69, $vgpr0_vgpr1, $vgpr2_vgpr3
+
+ %0:vreg_64_align2 = COPY $vgpr0_vgpr1
+ %1:vreg_64_align2 = COPY $vgpr2_vgpr3
+ $exec = S_MOV_B64_term killed renamable $sgpr30_sgpr31
+ S_CBRANCH_EXECZ %bb.2, implicit $exec
+ S_BRANCH %bb.1
+
+ bb.1:
+ successors: %bb.2(0x80000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69
+
+ bb.2:
+ successors: %bb.5(0x40000000), %bb.3(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr34_sgpr35, $sgpr64_sgpr65, $sgpr68_sgpr69
+
+ %2:vreg_64_align2 = COPY $sgpr68_sgpr69, implicit $exec
+ $exec = S_MOV_B64_term killed renamable $sgpr34_sgpr35
+ S_CBRANCH_EXECZ %bb.3, implicit $exec
+ S_BRANCH %bb.5
+
+ bb.3:
+ successors: %bb.4(0x40000000), %bb.6(0x40000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr64_sgpr65
+
+ renamable $sgpr28_sgpr29 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr28_sgpr29, implicit-def $exec, implicit-def $scc, implicit $exec
+ early-clobber renamable $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15 = S_LOAD_DWORDX8_IMM_ec renamable $sgpr4_sgpr5, 32, 0 :: (dereferenceable invariant load (s256), align 16, addrspace 4)
+ %3:vreg_64_align2 = V_LSHL_ADD_U64_e64 $sgpr64_sgpr65, 0, %1, implicit $exec
+ $exec = S_XOR_B64_term $exec, renamable $sgpr28_sgpr29, implicit-def $scc
+ S_CBRANCH_EXECZ %bb.6, implicit $exec
+ S_BRANCH %bb.4
+
+ bb.4:
+ successors: %bb.6(0x80000000)
+ liveins: $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15
+
+ S_BRANCH %bb.6
+
+ bb.5:
+ successors: %bb.3(0x80000000)
+ liveins: $sgpr4_sgpr5, $sgpr28_sgpr29, $sgpr64_sgpr65
+
+ %2:vreg_64_align2 = COPY $sgpr64_sgpr65, implicit $exec
+ S_BRANCH %bb.3
+
+ bb.6:
+ liveins: $sgpr8_sgpr9_sgpr10_sgpr11_sgpr12_sgpr13_sgpr14_sgpr15
+
+ %4:vreg_64_align2 = IMPLICIT_DEF
+ %5:vreg_64_align2 = IMPLICIT_DEF
+ %4:vreg_64_align2 = V_FMAC_F64_e32 %4, %5, %4, implicit $mode, implicit $exec
+ %6:vreg_64_align2 = V_FMA_F64_e64 1, %3, 0, %4, 0, 4607182418800017408, 0, 0, implicit $mode, implicit $exec
+ %7:vreg_64_align2 = V_LSHL_ADD_U64_e64 %2, 0, %0, implicit $exec
+ S_ENDPGM 0, implicit %6, implicit %7
+...
diff --git a/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir b/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
new file mode 100644
index 0000000000000..10aad7514eb42
--- /dev/null
+++ b/llvm/test/CodeGen/MIR/AMDGPU/bb-prolog-flag.mir
@@ -0,0 +1,17 @@
+# RUN: llc -mtriple=amdgpu9.00 -run-pass=none -o - %s | FileCheck %s
+#
+# Round-trip test for the 'bb-prolog' MIR text spelling of
+# MachineInstr::BBProlog.
+
+---
+name: bb_prolog_roundtrip
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ S_ENDPGM 0, implicit %0
+...
+
+# CHECK-LABEL: name: bb_prolog_roundtrip
+# CHECK: %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr0_sgpr1
diff --git a/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir b/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
index 6a517b7101c6e..933f45fae6b66 100644
--- a/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
+++ b/llvm/test/CodeGen/MIR/AMDGPU/lr-split-flag.mir
@@ -2,8 +2,7 @@
#
# Round-trip test for the 'lr-split' MIR text spelling of
# MachineInstr::LRSplit. The flag is set by SplitKit on copies inserted for
-# live-range splitting; AMDGPU's SIInstrInfo::canAddToBBProlog uses it to
-# recognize WWM_COPY / IMPLICIT_DEF instructions as part of the BB prologue.
+# live-range splitting.
---
name: lr_split_roundtrip
diff --git a/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir b/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
new file mode 100644
index 0000000000000..245c257643324
--- /dev/null
+++ b/llvm/test/MachineVerifier/AMDGPU/bb-prolog.mir
@@ -0,0 +1,43 @@
+# RUN: not --crash llc -mtriple=amdgpu9.00 -verify-machineinstrs -run-pass=none -filetype=null %s 2>&1 | FileCheck %s
+
+# Block prolog instructions must form a block prefix, only preceded by PHIs,
+# labels, debug instructions and pseudo probes.
+
+# CHECK-NOT: Bad machine code
+# CHECK: *** Bad machine code: Block prolog instruction after non-prolog instruction ***
+# CHECK-NEXT: - function: bad
+# CHECK-NEXT: - basic block: %bb.0
+# CHECK-NEXT: - instruction: %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64
+# CHECK-NEXT: First non-prolog instruction: %0:sreg_64 = COPY $sgpr0_sgpr1
+# CHECK-NOT: Bad machine code
+
+---
+name: good
+tracksRegLiveness: true
+body: |
+ bb.0:
+ successors: %bb.1
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = COPY $sgpr0_sgpr1
+ S_BRANCH %bb.1
+
+ bb.1:
+ %1:sreg_64 = PHI %0, %bb.0
+ DBG_VALUE
+ PSEUDO_PROBE 1, 1, 0, 0
+ %2:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %1, implicit-def $exec, implicit-def $scc, implicit $exec
+ DBG_VALUE
+ $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def $scc
+ S_ENDPGM 0
+...
+
+---
+name: bad
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $sgpr0_sgpr1
+ %0:sreg_64 = COPY $sgpr0_sgpr1
+ %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %0, implicit-def $exec, implicit-def $scc, implicit $exec
+ S_ENDPGM 0, implicit %1
+...
diff --git a/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp b/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
index ef0d40b8644f9..00847acf1125c 100644
--- a/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
+++ b/llvm/unittests/CodeGen/MachineBasicBlockTest.cpp
@@ -105,4 +105,105 @@ TEST(FindDebugLocTest, DifferentIterators) {
DIB.finalize();
}
+static MachineInstr *createMI(MachineFunction &MF, unsigned Opcode,
+ bool BBProlog = false) {
+ MCInstrDesc Desc = {Opcode, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0};
+ MachineInstr *MI = MF.CreateMachineInstr(Desc, DebugLoc());
+ if (BBProlog)
+ MI->setFlag(MachineInstr::BBProlog);
+ return MI;
+}
+
+// Instructions inserted in front of a block prolog instruction join the prolog.
+TEST(BBPrologTest, InheritBBProlog) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ MBB.push_back(Prolog);
+ MBB.push_back(Body);
+
+ MachineInstr *A = createMI(*MF, TargetOpcode::COPY);
+ MachineInstr *B = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Prolog), A);
+ MBB.insert(MachineBasicBlock::iterator(Prolog), B);
+ MBB.inheritBBProlog(A->getIterator(), Prolog->getIterator());
+ EXPECT_TRUE(A->getFlag(MachineInstr::BBProlog));
+ EXPECT_TRUE(B->getFlag(MachineInstr::BBProlog));
+
+ // At or after the end of the prolog.
+ MachineInstr *C = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Body), C);
+ MBB.inheritBBProlog(C->getIterator(), Body->getIterator());
+ EXPECT_FALSE(C->getFlag(MachineInstr::BBProlog));
+
+ MachineInstr *D = createMI(*MF, TargetOpcode::COPY);
+ MBB.push_back(D);
+ MBB.inheritBBProlog(D->getIterator(), MBB.end());
+ EXPECT_FALSE(D->getFlag(MachineInstr::BBProlog));
+}
+
+// Debug instructions, labels and pseudo probes in front of the prolog are
+// looked through, and never become prolog instructions themselves.
+TEST(BBPrologTest, InheritLooksThroughDebugLabelsAndProbes) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *Dbg = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Label = createMI(*MF, TargetOpcode::EH_LABEL);
+ MachineInstr *Probe = createMI(*MF, TargetOpcode::PSEUDO_PROBE);
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Dbg2 = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ for (MachineInstr *MI : {Dbg, Label, Probe, Prolog, Dbg2, Body})
+ MBB.push_back(MI);
+
+ MachineInstr *A = createMI(*MF, TargetOpcode::COPY);
+ MachineInstr *DbgIn = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *ProbeIn = createMI(*MF, TargetOpcode::PSEUDO_PROBE);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), A);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), DbgIn);
+ MBB.insert(MachineBasicBlock::iterator(Dbg), ProbeIn);
+ MBB.inheritBBProlog(A->getIterator(), Dbg->getIterator());
+ EXPECT_TRUE(A->getFlag(MachineInstr::BBProlog));
+ EXPECT_FALSE(DbgIn->getFlag(MachineInstr::BBProlog));
+ EXPECT_FALSE(ProbeIn->getFlag(MachineInstr::BBProlog));
+
+ MachineInstr *B = createMI(*MF, TargetOpcode::COPY);
+ MBB.insert(MachineBasicBlock::iterator(Dbg2), B);
+ MBB.inheritBBProlog(B->getIterator(), Dbg2->getIterator());
+ EXPECT_FALSE(B->getFlag(MachineInstr::BBProlog));
+}
+
+// The block-entry walks stop after the prolog.
+TEST(BBPrologTest, SkipPHIsSkipsProlog) {
+ LLVMContext Ctx;
+ Module Mod("Module", Ctx);
+ auto MF = createMachineFunction(Ctx, Mod);
+ auto &MBB = *MF->CreateMachineBasicBlock();
+
+ MachineInstr *PHI = createMI(*MF, TargetOpcode::PHI);
+ MachineInstr *Label = createMI(*MF, TargetOpcode::EH_LABEL);
+ MachineInstr *Prolog = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Dbg = createMI(*MF, TargetOpcode::DBG_VALUE);
+ MachineInstr *Prolog2 = createMI(*MF, TargetOpcode::COPY, true);
+ MachineInstr *Body = createMI(*MF, TargetOpcode::COPY);
+ for (MachineInstr *MI : {PHI, Label, Prolog, Dbg, Prolog2, Body})
+ MBB.push_back(MI);
+
+ EXPECT_EQ(&*MBB.SkipPHIsLabelsAndDebug(MBB.begin()), Body);
+ MBB.erase(Dbg);
+ EXPECT_EQ(&*MBB.SkipPHIsAndLabels(MBB.begin()), Body);
+
+ Prolog->clearFlag(MachineInstr::BBProlog);
+ Prolog2->clearFlag(MachineInstr::BBProlog);
+ EXPECT_EQ(&*MBB.SkipPHIsLabelsAndDebug(MBB.begin()), Prolog);
+ EXPECT_EQ(&*MBB.SkipPHIsAndLabels(MBB.begin()), Prolog);
+}
+
} // end namespace
diff --git a/llvm/utils/UpdateTestChecks/mir.py b/llvm/utils/UpdateTestChecks/mir.py
index ff143177e65d9..e57bad288183f 100644
--- a/llvm/utils/UpdateTestChecks/mir.py
+++ b/llvm/utils/UpdateTestChecks/mir.py
@@ -21,7 +21,7 @@
MI_FLAGS_STR = (
r"(frame-setup |frame-destroy |nnan |ninf |nsz |arcp |contract |afn "
r"|reassoc |nuw |nsw |exact |nofpexcept |nomerge |unpredictable "
- r"|noconvergent |nneg |disjoint |nusw |samesign |inbounds |lr-split "
+ r"|noconvergent |nneg |disjoint |nusw |samesign |inbounds |lr-split |bb-prolog "
r"|nonnull )*"
)
VREG_DEF_FLAGS_STR = r"(?:dead |undef )*"
>From ba11fda61b905e7ae9247114dac65af79d740df8 Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:17 +0000
Subject: [PATCH 2/3] [AMDGPU] Update tests for the BBProlog flag
MIR inputs that start after control flow lowering must carry bb-prolog on
their exec restores and SI_END_CFs; without it the prolog is no longer
inferred. The flag was added mechanically following the heuristic the
removed isBasicBlockPrologue implemented: at the block prefix, a run of
RA-inserted-looking instructions and non-terminator EXEC writes up to the
last EXEC write.
Expectations are regenerated. Most differences only print the flag. The
codegen differences are reorderings at block starts: sgpr-regalloc
insertions now follow the exec restore, and -O0 VGPR reloads precede
SGPR spill reloads that used to count as prolog.
Co-Authored-By: Claude Fable 5.1 <noreply at anthropic.com>
---
...vergence-divergent-i1-used-outside-loop.ll | 38 +-
.../GlobalISel/divergence-structurizer.ll | 24 +-
.../GlobalISel/image-waterfall-loop-O0.ll | 14 +-
.../GlobalISel/llvm.amdgcn.wqm.demote.ll | 44 +-
.../AMDGPU/NextUseAnalysis/acyclic-014bb.mir | 12 +-
.../AMDGPU/NextUseAnalysis/acyclic-770bb.mir | 768 +++++++++---------
.../acyclic-cfg-with-self-loop.mir | 24 +-
.../acyclic-phi-merge-distances.mir | 6 +-
.../complex-acyclic-cfg-with-4-self-loops.mir | 30 +-
.../complex-control-flow-11blocks.mir | 24 +-
.../complex-control-flow-15blocks.mir | 24 +-
.../NextUseAnalysis/complex-single-loop-a.mir | 24 +-
.../NextUseAnalysis/complex-single-loop.mir | 24 +-
.../double-nested-loops-complex-cfg.mir | 84 +-
...lse_with_loops_nested_in_2_outer_loops.mir | 42 +-
.../inner_cfg_in_2_nested_loops.mir | 12 +-
...op_nested_in_3_outer_loops_complex_cfg.mir | 36 +-
.../nested-loops-with-side-exits-a.mir | 48 +-
.../NextUseAnalysis/sequence_2_loops.mir | 12 +-
.../NextUseAnalysis/simple-loop-3blocks.mir | 6 +-
.../NextUseAnalysis/test_ers_basic_case.mir | 6 +-
...t_ers_do_not_spill_restore_inside_loop.mir | 18 +-
...t_ers_emit_restore_in_common_dominator.mir | 24 +-
...st_ers_emit_restore_in_loop_preheader1.mir | 6 +-
...st_ers_emit_restore_in_loop_preheader3.mir | 48 +-
...st_ers_emit_restore_in_loop_preheader4.mir | 12 +-
.../test_ers_keep_spilled_reg_live.mir | 6 +-
.../test_ers_multiple_spills1.mir | 24 +-
.../test_ers_multiple_spills2.mir | 18 +-
.../test_ers_multiple_spills3.mir | 30 +-
...common_dominator_and_optimize_restores.mir | 6 +-
.../test_ers_spill_loop_livethrough_reg.mir | 24 +-
...est_ers_spill_loop_value_in_exit_block.mir | 6 +-
.../NextUseAnalysis/triple-nested-loops.mir | 18 +-
.../NextUseAnalysis/two-sequential-loops.mir | 18 +-
.../AMDGPU/SRSRC-GIT-clobber-check.mir | 2 +-
.../AMDGPU/bb-prolog-spill-during-regalloc.ll | 16 +-
.../block-should-not-be-in-alive-blocks.mir | 6 +-
.../branch-folding-implicit-def-subreg.ll | 40 +-
.../coalescer-extend-pruned-subrange.mir | 4 +-
...scer-subranges-another-copymi-not-live.mir | 2 +-
...oalescer-subranges-another-prune-error.mir | 12 +-
.../AMDGPU/coalescer-subregjoin-fullcopy.mir | 4 +-
.../coalescer-with-subregs-bad-identical.mir | 4 +-
.../coalescing-with-subregs-in-loop-bug.mir | 2 +-
.../codegen-prepare-addrspacecast-non-null.ll | 14 +-
.../CodeGen/AMDGPU/collapse-endcf-broken.mir | 8 +-
llvm/test/CodeGen/AMDGPU/collapse-endcf.ll | 60 +-
llvm/test/CodeGen/AMDGPU/collapse-endcf.mir | 100 +--
llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir | 8 +-
.../AMDGPU/constant-fold-imm-immreg.mir | 2 +-
llvm/test/CodeGen/AMDGPU/convergent.mir | 36 +-
.../AMDGPU/cross-block-use-is-not-abi-copy.ll | 2 +-
llvm/test/CodeGen/AMDGPU/div_i128.ll | 158 ++--
llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir | 6 +-
llvm/test/CodeGen/AMDGPU/dpp_combine.mir | 2 +-
.../test/CodeGen/AMDGPU/dpp_combine_gfx11.mir | 2 +-
.../AMDGPU/extend-wwm-virt-reg-liveness.mir | 4 +-
.../CodeGen/AMDGPU/extract_vector_dynelt.ll | 117 ++-
.../fold-operands-s-add-copy-to-vgpr.mir | 4 +-
.../CodeGen/AMDGPU/fold-restore-undef-use.mir | 4 +-
.../greedy-alloc-fail-sgpr1024-spill.mir | 8 +-
.../greedy-broken-ssa-verifier-error.mir | 2 +-
.../CodeGen/AMDGPU/indirect-addressing-si.ll | 277 +++----
.../CodeGen/AMDGPU/insert-skip-from-vcc.mir | 10 +-
.../CodeGen/AMDGPU/insert-waitcnts-merge.ll | 4 +-
.../CodeGen/AMDGPU/insert_vector_dynelt.ll | 134 ++-
llvm/test/CodeGen/AMDGPU/licm-wwm.mir | 4 +-
...rval-bug-in-rename-independent-subregs.mir | 12 +-
.../AMDGPU/loop-header-align-gfx950.mir | 4 +-
.../lower-control-flow-live-intervals.mir | 22 +-
...wer-control-flow-live-variables-update.mir | 4 +-
.../lower-control-flow-other-terminators.mir | 2 +-
.../AMDGPU/lower-i1-copies-clear-kills.mir | 6 +-
...ne-scheduler-rematerialization-scoring.mir | 12 +-
.../machine-scheduler-sink-trivial-remats.mir | 48 +-
.../machine-sink-ignorable-exec-use.mir | 16 +-
.../CodeGen/AMDGPU/machine-sink-lane-mask.mir | 6 +-
...-var-out-of-divergent-loop-swdev407790.mir | 4 +-
...e-sink-temporal-divergence-swdev407790.mir | 16 +-
.../memory-legalizer-atomic-insert-end.mir | 4 +-
llvm/test/CodeGen/AMDGPU/merge-m0.mir | 6 +-
...uf-legalize-operands-non-ptr-intrinsics.ll | 125 ++-
.../CodeGen/AMDGPU/mubuf-legalize-operands.ll | 134 ++-
.../CodeGen/AMDGPU/no-remat-indirect-mov.mir | 4 +-
.../CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir | 6 +-
...ptimize-exec-mask-pre-ra-alloc-failure.mir | 4 +-
...ptimize-exec-mask-pre-ra-def-after-use.mir | 10 +-
...ask-pre-ra-non-empty-but-used-interval.mir | 4 +-
.../AMDGPU/optimize-exec-masking-pre-ra.mir | 16 +-
.../AMDGPU/optimize-if-exec-masking.mir | 22 +-
.../AMDGPU/partial-sgpr-to-vgpr-spills.ll | 33 +-
.../CodeGen/AMDGPU/phi-elimination-end-cf.mir | 6 +-
.../AMDGPU/postra-sink-update-dependency.mir | 6 +-
.../ran-out-of-sgprs-allocation-failure.mir | 72 +-
llvm/test/CodeGen/AMDGPU/readlane_exec0.mir | 2 +-
...oc-hoist-spill-live-range-interference.mir | 13 +-
.../regalloc-hoist-spill-live-range-upd.mir | 4 +-
llvm/test/CodeGen/AMDGPU/rem_i128.ll | 78 +-
...hort-exec-branch-on-unconditional-jump.mir | 12 +-
.../AMDGPU/s_or_saveexec_xor_combine.mir | 24 +-
.../CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll | 7 +-
.../AMDGPU/sgpr-spill-overlap-wwm-reserve.mir | 150 ++--
.../CodeGen/AMDGPU/si-fix-sgpr-copies.mir | 2 +-
.../CodeGen/AMDGPU/si-init-whole-wave.mir | 8 +-
...i-lower-control-flow-preserve-dom-tree.mir | 4 +-
...w-remove-redundant-block-liveintervals.mir | 6 +-
...lower-i1-copies-order-of-phi-incomings.mir | 2 +-
.../CodeGen/AMDGPU/si-lower-i1-copies.mir | 4 +-
.../si-opt-vgpr-liverange-bug-deadlanes.mir | 2 +-
...-opt-vgpr-liverange-undef-use-livevars.mir | 2 +-
.../si-optimize-vgpr-live-range-dbg-instr.mir | 2 +-
.../si-pre-allocate-wwm-regs-preserve-rci.mir | 4 +-
.../AMDGPU/sink-after-control-flow-postra.mir | 48 +-
.../AMDGPU/sink-after-control-flow.mir | 12 +-
.../AMDGPU/skip-branch-taildup-ret.mir | 24 +-
.../test/CodeGen/AMDGPU/spill-before-exec.mir | 6 +-
.../CodeGen/AMDGPU/spill-scavenge-offset.ll | 459 ++++++-----
.../AMDGPU/stale-livevar-in-twoaddr-pass.mir | 2 +-
.../stop-tail-duplicate-cfg-intrinsic.mir | 2 +-
.../AMDGPU/subreg-split-live-in-error.mir | 2 +-
.../transform-block-with-return-to-epilog.ll | 6 +-
llvm/test/CodeGen/AMDGPU/trap-abis.ll | 5 +-
.../AMDGPU/undefined-physreg-sgpr-spill.mir | 8 +-
.../CodeGen/AMDGPU/v-cmp-cse-across-loop.mir | 2 +-
.../AMDGPU/vgpr-mark-last-scratch-load.mir | 8 +-
.../waitcnt-kmcnt-scc-different-block.mir | 2 +-
...terfall-call-target-av-register-failure.ll | 11 +-
.../AMDGPU/whole-wave-functions-pei.mir | 6 +-
.../AMDGPU/wqm-debug-instr-terminator.mir | 6 +-
llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir | 2 +-
.../wqm-propagate-for-execz-side-effect.mir | 24 +-
llvm/test/CodeGen/AMDGPU/wqm-terminators.mir | 2 +-
llvm/test/CodeGen/AMDGPU/wqm.mir | 79 +-
.../wwm-regalloc-preallocation-guard.mir | 4 +-
.../test/CodeGen/AMDGPU/wwm-reserved-spill.ll | 4 +-
llvm/test/CodeGen/AMDGPU/wwm-reserved.ll | 8 +-
.../AMDGPU/wwm-spill-superclass-pseudo.mir | 2 +-
138 files changed, 2139 insertions(+), 2143 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-divergent-i1-used-outside-loop.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-divergent-i1-used-outside-loop.ll
index 03f235a1ee26f..c5c2de013174f 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-divergent-i1-used-outside-loop.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-divergent-i1-used-outside-loop.ll
@@ -344,23 +344,23 @@ define void @divergent_i1_icmp_used_outside_loop(i32 %v0, i32 %v1, ptr addrspace
; GFX10-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GFX10-NEXT: s_mov_b32 s4, 0
; GFX10-NEXT: s_mov_b32 s6, 0
-; GFX10-NEXT: ; implicit-def: $sgpr7
+; GFX10-NEXT: ; implicit-def: $sgpr5
; GFX10-NEXT: s_branch .LBB5_2
; GFX10-NEXT: .LBB5_1: ; %Flow
; GFX10-NEXT: ; in Loop: Header=BB5_2 Depth=1
-; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s8
+; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s9
; GFX10-NEXT: s_and_b32 s5, exec_lo, s5
; GFX10-NEXT: s_or_b32 s6, s5, s6
+; GFX10-NEXT: s_or_b32 s5, s7, s8
; GFX10-NEXT: s_andn2_b32 exec_lo, exec_lo, s6
; GFX10-NEXT: s_cbranch_execz .LBB5_6
; GFX10-NEXT: .LBB5_2: ; %cond.block.0
; GFX10-NEXT: ; =>This Inner Loop Header: Depth=1
; GFX10-NEXT: v_cmp_eq_u32_e32 vcc_lo, s4, v0
; GFX10-NEXT: v_mov_b32_e32 v4, s4
-; GFX10-NEXT: s_andn2_b32 s5, s7, exec_lo
-; GFX10-NEXT: s_and_b32 s7, exec_lo, vcc_lo
-; GFX10-NEXT: s_or_b32 s7, s5, s7
-; GFX10-NEXT: s_and_saveexec_b32 s8, vcc_lo
+; GFX10-NEXT: s_andn2_b32 s7, s5, exec_lo
+; GFX10-NEXT: s_and_b32 s8, exec_lo, vcc_lo
+; GFX10-NEXT: s_and_saveexec_b32 s9, vcc_lo
; GFX10-NEXT: s_cbranch_execz .LBB5_4
; GFX10-NEXT: ; %bb.3: ; %if.block.0
; GFX10-NEXT: ; in Loop: Header=BB5_2 Depth=1
@@ -375,10 +375,10 @@ define void @divergent_i1_icmp_used_outside_loop(i32 %v0, i32 %v1, ptr addrspace
; GFX10-NEXT: .LBB5_4: ; %loop.break.block
; GFX10-NEXT: ; in Loop: Header=BB5_2 Depth=1
; GFX10-NEXT: s_waitcnt_depctr depctr_vm_vsrc(0)
-; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s8
+; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s9
; GFX10-NEXT: v_cmp_ne_u32_e32 vcc_lo, s4, v1
; GFX10-NEXT: s_mov_b32 s5, exec_lo
-; GFX10-NEXT: s_and_saveexec_b32 s8, vcc_lo
+; GFX10-NEXT: s_and_saveexec_b32 s9, vcc_lo
; GFX10-NEXT: s_cbranch_execz .LBB5_1
; GFX10-NEXT: ; %bb.5: ; %loop.cond
; GFX10-NEXT: ; in Loop: Header=BB5_2 Depth=1
@@ -387,7 +387,7 @@ define void @divergent_i1_icmp_used_outside_loop(i32 %v0, i32 %v1, ptr addrspace
; GFX10-NEXT: s_branch .LBB5_1
; GFX10-NEXT: .LBB5_6: ; %cond.block.1
; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s6
-; GFX10-NEXT: s_and_saveexec_b32 s4, s7
+; GFX10-NEXT: s_and_saveexec_b32 s4, s5
; GFX10-NEXT: s_cbranch_execz .LBB5_8
; GFX10-NEXT: ; %bb.7: ; %if.block.1
; GFX10-NEXT: global_store_dword v[6:7], v4, off
@@ -454,34 +454,34 @@ define amdgpu_ps void @divergent_i1_freeze_used_outside_loop(i32 %n, ptr addrspa
; GFX10-NEXT: s_mov_b32 s1, exec_lo
; GFX10-NEXT: s_mov_b32 s0, 0
; GFX10-NEXT: s_mov_b32 s2, 0
-; GFX10-NEXT: ; implicit-def: $sgpr4
; GFX10-NEXT: ; implicit-def: $sgpr3
+; GFX10-NEXT: ; implicit-def: $sgpr4
; GFX10-NEXT: s_branch .LBB6_2
; GFX10-NEXT: .LBB6_1: ; %loop.cond
; GFX10-NEXT: ; in Loop: Header=BB6_2 Depth=1
; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s5
; GFX10-NEXT: v_cmp_lt_i32_e32 vcc_lo, s0, v0
-; GFX10-NEXT: s_andn2_b32 s3, s3, exec_lo
-; GFX10-NEXT: s_and_b32 s5, exec_lo, s4
+; GFX10-NEXT: s_andn2_b32 s4, s4, exec_lo
+; GFX10-NEXT: s_and_b32 s5, exec_lo, s3
; GFX10-NEXT: s_andn2_b32 s1, s1, exec_lo
; GFX10-NEXT: s_add_i32 s0, s0, 1
; GFX10-NEXT: s_or_b32 s2, vcc_lo, s2
-; GFX10-NEXT: s_or_b32 s3, s3, s5
+; GFX10-NEXT: s_or_b32 s4, s4, s5
; GFX10-NEXT: s_or_b32 s1, s1, s5
; GFX10-NEXT: s_andn2_b32 exec_lo, exec_lo, s2
; GFX10-NEXT: s_cbranch_execz .LBB6_4
; GFX10-NEXT: .LBB6_2: ; %loop.start
; GFX10-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX10-NEXT: s_andn2_b32 s4, s4, exec_lo
+; GFX10-NEXT: s_andn2_b32 s3, s3, exec_lo
; GFX10-NEXT: s_and_b32 s5, exec_lo, s1
-; GFX10-NEXT: s_or_b32 s4, s4, s5
+; GFX10-NEXT: s_or_b32 s3, s3, s5
; GFX10-NEXT: s_and_saveexec_b32 s5, s1
; GFX10-NEXT: s_cbranch_execz .LBB6_1
; GFX10-NEXT: ; %bb.3: ; %is.eq.zero
; GFX10-NEXT: ; in Loop: Header=BB6_2 Depth=1
; GFX10-NEXT: s_ashr_i32 s1, s0, 31
; GFX10-NEXT: s_lshl_b64 s[6:7], s[0:1], 2
-; GFX10-NEXT: s_andn2_b32 s1, s4, exec_lo
+; GFX10-NEXT: s_andn2_b32 s1, s3, exec_lo
; GFX10-NEXT: v_mov_b32_e32 v5, s6
; GFX10-NEXT: v_mov_b32_e32 v6, s7
; GFX10-NEXT: v_add_co_u32 v5, vcc_lo, v1, v5
@@ -489,13 +489,13 @@ define amdgpu_ps void @divergent_i1_freeze_used_outside_loop(i32 %n, ptr addrspa
; GFX10-NEXT: global_load_dword v5, v[5:6], off
; GFX10-NEXT: s_waitcnt vmcnt(0)
; GFX10-NEXT: v_cmp_eq_u32_e32 vcc_lo, 0, v5
-; GFX10-NEXT: s_and_b32 s4, exec_lo, vcc_lo
-; GFX10-NEXT: s_or_b32 s4, s1, s4
+; GFX10-NEXT: s_and_b32 s3, exec_lo, vcc_lo
+; GFX10-NEXT: s_or_b32 s3, s1, s3
; GFX10-NEXT: ; implicit-def: $sgpr1
; GFX10-NEXT: s_branch .LBB6_1
; GFX10-NEXT: .LBB6_4: ; %exit
; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s2
-; GFX10-NEXT: v_cndmask_b32_e64 v0, 0, 1.0, s3
+; GFX10-NEXT: v_cndmask_b32_e64 v0, 0, 1.0, s4
; GFX10-NEXT: flat_store_dword v[3:4], v0
; GFX10-NEXT: s_endpgm
entry:
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-structurizer.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-structurizer.ll
index 6f3245a39a9a7..3a4da937b5a5d 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-structurizer.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/divergence-structurizer.ll
@@ -485,17 +485,18 @@ define amdgpu_ps i32 @irreducible_cfg(i32 %x, i32 %y, i32 %a0, i32 %a1, i32 %a2,
; GFX10-NEXT: s_mov_b32 s0, exec_lo
; GFX10-NEXT: s_mov_b32 s1, 0
; GFX10-NEXT: s_and_b32 s2, s0, 1
+; GFX10-NEXT: ; implicit-def: $sgpr3
; GFX10-NEXT: s_xor_b32 s0, vcc_lo, s0
; GFX10-NEXT: s_cmp_lg_u32 s2, 0
-; GFX10-NEXT: ; implicit-def: $sgpr2
-; GFX10-NEXT: s_cselect_b32 s3, exec_lo, 0
+; GFX10-NEXT: s_cselect_b32 s2, exec_lo, 0
; GFX10-NEXT: s_branch .LBB6_2
; GFX10-NEXT: .LBB6_1: ; %Flow2
; GFX10-NEXT: ; in Loop: Header=BB6_2 Depth=1
-; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s6
+; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s7
; GFX10-NEXT: s_and_b32 s4, exec_lo, s5
; GFX10-NEXT: s_mov_b32 s0, exec_lo
; GFX10-NEXT: s_or_b32 s1, s4, s1
+; GFX10-NEXT: s_or_b32 s3, s3, s6
; GFX10-NEXT: s_andn2_b32 exec_lo, exec_lo, s1
; GFX10-NEXT: s_cbranch_execz .LBB6_8
; GFX10-NEXT: .LBB6_2: ; %irr.guard
@@ -510,22 +511,21 @@ define amdgpu_ps i32 @irreducible_cfg(i32 %x, i32 %y, i32 %a0, i32 %a1, i32 %a2,
; GFX10-NEXT: s_mov_b32 s6, exec_lo
; GFX10-NEXT: s_mov_b32 s7, exec_lo
; GFX10-NEXT: s_xor_b32 s6, vcc_lo, s6
-; GFX10-NEXT: s_andn2_b32 s3, s3, exec_lo
+; GFX10-NEXT: s_andn2_b32 s2, s2, exec_lo
; GFX10-NEXT: s_or_b32 s6, s0, s6
; GFX10-NEXT: s_and_b32 s0, exec_lo, s0
; GFX10-NEXT: s_xor_b32 s6, s6, s7
; GFX10-NEXT: s_andn2_b32 s4, s4, exec_lo
; GFX10-NEXT: s_and_b32 s6, exec_lo, s6
-; GFX10-NEXT: s_or_b32 s3, s3, s0
+; GFX10-NEXT: s_or_b32 s2, s2, s0
; GFX10-NEXT: s_or_b32 s4, s4, s6
; GFX10-NEXT: ; %bb.4: ; %Flow1
; GFX10-NEXT: ; in Loop: Header=BB6_2 Depth=1
; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s5
-; GFX10-NEXT: s_andn2_b32 s0, s2, exec_lo
-; GFX10-NEXT: s_and_b32 s2, exec_lo, s3
; GFX10-NEXT: s_mov_b32 s5, exec_lo
-; GFX10-NEXT: s_or_b32 s2, s0, s2
-; GFX10-NEXT: s_and_saveexec_b32 s6, s4
+; GFX10-NEXT: s_andn2_b32 s3, s3, exec_lo
+; GFX10-NEXT: s_and_b32 s6, exec_lo, s2
+; GFX10-NEXT: s_and_saveexec_b32 s7, s4
; GFX10-NEXT: s_cbranch_execz .LBB6_1
; GFX10-NEXT: ; %bb.5: ; %.preheader
; GFX10-NEXT: ; in Loop: Header=BB6_2 Depth=1
@@ -534,8 +534,8 @@ define amdgpu_ps i32 @irreducible_cfg(i32 %x, i32 %y, i32 %a0, i32 %a1, i32 %a2,
; GFX10-NEXT: .LBB6_6: ; %.inner_loop
; GFX10-NEXT: ; Parent Loop BB6_2 Depth=1
; GFX10-NEXT: ; => This Inner Loop Header: Depth=2
-; GFX10-NEXT: s_and_b32 s7, exec_lo, s0
-; GFX10-NEXT: s_or_b32 s4, s7, s4
+; GFX10-NEXT: s_and_b32 s8, exec_lo, s0
+; GFX10-NEXT: s_or_b32 s4, s8, s4
; GFX10-NEXT: s_andn2_b32 exec_lo, exec_lo, s4
; GFX10-NEXT: s_cbranch_execnz .LBB6_6
; GFX10-NEXT: ; %bb.7: ; %Flow
@@ -545,7 +545,7 @@ define amdgpu_ps i32 @irreducible_cfg(i32 %x, i32 %y, i32 %a0, i32 %a1, i32 %a2,
; GFX10-NEXT: s_branch .LBB6_1
; GFX10-NEXT: .LBB6_8: ; %.exit
; GFX10-NEXT: s_or_b32 exec_lo, exec_lo, s1
-; GFX10-NEXT: v_cndmask_b32_e64 v0, v2, v3, s2
+; GFX10-NEXT: v_cndmask_b32_e64 v0, v2, v3, s3
; GFX10-NEXT: v_readfirstlane_b32 s0, v0
; GFX10-NEXT: ; return to shader part epilog
.entry:
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
index 33d7bd4c75ace..cd304a7fd931f 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/image-waterfall-loop-O0.ll
@@ -66,9 +66,6 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_store_dword v16, off, s[0:3], s32 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; CHECK-NEXT: s_or_saveexec_b32 s21, -1
-; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
-; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
@@ -85,7 +82,10 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; CHECK-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(0)
+; CHECK-NEXT: s_or_saveexec_b32 s21, -1
+; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
+; CHECK-NEXT: s_mov_b32 exec_lo, s21
+; CHECK-NEXT: s_waitcnt vmcnt(1)
; CHECK-NEXT: v_readfirstlane_b32 s12, v7
; CHECK-NEXT: v_readfirstlane_b32 s10, v6
; CHECK-NEXT: v_readfirstlane_b32 s9, v5
@@ -102,6 +102,7 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: s_mov_b32 s17, s6
; CHECK-NEXT: s_mov_b32 s18, s5
; CHECK-NEXT: s_mov_b32 s19, s4
+; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: v_writelane_b32 v16, s12, 5
; CHECK-NEXT: v_writelane_b32 v16, s13, 6
; CHECK-NEXT: v_writelane_b32 v16, s14, 7
@@ -135,6 +136,8 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: buffer_store_dword v16, off, s[0:3], s32 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b32 exec_lo, s21
; CHECK-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b32 s21, -1
; CHECK-NEXT: buffer_load_dword v16, off, s[0:3], s32 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b32 exec_lo, s21
@@ -152,9 +155,6 @@ define <4 x float> @waterfall_loop(<8 x i32> %vgpr_srd) {
; CHECK-NEXT: v_readlane_b32 s17, v16, 1
; CHECK-NEXT: v_readlane_b32 s18, v16, 2
; CHECK-NEXT: v_readlane_b32 s19, v16, 3
-; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: image_sample v0, [v0, v1], s[8:15], s[16:19] dmask:0x1 dim:SQ_RSRC_IMG_2D
; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
diff --git a/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll b/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
index 02b68822047c3..7b817e1c91eaa 100644
--- a/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
+++ b/llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.wqm.demote.ll
@@ -667,12 +667,10 @@ define amdgpu_ps void @wqm_deriv(<2 x float> %input, float %arg, i32 %index) {
; SI-NEXT: s_wqm_b64 exec, exec
; SI-NEXT: v_cvt_i32_f32_e32 v0, v0
; SI-NEXT: s_movk_i32 s2, 0x3c00
-; SI-NEXT: s_bfe_u32 s3, 0, 0x100000
-; SI-NEXT: s_bfe_u32 s2, s2, 0x100000
-; SI-NEXT: s_lshl_b32 s4, s3, 16
-; SI-NEXT: s_or_b32 s4, s2, s4
-; SI-NEXT: s_lshl_b32 s2, s2, 16
-; SI-NEXT: s_or_b32 s5, s3, s2
+; SI-NEXT: s_bfe_u32 s4, s2, 0x100000
+; SI-NEXT: s_bfe_u32 s5, 0, 0x100000
+; SI-NEXT: s_lshl_b32 s6, s5, 16
+; SI-NEXT: s_lshl_b32 s7, s4, 16
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; SI-NEXT: s_and_saveexec_b64 s[2:3], vcc
; SI-NEXT: s_xor_b64 s[2:3], exec, s[2:3]
@@ -681,10 +679,12 @@ define amdgpu_ps void @wqm_deriv(<2 x float> %input, float %arg, i32 %index) {
; SI-NEXT: s_andn2_b64 s[0:1], s[0:1], exec
; SI-NEXT: s_cbranch_scc0 .LBB6_7
; SI-NEXT: ; %bb.2: ; %.demote0
-; SI-NEXT: s_wqm_b64 s[6:7], s[0:1]
-; SI-NEXT: s_and_b64 exec, exec, s[6:7]
+; SI-NEXT: s_wqm_b64 s[8:9], s[0:1]
+; SI-NEXT: s_and_b64 exec, exec, s[8:9]
; SI-NEXT: .LBB6_3: ; %.continue0
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
+; SI-NEXT: s_or_b32 s4, s4, s6
+; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: s_mov_b64 s[6:7], s[0:1]
; SI-NEXT: v_cndmask_b32_e64 v0, 1.0, 0, s[6:7]
; SI-NEXT: v_mov_b32_e32 v1, v0
@@ -901,13 +901,11 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: s_wqm_b64 exec, exec
; SI-NEXT: v_cvt_i32_f32_e32 v0, v0
; SI-NEXT: s_movk_i32 s2, 0x3c00
-; SI-NEXT: s_bfe_u32 s3, 0, 0x100000
-; SI-NEXT: s_bfe_u32 s2, s2, 0x100000
-; SI-NEXT: s_lshl_b32 s4, s3, 16
-; SI-NEXT: s_or_b32 s6, s2, s4
-; SI-NEXT: s_lshl_b32 s2, s2, 16
-; SI-NEXT: s_or_b32 s7, s3, s2
-; SI-NEXT: s_mov_b32 s8, 0
+; SI-NEXT: s_bfe_u32 s4, s2, 0x100000
+; SI-NEXT: s_bfe_u32 s5, 0, 0x100000
+; SI-NEXT: s_lshl_b32 s7, s5, 16
+; SI-NEXT: s_lshl_b32 s8, s4, 16
+; SI-NEXT: s_mov_b32 s6, 0
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
; SI-NEXT: s_and_saveexec_b64 s[2:3], vcc
; SI-NEXT: s_xor_b64 s[2:3], exec, s[2:3]
@@ -916,23 +914,25 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: s_andn2_b64 s[0:1], s[0:1], exec
; SI-NEXT: s_cbranch_scc0 .LBB7_9
; SI-NEXT: ; %bb.2: ; %.demote0
-; SI-NEXT: s_wqm_b64 s[4:5], s[0:1]
-; SI-NEXT: s_and_b64 exec, exec, s[4:5]
+; SI-NEXT: s_wqm_b64 s[10:11], s[0:1]
+; SI-NEXT: s_and_b64 exec, exec, s[10:11]
; SI-NEXT: .LBB7_3: ; %.continue0.preheader
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
+; SI-NEXT: s_or_b32 s7, s4, s7
+; SI-NEXT: s_or_b32 s8, s5, s8
; SI-NEXT: s_mov_b64 s[2:3], 0
; SI-NEXT: s_branch .LBB7_5
; SI-NEXT: .LBB7_4: ; %.continue1
; SI-NEXT: ; in Loop: Header=BB7_5 Depth=1
; SI-NEXT: s_or_b64 exec, exec, s[4:5]
-; SI-NEXT: s_add_i32 s8, s8, 1
-; SI-NEXT: v_cmp_ge_i32_e32 vcc, s8, v1
+; SI-NEXT: s_add_i32 s6, s6, 1
+; SI-NEXT: v_cmp_ge_i32_e32 vcc, s6, v1
; SI-NEXT: s_or_b64 s[2:3], vcc, s[2:3]
; SI-NEXT: s_andn2_b64 exec, exec, s[2:3]
; SI-NEXT: s_cbranch_execz .LBB7_8
; SI-NEXT: .LBB7_5: ; %.continue0
; SI-NEXT: ; =>This Inner Loop Header: Depth=1
-; SI-NEXT: v_mov_b32_e32 v0, s8
+; SI-NEXT: v_mov_b32_e32 v0, s6
; SI-NEXT: s_mov_b64 s[10:11], s[0:1]
; SI-NEXT: v_cndmask_b32_e64 v0, v0, 0, s[10:11]
; SI-NEXT: v_mov_b32_e32 v2, v0
@@ -960,8 +960,8 @@ define amdgpu_ps void @wqm_deriv_loop(<2 x float> %input, float %arg, i32 %index
; SI-NEXT: .LBB7_8: ; %.return
; SI-NEXT: s_or_b64 exec, exec, s[2:3]
; SI-NEXT: s_and_b64 exec, exec, s[0:1]
-; SI-NEXT: v_mov_b32_e32 v0, s6
-; SI-NEXT: v_mov_b32_e32 v1, s7
+; SI-NEXT: v_mov_b32_e32 v0, s7
+; SI-NEXT: v_mov_b32_e32 v1, s8
; SI-NEXT: exp mrt0, v0, v0, v1, v1 done compr vm
; SI-NEXT: s_endpgm
; SI-NEXT: .LBB7_9:
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
index d0fbac674e523..b576c3f185775 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-014bb.mir
@@ -89,7 +89,7 @@ body: |
successors: %bb.7(0x80000000)
%28:vgpr_32 = PHI %25, %bb.4, %27, %bb.5
- SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.7:
successors: %bb.8(0x80000000)
@@ -134,7 +134,7 @@ body: |
successors: %bb.13(0x80000000)
%44:vgpr_32 = PHI %41, %bb.10, %43, %bb.11
- SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.13:
@@ -933,7 +933,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1455,7 +1455,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2449,7 +2449,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2967,7 +2967,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
index 1085000dd83ec..d9befc88206e0 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-770bb.mir
@@ -89,7 +89,7 @@ body: |
successors: %bb.7(0x80000000)
%28:vgpr_32 = PHI %25, %bb.4, %27, %bb.5
- SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.7:
successors: %bb.8(0x80000000)
@@ -134,7 +134,7 @@ body: |
successors: %bb.13(0x80000000)
%44:vgpr_32 = PHI %41, %bb.10, %43, %bb.11
- SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.13:
successors: %bb.14(0x80000000)
@@ -179,7 +179,7 @@ body: |
successors: %bb.19(0x80000000)
%60:vgpr_32 = PHI %57, %bb.16, %59, %bb.17
- SI_END_CF killed %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.19:
successors: %bb.20(0x80000000)
@@ -224,7 +224,7 @@ body: |
successors: %bb.25(0x80000000)
%76:vgpr_32 = PHI %73, %bb.22, %75, %bb.23
- SI_END_CF killed %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.25:
successors: %bb.26(0x80000000)
@@ -269,7 +269,7 @@ body: |
successors: %bb.31(0x80000000)
%92:vgpr_32 = PHI %89, %bb.28, %91, %bb.29
- SI_END_CF killed %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.31:
successors: %bb.32(0x80000000)
@@ -314,7 +314,7 @@ body: |
successors: %bb.37(0x80000000)
%108:vgpr_32 = PHI %105, %bb.34, %107, %bb.35
- SI_END_CF killed %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.37:
successors: %bb.38(0x80000000)
@@ -359,7 +359,7 @@ body: |
successors: %bb.43(0x80000000)
%124:vgpr_32 = PHI %121, %bb.40, %123, %bb.41
- SI_END_CF killed %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.43:
successors: %bb.44(0x80000000)
@@ -404,7 +404,7 @@ body: |
successors: %bb.49(0x80000000)
%140:vgpr_32 = PHI %137, %bb.46, %139, %bb.47
- SI_END_CF killed %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.49:
successors: %bb.50(0x80000000)
@@ -449,7 +449,7 @@ body: |
successors: %bb.55(0x80000000)
%156:vgpr_32 = PHI %153, %bb.52, %155, %bb.53
- SI_END_CF killed %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.55:
successors: %bb.56(0x80000000)
@@ -494,7 +494,7 @@ body: |
successors: %bb.61(0x80000000)
%172:vgpr_32 = PHI %169, %bb.58, %171, %bb.59
- SI_END_CF killed %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.61:
successors: %bb.62(0x80000000)
@@ -539,7 +539,7 @@ body: |
successors: %bb.67(0x80000000)
%188:vgpr_32 = PHI %185, %bb.64, %187, %bb.65
- SI_END_CF killed %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.67:
successors: %bb.68(0x80000000)
@@ -584,7 +584,7 @@ body: |
successors: %bb.73(0x80000000)
%204:vgpr_32 = PHI %201, %bb.70, %203, %bb.71
- SI_END_CF killed %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.73:
successors: %bb.74(0x80000000)
@@ -629,7 +629,7 @@ body: |
successors: %bb.79(0x80000000)
%220:vgpr_32 = PHI %217, %bb.76, %219, %bb.77
- SI_END_CF killed %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.79:
successors: %bb.80(0x80000000)
@@ -674,7 +674,7 @@ body: |
successors: %bb.85(0x80000000)
%236:vgpr_32 = PHI %233, %bb.82, %235, %bb.83
- SI_END_CF killed %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.85:
successors: %bb.86(0x80000000)
@@ -719,7 +719,7 @@ body: |
successors: %bb.91(0x80000000)
%252:vgpr_32 = PHI %249, %bb.88, %251, %bb.89
- SI_END_CF killed %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.91:
successors: %bb.92(0x80000000)
@@ -764,7 +764,7 @@ body: |
successors: %bb.97(0x80000000)
%268:vgpr_32 = PHI %265, %bb.94, %267, %bb.95
- SI_END_CF killed %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.97:
successors: %bb.98(0x80000000)
@@ -809,7 +809,7 @@ body: |
successors: %bb.103(0x80000000)
%284:vgpr_32 = PHI %281, %bb.100, %283, %bb.101
- SI_END_CF killed %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.103:
successors: %bb.104(0x80000000)
@@ -854,7 +854,7 @@ body: |
successors: %bb.109(0x80000000)
%300:vgpr_32 = PHI %297, %bb.106, %299, %bb.107
- SI_END_CF killed %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.109:
successors: %bb.110(0x80000000)
@@ -899,7 +899,7 @@ body: |
successors: %bb.115(0x80000000)
%316:vgpr_32 = PHI %313, %bb.112, %315, %bb.113
- SI_END_CF killed %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.115:
successors: %bb.116(0x80000000)
@@ -944,7 +944,7 @@ body: |
successors: %bb.121(0x80000000)
%332:vgpr_32 = PHI %329, %bb.118, %331, %bb.119
- SI_END_CF killed %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.121:
successors: %bb.122(0x80000000)
@@ -989,7 +989,7 @@ body: |
successors: %bb.127(0x80000000)
%348:vgpr_32 = PHI %345, %bb.124, %347, %bb.125
- SI_END_CF killed %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.127:
successors: %bb.128(0x80000000)
@@ -1034,7 +1034,7 @@ body: |
successors: %bb.133(0x80000000)
%364:vgpr_32 = PHI %361, %bb.130, %363, %bb.131
- SI_END_CF killed %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.133:
successors: %bb.134(0x80000000)
@@ -1079,7 +1079,7 @@ body: |
successors: %bb.139(0x80000000)
%380:vgpr_32 = PHI %377, %bb.136, %379, %bb.137
- SI_END_CF killed %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.139:
successors: %bb.140(0x80000000)
@@ -1124,7 +1124,7 @@ body: |
successors: %bb.145(0x80000000)
%396:vgpr_32 = PHI %393, %bb.142, %395, %bb.143
- SI_END_CF killed %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.145:
successors: %bb.146(0x80000000)
@@ -1169,7 +1169,7 @@ body: |
successors: %bb.151(0x80000000)
%412:vgpr_32 = PHI %409, %bb.148, %411, %bb.149
- SI_END_CF killed %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.151:
successors: %bb.152(0x80000000)
@@ -1214,7 +1214,7 @@ body: |
successors: %bb.157(0x80000000)
%428:vgpr_32 = PHI %425, %bb.154, %427, %bb.155
- SI_END_CF killed %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.157:
successors: %bb.158(0x80000000)
@@ -1259,7 +1259,7 @@ body: |
successors: %bb.163(0x80000000)
%444:vgpr_32 = PHI %441, %bb.160, %443, %bb.161
- SI_END_CF killed %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.163:
successors: %bb.164(0x80000000)
@@ -1304,7 +1304,7 @@ body: |
successors: %bb.169(0x80000000)
%460:vgpr_32 = PHI %457, %bb.166, %459, %bb.167
- SI_END_CF killed %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.169:
successors: %bb.170(0x80000000)
@@ -1349,7 +1349,7 @@ body: |
successors: %bb.175(0x80000000)
%476:vgpr_32 = PHI %473, %bb.172, %475, %bb.173
- SI_END_CF killed %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.175:
successors: %bb.176(0x80000000)
@@ -1394,7 +1394,7 @@ body: |
successors: %bb.181(0x80000000)
%492:vgpr_32 = PHI %489, %bb.178, %491, %bb.179
- SI_END_CF killed %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.181:
successors: %bb.182(0x80000000)
@@ -1439,7 +1439,7 @@ body: |
successors: %bb.187(0x80000000)
%508:vgpr_32 = PHI %505, %bb.184, %507, %bb.185
- SI_END_CF killed %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.187:
successors: %bb.188(0x80000000)
@@ -1484,7 +1484,7 @@ body: |
successors: %bb.193(0x80000000)
%524:vgpr_32 = PHI %521, %bb.190, %523, %bb.191
- SI_END_CF killed %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.193:
successors: %bb.194(0x80000000)
@@ -1529,7 +1529,7 @@ body: |
successors: %bb.199(0x80000000)
%540:vgpr_32 = PHI %537, %bb.196, %539, %bb.197
- SI_END_CF killed %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.199:
successors: %bb.200(0x80000000)
@@ -1574,7 +1574,7 @@ body: |
successors: %bb.205(0x80000000)
%556:vgpr_32 = PHI %553, %bb.202, %555, %bb.203
- SI_END_CF killed %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.205:
successors: %bb.206(0x80000000)
@@ -1619,7 +1619,7 @@ body: |
successors: %bb.211(0x80000000)
%572:vgpr_32 = PHI %569, %bb.208, %571, %bb.209
- SI_END_CF killed %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.211:
successors: %bb.212(0x80000000)
@@ -1664,7 +1664,7 @@ body: |
successors: %bb.217(0x80000000)
%588:vgpr_32 = PHI %585, %bb.214, %587, %bb.215
- SI_END_CF killed %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.217:
successors: %bb.218(0x80000000)
@@ -1709,7 +1709,7 @@ body: |
successors: %bb.223(0x80000000)
%604:vgpr_32 = PHI %601, %bb.220, %603, %bb.221
- SI_END_CF killed %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.223:
successors: %bb.224(0x80000000)
@@ -1754,7 +1754,7 @@ body: |
successors: %bb.229(0x80000000)
%620:vgpr_32 = PHI %617, %bb.226, %619, %bb.227
- SI_END_CF killed %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.229:
successors: %bb.230(0x80000000)
@@ -1799,7 +1799,7 @@ body: |
successors: %bb.235(0x80000000)
%636:vgpr_32 = PHI %633, %bb.232, %635, %bb.233
- SI_END_CF killed %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.235:
successors: %bb.236(0x80000000)
@@ -1844,7 +1844,7 @@ body: |
successors: %bb.241(0x80000000)
%652:vgpr_32 = PHI %649, %bb.238, %651, %bb.239
- SI_END_CF killed %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.241:
successors: %bb.242(0x80000000)
@@ -1889,7 +1889,7 @@ body: |
successors: %bb.247(0x80000000)
%668:vgpr_32 = PHI %665, %bb.244, %667, %bb.245
- SI_END_CF killed %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.247:
successors: %bb.248(0x80000000)
@@ -1934,7 +1934,7 @@ body: |
successors: %bb.253(0x80000000)
%684:vgpr_32 = PHI %681, %bb.250, %683, %bb.251
- SI_END_CF killed %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.253:
successors: %bb.254(0x80000000)
@@ -1979,7 +1979,7 @@ body: |
successors: %bb.259(0x80000000)
%700:vgpr_32 = PHI %697, %bb.256, %699, %bb.257
- SI_END_CF killed %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.259:
successors: %bb.260(0x80000000)
@@ -2024,7 +2024,7 @@ body: |
successors: %bb.265(0x80000000)
%716:vgpr_32 = PHI %713, %bb.262, %715, %bb.263
- SI_END_CF killed %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.265:
successors: %bb.266(0x80000000)
@@ -2069,7 +2069,7 @@ body: |
successors: %bb.271(0x80000000)
%732:vgpr_32 = PHI %729, %bb.268, %731, %bb.269
- SI_END_CF killed %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.271:
successors: %bb.272(0x80000000)
@@ -2114,7 +2114,7 @@ body: |
successors: %bb.277(0x80000000)
%748:vgpr_32 = PHI %745, %bb.274, %747, %bb.275
- SI_END_CF killed %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.277:
successors: %bb.278(0x80000000)
@@ -2159,7 +2159,7 @@ body: |
successors: %bb.283(0x80000000)
%764:vgpr_32 = PHI %761, %bb.280, %763, %bb.281
- SI_END_CF killed %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.283:
successors: %bb.284(0x80000000)
@@ -2204,7 +2204,7 @@ body: |
successors: %bb.289(0x80000000)
%780:vgpr_32 = PHI %777, %bb.286, %779, %bb.287
- SI_END_CF killed %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.289:
successors: %bb.290(0x80000000)
@@ -2249,7 +2249,7 @@ body: |
successors: %bb.295(0x80000000)
%796:vgpr_32 = PHI %793, %bb.292, %795, %bb.293
- SI_END_CF killed %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.295:
successors: %bb.296(0x80000000)
@@ -2294,7 +2294,7 @@ body: |
successors: %bb.301(0x80000000)
%812:vgpr_32 = PHI %809, %bb.298, %811, %bb.299
- SI_END_CF killed %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.301:
successors: %bb.302(0x80000000)
@@ -2339,7 +2339,7 @@ body: |
successors: %bb.307(0x80000000)
%828:vgpr_32 = PHI %825, %bb.304, %827, %bb.305
- SI_END_CF killed %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.307:
successors: %bb.308(0x80000000)
@@ -2384,7 +2384,7 @@ body: |
successors: %bb.313(0x80000000)
%844:vgpr_32 = PHI %841, %bb.310, %843, %bb.311
- SI_END_CF killed %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.313:
successors: %bb.314(0x80000000)
@@ -2429,7 +2429,7 @@ body: |
successors: %bb.319(0x80000000)
%860:vgpr_32 = PHI %857, %bb.316, %859, %bb.317
- SI_END_CF killed %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.319:
successors: %bb.320(0x80000000)
@@ -2474,7 +2474,7 @@ body: |
successors: %bb.325(0x80000000)
%876:vgpr_32 = PHI %873, %bb.322, %875, %bb.323
- SI_END_CF killed %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.325:
successors: %bb.326(0x80000000)
@@ -2519,7 +2519,7 @@ body: |
successors: %bb.331(0x80000000)
%892:vgpr_32 = PHI %889, %bb.328, %891, %bb.329
- SI_END_CF killed %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.331:
successors: %bb.332(0x80000000)
@@ -2564,7 +2564,7 @@ body: |
successors: %bb.337(0x80000000)
%908:vgpr_32 = PHI %905, %bb.334, %907, %bb.335
- SI_END_CF killed %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.337:
successors: %bb.338(0x80000000)
@@ -2609,7 +2609,7 @@ body: |
successors: %bb.343(0x80000000)
%924:vgpr_32 = PHI %921, %bb.340, %923, %bb.341
- SI_END_CF killed %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.343:
successors: %bb.344(0x80000000)
@@ -2654,7 +2654,7 @@ body: |
successors: %bb.349(0x80000000)
%940:vgpr_32 = PHI %937, %bb.346, %939, %bb.347
- SI_END_CF killed %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.349:
successors: %bb.350(0x80000000)
@@ -2699,7 +2699,7 @@ body: |
successors: %bb.355(0x80000000)
%956:vgpr_32 = PHI %953, %bb.352, %955, %bb.353
- SI_END_CF killed %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.355:
successors: %bb.356(0x80000000)
@@ -2744,7 +2744,7 @@ body: |
successors: %bb.361(0x80000000)
%972:vgpr_32 = PHI %969, %bb.358, %971, %bb.359
- SI_END_CF killed %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.361:
successors: %bb.362(0x80000000)
@@ -2789,7 +2789,7 @@ body: |
successors: %bb.367(0x80000000)
%988:vgpr_32 = PHI %985, %bb.364, %987, %bb.365
- SI_END_CF killed %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.367:
successors: %bb.368(0x80000000)
@@ -2834,7 +2834,7 @@ body: |
successors: %bb.373(0x80000000)
%1004:vgpr_32 = PHI %1001, %bb.370, %1003, %bb.371
- SI_END_CF killed %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.373:
successors: %bb.374(0x80000000)
@@ -2879,7 +2879,7 @@ body: |
successors: %bb.379(0x80000000)
%1020:vgpr_32 = PHI %1017, %bb.376, %1019, %bb.377
- SI_END_CF killed %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.379:
successors: %bb.380(0x80000000)
@@ -2924,7 +2924,7 @@ body: |
successors: %bb.385(0x80000000)
%1036:vgpr_32 = PHI %1033, %bb.382, %1035, %bb.383
- SI_END_CF killed %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.385:
successors: %bb.386(0x80000000)
@@ -2969,7 +2969,7 @@ body: |
successors: %bb.391(0x80000000)
%1052:vgpr_32 = PHI %1049, %bb.388, %1051, %bb.389
- SI_END_CF killed %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.391:
successors: %bb.392(0x80000000)
@@ -3014,7 +3014,7 @@ body: |
successors: %bb.397(0x80000000)
%1068:vgpr_32 = PHI %1065, %bb.394, %1067, %bb.395
- SI_END_CF killed %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.397:
successors: %bb.398(0x80000000)
@@ -3059,7 +3059,7 @@ body: |
successors: %bb.403(0x80000000)
%1084:vgpr_32 = PHI %1081, %bb.400, %1083, %bb.401
- SI_END_CF killed %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.403:
successors: %bb.404(0x80000000)
@@ -3104,7 +3104,7 @@ body: |
successors: %bb.409(0x80000000)
%1100:vgpr_32 = PHI %1097, %bb.406, %1099, %bb.407
- SI_END_CF killed %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.409:
successors: %bb.410(0x80000000)
@@ -3149,7 +3149,7 @@ body: |
successors: %bb.415(0x80000000)
%1116:vgpr_32 = PHI %1113, %bb.412, %1115, %bb.413
- SI_END_CF killed %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.415:
successors: %bb.416(0x80000000)
@@ -3194,7 +3194,7 @@ body: |
successors: %bb.421(0x80000000)
%1132:vgpr_32 = PHI %1129, %bb.418, %1131, %bb.419
- SI_END_CF killed %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.421:
successors: %bb.422(0x80000000)
@@ -3239,7 +3239,7 @@ body: |
successors: %bb.427(0x80000000)
%1148:vgpr_32 = PHI %1145, %bb.424, %1147, %bb.425
- SI_END_CF killed %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.427:
successors: %bb.428(0x80000000)
@@ -3284,7 +3284,7 @@ body: |
successors: %bb.433(0x80000000)
%1164:vgpr_32 = PHI %1161, %bb.430, %1163, %bb.431
- SI_END_CF killed %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.433:
successors: %bb.434(0x80000000)
@@ -3329,7 +3329,7 @@ body: |
successors: %bb.439(0x80000000)
%1180:vgpr_32 = PHI %1177, %bb.436, %1179, %bb.437
- SI_END_CF killed %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.439:
successors: %bb.440(0x80000000)
@@ -3374,7 +3374,7 @@ body: |
successors: %bb.445(0x80000000)
%1196:vgpr_32 = PHI %1193, %bb.442, %1195, %bb.443
- SI_END_CF killed %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.445:
successors: %bb.446(0x80000000)
@@ -3419,7 +3419,7 @@ body: |
successors: %bb.451(0x80000000)
%1212:vgpr_32 = PHI %1209, %bb.448, %1211, %bb.449
- SI_END_CF killed %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.451:
successors: %bb.452(0x80000000)
@@ -3464,7 +3464,7 @@ body: |
successors: %bb.457(0x80000000)
%1228:vgpr_32 = PHI %1225, %bb.454, %1227, %bb.455
- SI_END_CF killed %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.457:
successors: %bb.458(0x80000000)
@@ -3509,7 +3509,7 @@ body: |
successors: %bb.463(0x80000000)
%1244:vgpr_32 = PHI %1241, %bb.460, %1243, %bb.461
- SI_END_CF killed %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.463:
successors: %bb.464(0x80000000)
@@ -3554,7 +3554,7 @@ body: |
successors: %bb.469(0x80000000)
%1260:vgpr_32 = PHI %1257, %bb.466, %1259, %bb.467
- SI_END_CF killed %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.469:
successors: %bb.470(0x80000000)
@@ -3599,7 +3599,7 @@ body: |
successors: %bb.475(0x80000000)
%1276:vgpr_32 = PHI %1273, %bb.472, %1275, %bb.473
- SI_END_CF killed %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.475:
successors: %bb.476(0x80000000)
@@ -3644,7 +3644,7 @@ body: |
successors: %bb.481(0x80000000)
%1292:vgpr_32 = PHI %1289, %bb.478, %1291, %bb.479
- SI_END_CF killed %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.481:
successors: %bb.482(0x80000000)
@@ -3689,7 +3689,7 @@ body: |
successors: %bb.487(0x80000000)
%1308:vgpr_32 = PHI %1305, %bb.484, %1307, %bb.485
- SI_END_CF killed %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.487:
successors: %bb.488(0x80000000)
@@ -3734,7 +3734,7 @@ body: |
successors: %bb.493(0x80000000)
%1324:vgpr_32 = PHI %1321, %bb.490, %1323, %bb.491
- SI_END_CF killed %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.493:
successors: %bb.494(0x80000000)
@@ -3779,7 +3779,7 @@ body: |
successors: %bb.499(0x80000000)
%1340:vgpr_32 = PHI %1337, %bb.496, %1339, %bb.497
- SI_END_CF killed %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.499:
successors: %bb.500(0x80000000)
@@ -3824,7 +3824,7 @@ body: |
successors: %bb.505(0x80000000)
%1356:vgpr_32 = PHI %1353, %bb.502, %1355, %bb.503
- SI_END_CF killed %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.505:
successors: %bb.506(0x80000000)
@@ -3869,7 +3869,7 @@ body: |
successors: %bb.511(0x80000000)
%1372:vgpr_32 = PHI %1369, %bb.508, %1371, %bb.509
- SI_END_CF killed %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.511:
successors: %bb.512(0x80000000)
@@ -3914,7 +3914,7 @@ body: |
successors: %bb.517(0x80000000)
%1388:vgpr_32 = PHI %1385, %bb.514, %1387, %bb.515
- SI_END_CF killed %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.517:
successors: %bb.518(0x80000000)
@@ -3959,7 +3959,7 @@ body: |
successors: %bb.523(0x80000000)
%1404:vgpr_32 = PHI %1401, %bb.520, %1403, %bb.521
- SI_END_CF killed %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.523:
successors: %bb.524(0x80000000)
@@ -4004,7 +4004,7 @@ body: |
successors: %bb.529(0x80000000)
%1420:vgpr_32 = PHI %1417, %bb.526, %1419, %bb.527
- SI_END_CF killed %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.529:
successors: %bb.530(0x80000000)
@@ -4049,7 +4049,7 @@ body: |
successors: %bb.535(0x80000000)
%1436:vgpr_32 = PHI %1433, %bb.532, %1435, %bb.533
- SI_END_CF killed %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.535:
successors: %bb.536(0x80000000)
@@ -4094,7 +4094,7 @@ body: |
successors: %bb.541(0x80000000)
%1452:vgpr_32 = PHI %1449, %bb.538, %1451, %bb.539
- SI_END_CF killed %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.541:
successors: %bb.542(0x80000000)
@@ -4139,7 +4139,7 @@ body: |
successors: %bb.547(0x80000000)
%1468:vgpr_32 = PHI %1465, %bb.544, %1467, %bb.545
- SI_END_CF killed %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.547:
successors: %bb.548(0x80000000)
@@ -4184,7 +4184,7 @@ body: |
successors: %bb.553(0x80000000)
%1484:vgpr_32 = PHI %1481, %bb.550, %1483, %bb.551
- SI_END_CF killed %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.553:
successors: %bb.554(0x80000000)
@@ -4229,7 +4229,7 @@ body: |
successors: %bb.559(0x80000000)
%1500:vgpr_32 = PHI %1497, %bb.556, %1499, %bb.557
- SI_END_CF killed %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.559:
successors: %bb.560(0x80000000)
@@ -4274,7 +4274,7 @@ body: |
successors: %bb.565(0x80000000)
%1516:vgpr_32 = PHI %1513, %bb.562, %1515, %bb.563
- SI_END_CF killed %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.565:
successors: %bb.566(0x80000000)
@@ -4319,7 +4319,7 @@ body: |
successors: %bb.571(0x80000000)
%1532:vgpr_32 = PHI %1529, %bb.568, %1531, %bb.569
- SI_END_CF killed %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.571:
successors: %bb.572(0x80000000)
@@ -4364,7 +4364,7 @@ body: |
successors: %bb.577(0x80000000)
%1548:vgpr_32 = PHI %1545, %bb.574, %1547, %bb.575
- SI_END_CF killed %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.577:
successors: %bb.578(0x80000000)
@@ -4409,7 +4409,7 @@ body: |
successors: %bb.583(0x80000000)
%1564:vgpr_32 = PHI %1561, %bb.580, %1563, %bb.581
- SI_END_CF killed %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.583:
successors: %bb.584(0x80000000)
@@ -4454,7 +4454,7 @@ body: |
successors: %bb.589(0x80000000)
%1580:vgpr_32 = PHI %1577, %bb.586, %1579, %bb.587
- SI_END_CF killed %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.589:
successors: %bb.590(0x80000000)
@@ -4499,7 +4499,7 @@ body: |
successors: %bb.595(0x80000000)
%1596:vgpr_32 = PHI %1593, %bb.592, %1595, %bb.593
- SI_END_CF killed %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.595:
successors: %bb.596(0x80000000)
@@ -4544,7 +4544,7 @@ body: |
successors: %bb.601(0x80000000)
%1612:vgpr_32 = PHI %1609, %bb.598, %1611, %bb.599
- SI_END_CF killed %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.601:
successors: %bb.602(0x80000000)
@@ -4589,7 +4589,7 @@ body: |
successors: %bb.607(0x80000000)
%1628:vgpr_32 = PHI %1625, %bb.604, %1627, %bb.605
- SI_END_CF killed %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.607:
successors: %bb.608(0x80000000)
@@ -4634,7 +4634,7 @@ body: |
successors: %bb.613(0x80000000)
%1644:vgpr_32 = PHI %1641, %bb.610, %1643, %bb.611
- SI_END_CF killed %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.613:
successors: %bb.614(0x80000000)
@@ -4679,7 +4679,7 @@ body: |
successors: %bb.619(0x80000000)
%1660:vgpr_32 = PHI %1657, %bb.616, %1659, %bb.617
- SI_END_CF killed %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.619:
successors: %bb.620(0x80000000)
@@ -4724,7 +4724,7 @@ body: |
successors: %bb.625(0x80000000)
%1676:vgpr_32 = PHI %1673, %bb.622, %1675, %bb.623
- SI_END_CF killed %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.625:
successors: %bb.626(0x80000000)
@@ -4769,7 +4769,7 @@ body: |
successors: %bb.631(0x80000000)
%1692:vgpr_32 = PHI %1689, %bb.628, %1691, %bb.629
- SI_END_CF killed %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.631:
successors: %bb.632(0x80000000)
@@ -4814,7 +4814,7 @@ body: |
successors: %bb.637(0x80000000)
%1708:vgpr_32 = PHI %1705, %bb.634, %1707, %bb.635
- SI_END_CF killed %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.637:
successors: %bb.638(0x80000000)
@@ -4859,7 +4859,7 @@ body: |
successors: %bb.643(0x80000000)
%1724:vgpr_32 = PHI %1721, %bb.640, %1723, %bb.641
- SI_END_CF killed %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.643:
successors: %bb.644(0x80000000)
@@ -4904,7 +4904,7 @@ body: |
successors: %bb.649(0x80000000)
%1740:vgpr_32 = PHI %1737, %bb.646, %1739, %bb.647
- SI_END_CF killed %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.649:
successors: %bb.650(0x80000000)
@@ -4949,7 +4949,7 @@ body: |
successors: %bb.655(0x80000000)
%1756:vgpr_32 = PHI %1753, %bb.652, %1755, %bb.653
- SI_END_CF killed %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.655:
successors: %bb.656(0x80000000)
@@ -4994,7 +4994,7 @@ body: |
successors: %bb.661(0x80000000)
%1772:vgpr_32 = PHI %1769, %bb.658, %1771, %bb.659
- SI_END_CF killed %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.661:
successors: %bb.662(0x80000000)
@@ -5039,7 +5039,7 @@ body: |
successors: %bb.667(0x80000000)
%1788:vgpr_32 = PHI %1785, %bb.664, %1787, %bb.665
- SI_END_CF killed %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.667:
successors: %bb.668(0x80000000)
@@ -5084,7 +5084,7 @@ body: |
successors: %bb.673(0x80000000)
%1804:vgpr_32 = PHI %1801, %bb.670, %1803, %bb.671
- SI_END_CF killed %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.673:
successors: %bb.674(0x80000000)
@@ -5129,7 +5129,7 @@ body: |
successors: %bb.679(0x80000000)
%1820:vgpr_32 = PHI %1817, %bb.676, %1819, %bb.677
- SI_END_CF killed %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.679:
successors: %bb.680(0x80000000)
@@ -5174,7 +5174,7 @@ body: |
successors: %bb.685(0x80000000)
%1836:vgpr_32 = PHI %1833, %bb.682, %1835, %bb.683
- SI_END_CF killed %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.685:
successors: %bb.686(0x80000000)
@@ -5219,7 +5219,7 @@ body: |
successors: %bb.691(0x80000000)
%1852:vgpr_32 = PHI %1849, %bb.688, %1851, %bb.689
- SI_END_CF killed %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.691:
successors: %bb.692(0x80000000)
@@ -5264,7 +5264,7 @@ body: |
successors: %bb.697(0x80000000)
%1868:vgpr_32 = PHI %1865, %bb.694, %1867, %bb.695
- SI_END_CF killed %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.697:
successors: %bb.698(0x80000000)
@@ -5309,7 +5309,7 @@ body: |
successors: %bb.703(0x80000000)
%1884:vgpr_32 = PHI %1881, %bb.700, %1883, %bb.701
- SI_END_CF killed %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.703:
successors: %bb.704(0x80000000)
@@ -5354,7 +5354,7 @@ body: |
successors: %bb.709(0x80000000)
%1900:vgpr_32 = PHI %1897, %bb.706, %1899, %bb.707
- SI_END_CF killed %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.709:
successors: %bb.710(0x80000000)
@@ -5399,7 +5399,7 @@ body: |
successors: %bb.715(0x80000000)
%1916:vgpr_32 = PHI %1913, %bb.712, %1915, %bb.713
- SI_END_CF killed %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.715:
successors: %bb.716(0x80000000)
@@ -5444,7 +5444,7 @@ body: |
successors: %bb.721(0x80000000)
%1932:vgpr_32 = PHI %1929, %bb.718, %1931, %bb.719
- SI_END_CF killed %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.721:
successors: %bb.722(0x80000000)
@@ -5489,7 +5489,7 @@ body: |
successors: %bb.727(0x80000000)
%1948:vgpr_32 = PHI %1945, %bb.724, %1947, %bb.725
- SI_END_CF killed %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.727:
successors: %bb.728(0x80000000)
@@ -5534,7 +5534,7 @@ body: |
successors: %bb.733(0x80000000)
%1964:vgpr_32 = PHI %1961, %bb.730, %1963, %bb.731
- SI_END_CF killed %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.733:
successors: %bb.734(0x80000000)
@@ -5579,7 +5579,7 @@ body: |
successors: %bb.739(0x80000000)
%1980:vgpr_32 = PHI %1977, %bb.736, %1979, %bb.737
- SI_END_CF killed %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.739:
successors: %bb.740(0x80000000)
@@ -5624,7 +5624,7 @@ body: |
successors: %bb.745(0x80000000)
%1996:vgpr_32 = PHI %1993, %bb.742, %1995, %bb.743
- SI_END_CF killed %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.745:
successors: %bb.746(0x80000000)
@@ -5669,7 +5669,7 @@ body: |
successors: %bb.751(0x80000000)
%2012:vgpr_32 = PHI %2009, %bb.748, %2011, %bb.749
- SI_END_CF killed %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.751:
successors: %bb.752(0x80000000)
@@ -5714,7 +5714,7 @@ body: |
successors: %bb.757(0x80000000)
%2028:vgpr_32 = PHI %2025, %bb.754, %2027, %bb.755
- SI_END_CF killed %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.757:
successors: %bb.758(0x80000000)
@@ -5759,7 +5759,7 @@ body: |
successors: %bb.763(0x80000000)
%2044:vgpr_32 = PHI %2041, %bb.760, %2043, %bb.761
- SI_END_CF killed %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.763:
successors: %bb.764(0x80000000)
@@ -5804,7 +5804,7 @@ body: |
successors: %bb.769(0x80000000)
%2060:vgpr_32 = PHI %2057, %bb.766, %2059, %bb.767
- SI_END_CF killed %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.769:
@@ -6599,7 +6599,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7120,7 +7120,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7641,7 +7641,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8162,7 +8162,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8683,7 +8683,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9204,7 +9204,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9725,7 +9725,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10246,7 +10246,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10767,7 +10767,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11288,7 +11288,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11809,7 +11809,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12330,7 +12330,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12851,7 +12851,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13372,7 +13372,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13893,7 +13893,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14414,7 +14414,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14935,7 +14935,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15456,7 +15456,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15977,7 +15977,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16498,7 +16498,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17019,7 +17019,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17540,7 +17540,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18061,7 +18061,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18582,7 +18582,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19103,7 +19103,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19624,7 +19624,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20145,7 +20145,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20666,7 +20666,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -21187,7 +21187,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -21708,7 +21708,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -22229,7 +22229,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -22750,7 +22750,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23271,7 +23271,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23792,7 +23792,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24313,7 +24313,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24834,7 +24834,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25355,7 +25355,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25876,7 +25876,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26397,7 +26397,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26918,7 +26918,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27439,7 +27439,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27960,7 +27960,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -28481,7 +28481,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29002,7 +29002,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29523,7 +29523,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -30044,7 +30044,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -30565,7 +30565,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -31086,7 +31086,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -31607,7 +31607,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32128,7 +32128,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32649,7 +32649,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33170,7 +33170,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33691,7 +33691,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34212,7 +34212,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34733,7 +34733,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35254,7 +35254,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35775,7 +35775,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -36296,7 +36296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -36817,7 +36817,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -37338,7 +37338,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -37859,7 +37859,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -38380,7 +38380,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -38901,7 +38901,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39422,7 +39422,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39943,7 +39943,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40464,7 +40464,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40985,7 +40985,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41506,7 +41506,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42027,7 +42027,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42548,7 +42548,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -43069,7 +43069,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -43590,7 +43590,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44111,7 +44111,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44632,7 +44632,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -45153,7 +45153,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -45674,7 +45674,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -46195,7 +46195,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -46716,7 +46716,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47237,7 +47237,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47758,7 +47758,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -48279,7 +48279,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -48800,7 +48800,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49321,7 +49321,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49842,7 +49842,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -50363,7 +50363,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -50884,7 +50884,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -51405,7 +51405,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -51926,7 +51926,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -52447,7 +52447,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -52968,7 +52968,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53489,7 +53489,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54010,7 +54010,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54531,7 +54531,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55052,7 +55052,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55573,7 +55573,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56094,7 +56094,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56615,7 +56615,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -57136,7 +57136,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -57657,7 +57657,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -58178,7 +58178,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -58699,7 +58699,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -59220,7 +59220,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -59741,7 +59741,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -60262,7 +60262,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -60783,7 +60783,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -61304,7 +61304,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -61825,7 +61825,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -62346,7 +62346,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -62867,7 +62867,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -63388,7 +63388,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -63909,7 +63909,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -64430,7 +64430,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -64951,7 +64951,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -65472,7 +65472,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -65993,7 +65993,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -66514,7 +66514,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -67035,7 +67035,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -67556,7 +67556,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -68077,7 +68077,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -68598,7 +68598,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -69119,7 +69119,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -69640,7 +69640,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -70161,7 +70161,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -70682,7 +70682,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -71203,7 +71203,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -71724,7 +71724,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -72245,7 +72245,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -72763,7 +72763,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -75777,7 +75777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -76302,7 +76302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -76827,7 +76827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -77352,7 +77352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %74, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -77877,7 +77877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -78402,7 +78402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %106, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -78927,7 +78927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -79452,7 +79452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %138, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -79977,7 +79977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %154, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -80502,7 +80502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %170, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -81027,7 +81027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %186, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -81552,7 +81552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %202, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -82077,7 +82077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %218, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -82602,7 +82602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %234, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -83127,7 +83127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %250, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -83652,7 +83652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %266, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -84177,7 +84177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %282, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -84702,7 +84702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %298, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -85227,7 +85227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %314, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -85752,7 +85752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %330, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -86277,7 +86277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -86802,7 +86802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %362, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -87327,7 +87327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %378, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -87852,7 +87852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %394, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -88377,7 +88377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %410, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -88902,7 +88902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %426, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -89427,7 +89427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -89952,7 +89952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %458, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -90477,7 +90477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %474, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -91002,7 +91002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %490, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -91527,7 +91527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %506, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -92052,7 +92052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %522, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -92577,7 +92577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %538, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -93102,7 +93102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %554, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -93627,7 +93627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %570, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -94152,7 +94152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %586, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -94677,7 +94677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %602, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -95202,7 +95202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %618, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -95727,7 +95727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %634, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -96252,7 +96252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %650, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -96777,7 +96777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %666, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -97302,7 +97302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %682, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -97827,7 +97827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %698, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -98352,7 +98352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %714, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -98877,7 +98877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %730, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -99402,7 +99402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %746, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -99927,7 +99927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %762, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -100452,7 +100452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %778, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -100977,7 +100977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %794, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -101502,7 +101502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %810, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -102027,7 +102027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %826, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -102552,7 +102552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %842, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -103077,7 +103077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %858, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -103602,7 +103602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %874, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -104127,7 +104127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %890, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -104652,7 +104652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %906, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -105177,7 +105177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %922, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -105702,7 +105702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %938, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -106227,7 +106227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %954, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -106752,7 +106752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %970, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -107277,7 +107277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %986, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -107802,7 +107802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1002, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -108327,7 +108327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1018, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -108852,7 +108852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1034, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -109377,7 +109377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1050, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -109902,7 +109902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1066, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -110427,7 +110427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1082, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -110952,7 +110952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1098, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -111477,7 +111477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -112002,7 +112002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1130, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -112527,7 +112527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1146, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -113052,7 +113052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1162, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -113577,7 +113577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1178, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -114102,7 +114102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1194, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -114627,7 +114627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1210, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -115152,7 +115152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1226, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -115677,7 +115677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1242, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -116202,7 +116202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1258, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -116727,7 +116727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1274, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -117252,7 +117252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1290, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -117777,7 +117777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1306, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -118302,7 +118302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1322, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -118827,7 +118827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1338, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -119352,7 +119352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1354, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -119877,7 +119877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1370, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -120402,7 +120402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1386, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -120927,7 +120927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1402, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -121452,7 +121452,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1418, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -121977,7 +121977,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1434, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -122502,7 +122502,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1450, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -123027,7 +123027,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1466, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -123552,7 +123552,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1482, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -124077,7 +124077,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1498, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -124602,7 +124602,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1514, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -125127,7 +125127,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1530, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -125652,7 +125652,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1546, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -126177,7 +126177,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1562, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -126702,7 +126702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1578, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -127227,7 +127227,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1594, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -127752,7 +127752,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1610, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -128277,7 +128277,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1626, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -128802,7 +128802,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1642, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -129327,7 +129327,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1658, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -129852,7 +129852,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1674, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -130377,7 +130377,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1690, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -130902,7 +130902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1706, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -131427,7 +131427,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1722, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -131952,7 +131952,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1738, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -132477,7 +132477,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1754, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -133002,7 +133002,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1770, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -133527,7 +133527,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1786, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -134052,7 +134052,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1802, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -134577,7 +134577,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1818, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -135102,7 +135102,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1834, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -135627,7 +135627,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1850, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -136152,7 +136152,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1866, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -136677,7 +136677,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1882, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -137202,7 +137202,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1898, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -137727,7 +137727,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1914, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -138252,7 +138252,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1930, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -138777,7 +138777,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1946, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -139302,7 +139302,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1962, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -139827,7 +139827,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1978, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -140352,7 +140352,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %1994, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -140877,7 +140877,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2010, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -141402,7 +141402,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2026, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -141927,7 +141927,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2042, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -142449,7 +142449,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %2058, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
index 366cd4b757844..9bfff07a7da50 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-cfg-with-self-loop.mir
@@ -110,7 +110,7 @@ body: |
%28:vgpr_32 = PHI %29, %bb.6, %30, %bb.12
%31:vgpr_32 = PHI %32, %bb.6, %33, %bb.12
%34:vgpr_32 = PHI %35, %bb.6, %36, %bb.12
- SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.13
bb.3:
@@ -184,7 +184,7 @@ body: |
successors: %bb.10(0x40000000), %bb.8(0x40000000)
%85:vgpr_32 = PHI %24, %bb.4, %74, %bb.5
- SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%86:vgpr_32, %87:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %10, 24, 0, implicit $exec
%88:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 0, %9, %87, 0, implicit $exec
%89:vgpr_32, %90:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %15.sub0, 16, 0, implicit $exec
@@ -253,7 +253,7 @@ body: |
%76:vgpr_32 = PHI %98, %bb.8, %125, %bb.9
%140:vreg_64 = PHI %96, %bb.8, %115, %bb.9
%141:vreg_64 = PHI %93, %bb.8, %116, %bb.9
- SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%142:vgpr_32 = GLOBAL_LOAD_DWORD %62, 16, 0, implicit $exec :: (load (s32) from %ir.gep7, addrspace 1)
%77:vgpr_32 = V_ADD_U32_e64 %142, %139, 0, implicit $exec
GLOBAL_STORE_DWORD %140, %77, 0, 0, implicit $exec :: (store (s32) into %ir.phi4, addrspace 1)
@@ -270,7 +270,7 @@ body: |
bb.12:
successors: %bb.2(0x80000000)
- SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.2
bb.13:
@@ -1242,7 +1242,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2873,7 +2873,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4589,7 +4589,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4948,7 +4948,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.12": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6186,7 +6186,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7858,7 +7858,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %66, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9574,7 +9574,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %114, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9933,7 +9933,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.12": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
index 7e9cf15988231..99b8568c40791 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/acyclic-phi-merge-distances.mir
@@ -92,7 +92,7 @@ body: |
bb.4:
%45:vreg_64 = PHI %8, %bb.1, %29, %bb.2
- SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%46:vgpr_32, %47:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %45.sub0, %1, 0, implicit $exec
%48:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 %45.sub1, %0, %47, 0, implicit $exec
%49:sreg_32_xm0 = V_READFIRSTLANE_B32 %46, implicit $exec
@@ -1363,7 +1363,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2857,7 +2857,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
index 6fe28d7318b13..b06c255fb452d 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-acyclic-cfg-with-4-self-loops.mir
@@ -196,7 +196,7 @@ body: |
bb.2:
successors: %bb.3(0x40000000), %bb.7(0x40000000)
- SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%112:vgpr_32 = GLOBAL_LOAD_USHORT %20, 0, 0, implicit $exec :: (load (s16) from %ir.p3, addrspace 1)
%113:vgpr_32 = GLOBAL_LOAD_USHORT %20, 2, 0, implicit $exec :: (load (s16) from %ir.p3 + 2, addrspace 1)
%114:vgpr_32 = V_LSHL_OR_B32_e64 %113, 16, %112, implicit $exec
@@ -225,7 +225,7 @@ body: |
%123:vgpr_32 = PHI %124, %bb.7, %125, %bb.11
%126:vgpr_32 = PHI %127, %bb.7, %128, %bb.11
- SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.13
bb.6:
@@ -278,7 +278,7 @@ body: |
bb.9:
successors: %bb.10(0x80000000)
- SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %19, %154, 0, 0, implicit $exec :: (store (s32) into %ir.p4, addrspace 1)
%162:vreg_64 = REG_SEQUENCE %158, %subreg.sub0, undef %163:vgpr_32, %subreg.sub1
%164:sreg_32 = S_MOV_B32 0
@@ -325,14 +325,14 @@ body: |
bb.11:
successors: %bb.5(0x80000000)
- SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%128:vgpr_32 = V_MUL_LO_U32_e64 %147, %143, implicit $exec
S_BRANCH %bb.5
bb.12:
successors: %bb.7(0x80000000)
- SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%146:vgpr_32 = V_MUL_LO_U32_e64 %114, %161, implicit $exec
%145:vgpr_32 = V_MUL_LO_U32_e64 %173, %195, implicit $exec
S_BRANCH %bb.7
@@ -4747,7 +4747,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5320,7 +5320,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6534,7 +6534,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.9": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7969,7 +7969,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.11": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8055,7 +8055,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.12": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13071,7 +13071,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13644,7 +13644,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %129, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14898,7 +14898,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.9": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %153, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16445,7 +16445,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.11": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %131, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16531,7 +16531,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.12": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %166, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
index 356e533e40233..6ff0e51124d2b 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-11blocks.mir
@@ -51,7 +51,7 @@ body: |
%8:sreg_32 = PHI %9, %bb.3, %10, %bb.9
%11:vreg_64 = PHI undef %12:vreg_64, %bb.3, %13, %bb.9
%14:vreg_64 = PHI %15, %bb.3, undef %16:vreg_64, %bb.9
- SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%18:sreg_32 = S_AND_B32 %8, $exec_lo, implicit-def dead $scc
%19:sreg_32 = COPY %18
S_BRANCH %bb.8
@@ -86,7 +86,7 @@ body: |
%33:sreg_32 = PHI %34, %bb.6, %35, %bb.7
%36:vreg_64 = PHI %37, %bb.6, %38, %bb.7
- SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%40:sreg_32 = SI_IF %33, %bb.10, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
@@ -106,7 +106,7 @@ body: |
%34:sreg_32 = PHI %6, %bb.8, %28, %bb.2
%37:vreg_64 = PHI %47, %bb.8, undef %48:vreg_64, %bb.2
%49:vreg_64 = PHI %50, %bb.8, %27, %bb.2
- SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%39:sreg_32 = SI_IF %46, %bb.4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
@@ -154,7 +154,7 @@ body: |
bb.10:
%74:vreg_64 = PHI %36, %bb.4, %45, %bb.5
- SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%75:sreg_32_xm0 = V_READFIRSTLANE_B32 %74.sub0, implicit $exec
%76:sreg_32_xm0 = V_READFIRSTLANE_B32 %74.sub1, implicit $exec
$sgpr0 = COPY %75
@@ -455,7 +455,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1099,7 +1099,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1400,7 +1400,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2397,7 +2397,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2894,7 +2894,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3538,7 +3538,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %39, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3839,7 +3839,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %51, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4836,7 +4836,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %40, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
index 7dfc0890aabf6..4b7d27a203823 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-control-flow-15blocks.mir
@@ -105,7 +105,7 @@ body: |
successors: %bb.5(0x40000000), %bb.3(0x40000000)
%28:vgpr_32 = PHI %25, %bb.0, %27, %bb.1
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%29:vgpr_32 = DS_READ_U8_gfx9 %2, 0, 0, implicit $exec :: (load (s8) from %ir.p2, addrspace 3)
%30:vgpr_32 = DS_READ_U8_gfx9 %2, 1, 0, implicit $exec :: (load (s8) from %ir.p2 + 1, addrspace 3)
%31:vgpr_32 = DS_READ_U8_gfx9 %2, 2, 0, implicit $exec :: (load (s8) from %ir.p2 + 2, addrspace 3)
@@ -142,7 +142,7 @@ body: |
successors: %bb.12(0x40000000), %bb.7(0x40000000)
%46:vgpr_32 = PHI %38, %bb.3, %45, %bb.4
- SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%47:vgpr_32 = DS_READ_B32_gfx9 %2, 12, 0, implicit $exec :: (load (s32) from %ir.gep2, align 8, addrspace 3)
%48:vgpr_32 = V_ADD_U32_e64 %47, %46, 0, implicit $exec
%49:sreg_32 = S_XOR_B32 %7, %9, implicit-def dead $scc
@@ -219,11 +219,11 @@ body: |
successors: %bb.14(0x80000000)
%93:vgpr_32 = PHI %63, %bb.9, %71, %bb.10
- SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.14:
%94:vgpr_32 = PHI %52, %bb.7, %93, %bb.13
- SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%95:vgpr_32 = V_ADD_U32_e64 %36, %94, 0, implicit $exec
%96:sreg_32_xm0 = V_READFIRSTLANE_B32 %95, implicit $exec
$sgpr0 = COPY %96
@@ -998,7 +998,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1613,7 +1613,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2950,7 +2950,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2998,7 +2998,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3964,7 +3964,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4579,7 +4579,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %43, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5916,7 +5916,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %70, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5964,7 +5964,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
index 5b4391c82a993..48bb16f4139d6 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop-a.mir
@@ -112,7 +112,7 @@ body: |
%44:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %58, %bb.5
%41:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %59, %bb.5
%39:vgpr_32 = PHI undef %52:vgpr_32, %bb.1, %60, %bb.5
- SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%37:sreg_32 = SI_IF_BREAK %32, %36, implicit-def dead $scc
SI_LOOP %37, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
@@ -120,7 +120,7 @@ body: |
bb.3:
successors: %bb.4(0x40000000), %bb.6(0x40000000)
- SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%61:sreg_32 = SI_IF %28, %bb.6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -156,7 +156,7 @@ body: |
%68:vgpr_32 = PHI %56, %bb.3, %63, %bb.4
%69:vgpr_32 = PHI %54, %bb.3, %62, %bb.4
%70:vgpr_32 = PHI %51, %bb.3, %21, %bb.4
- SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%42:vgpr_32 = V_ADD_U32_e64 1, %40, 0, implicit $exec
%71:vgpr_32 = V_ADD_U32_e64 %68, %42, 0, implicit $exec
GLOBAL_STORE_DWORD %11, %71, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
@@ -167,7 +167,7 @@ body: |
S_BRANCH %bb.7
bb.7:
- SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%73:vgpr_32 = V_ADD3_U32_e64 %71, %43, %71, implicit $exec
%74:vgpr_32 = V_ADD3_U32_e64 %73, %68, %69, implicit $exec
%75:vgpr_32 = V_ADD3_U32_e64 %74, %70, %38, implicit $exec
@@ -1497,7 +1497,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1634,7 +1634,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.3": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2459,7 +2459,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2717,7 +2717,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.7": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4341,7 +4341,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %48, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4516,7 +4516,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.3": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %37, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5482,7 +5482,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %61, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5827,7 +5827,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.7": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %35, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
index 0abdb905d0f2e..7d4c4e3dc809a 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/complex-single-loop.mir
@@ -111,7 +111,7 @@ body: |
%43:sreg_32 = PHI %44, %bb.5, %45, %bb.6
%46:vgpr_32 = PHI undef %47:vgpr_32, %bb.5, %48, %bb.6
- SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%50:sreg_32 = SI_IF %43, %bb.7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -128,7 +128,7 @@ body: |
%52:sreg_32 = PHI %27, %bb.1, %42, %bb.2
%44:sreg_32 = PHI %17, %bb.1, %39, %bb.2
%22:vreg_64 = PHI %20, %bb.1, %36, %bb.2
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%49:sreg_32 = SI_IF %52, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -146,7 +146,7 @@ body: |
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
%26:vgpr_32 = PHI %46, %bb.3, %51, %bb.4
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%24:sreg_32 = S_ADD_I32 %23, 1, implicit-def dead $scc
%56:sreg_32 = V_CMP_GE_U32_e64 %24, %0, implicit $exec
%19:sreg_32 = SI_IF_BREAK %56, %18, implicit-def dead $scc
@@ -154,7 +154,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %7, %26, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
S_ENDPGM 0
...
@@ -1306,7 +1306,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1556,7 +1556,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1843,7 +1843,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1991,7 +1991,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3291,7 +3291,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3575,7 +3575,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3902,7 +3902,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4080,7 +4080,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
index 6744895e4896f..a27c271c3bfc3 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/double-nested-loops-complex-cfg.mir
@@ -233,7 +233,7 @@ body: |
%128:vgpr_32 = PHI %71, %bb.1, %129, %bb.33
%130:vgpr_32 = PHI %73, %bb.1, %131, %bb.33
%132:vgpr_32 = PHI undef %120:vgpr_32, %bb.1, %133, %bb.33
- SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%59:sreg_32 = SI_IF_BREAK %48, %58, implicit-def dead $scc
%134:sreg_32 = S_ANDN2_B32 %43, $exec_lo, implicit-def dead $scc
%135:sreg_32 = S_AND_B32 %57, $exec_lo, implicit-def dead $scc
@@ -271,7 +271,7 @@ body: |
%166:vgpr_32 = PHI undef %162:vgpr_32, %bb.7, %167, %bb.9
%168:vgpr_32 = PHI undef %162:vgpr_32, %bb.7, %147, %bb.9
%169:vgpr_32 = PHI undef %170:vgpr_32, %bb.7, %167, %bb.9
- SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%172:sreg_32 = S_ORN2_B32 %156, $exec_lo, implicit-def dead $scc
S_BRANCH %bb.8
@@ -400,7 +400,7 @@ body: |
%274:vgpr_32 = PHI undef %266:vgpr_32, %bb.10, %275, %bb.15
%276:vgpr_32 = PHI undef %277:vgpr_32, %bb.10, %248, %bb.15
%278:vgpr_32 = PHI %236, %bb.10, %279, %bb.15
- SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%280:sreg_32 = S_MOV_B32 0
%281:sreg_32 = SI_IF %254, %bb.17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.16
@@ -444,7 +444,7 @@ body: |
%273:vgpr_32 = PHI %71, %bb.11, undef %297:vgpr_32, %bb.13
%275:vgpr_32 = PHI %73, %bb.11, undef %297:vgpr_32, %bb.13
%279:vgpr_32 = PHI undef %298:vgpr_32, %bb.11, %282, %bb.13
- SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%263:sreg_32 = COPY $exec_lo
%261:sreg_32 = S_XOR_B32 $exec_lo, -1, implicit-def dead $scc
%299:sreg_32 = S_ANDN2_B32 %239, $exec_lo, implicit-def dead $scc
@@ -495,7 +495,7 @@ body: |
%344:vgpr_32 = PHI %270, %bb.12, %345, %bb.19
%346:vgpr_32 = PHI %272, %bb.12, %347, %bb.19
%348:vgpr_32 = PHI %274, %bb.12, %349, %bb.19
- SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%350:sreg_32 = SI_IF %316, %bb.33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.20
@@ -539,7 +539,7 @@ body: |
%345:vgpr_32 = PHI %385, %bb.23, %69, %bb.22
%347:vgpr_32 = PHI %386, %bb.23, %71, %bb.22
%349:vgpr_32 = PHI %387, %bb.23, %73, %bb.22
- SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%389:sreg_32 = S_AND_B32 %373, $exec_lo, implicit-def dead $scc
%325:sreg_32 = COPY %389
%390:sreg_32 = S_ANDN2_B32 %262, $exec_lo, implicit-def dead $scc
@@ -579,7 +579,7 @@ body: |
%413:vgpr_32 = PHI undef %411:vgpr_32, %bb.18, %414, %bb.24
%415:vgpr_32 = PHI undef %411:vgpr_32, %bb.18, %69, %bb.24
%416:vgpr_32 = PHI undef %417:vgpr_32, %bb.18, %414, %bb.24
- SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%418:sreg_32 = S_AND_B32 %408, $exec_lo, implicit-def dead $scc
%419:sreg_32 = COPY %418
%420:sreg_32 = S_ANDN2_B32 %262, $exec_lo, implicit-def dead $scc
@@ -716,19 +716,19 @@ body: |
successors: %bb.27(0x80000000)
%456:vgpr_32 = PHI %119, %bb.36, %484, %bb.29
- SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.27
bb.31:
successors: %bb.25(0x80000000)
%437:vgpr_32 = PHI %454, %bb.27, %473, %bb.28
- SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.25
bb.32:
%488:vgpr_32 = PHI %435, %bb.25, %451, %bb.26
- SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%489:vgpr_32 = V_ADD3_U32_e64 %30, %488, 100, implicit $exec
%490:sreg_32_xm0 = V_READFIRSTLANE_B32 %489, implicit $exec
$sgpr0 = COPY %490
@@ -753,7 +753,7 @@ body: |
%129:vgpr_32 = PHI %346, %bb.17, %71, %bb.20
%131:vgpr_32 = PHI %348, %bb.17, %73, %bb.20
%133:vgpr_32 = PHI %276, %bb.17, %112, %bb.20
- SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%495:sreg_32 = S_ANDN2_B32 %90, $exec_lo, implicit-def dead $scc
%496:sreg_32 = S_AND_B32 %494, $exec_lo, implicit-def dead $scc
%110:sreg_32 = S_OR_B32 %495, %496, implicit-def dead $scc
@@ -771,7 +771,7 @@ body: |
bb.34:
successors: %bb.35(0x40000000), %bb.25(0x40000000)
- SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%503:sreg_32 = S_XOR_B32 %39, -1, implicit-def dead $scc
%504:sreg_32 = S_XOR_B32 %42, -1, implicit-def dead $scc
%443:sreg_32 = SI_IF %503, %bb.25, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -802,7 +802,7 @@ body: |
%247:vgpr_32 = PHI %209, %bb.8, %147, %bb.6
%250:vgpr_32 = PHI %210, %bb.8, %149, %bb.6
%249:vgpr_32 = PHI %211, %bb.8, %151, %bb.6
- SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%507:sreg_32 = S_XOR_B32 %506, -1, implicit-def dead $scc
%144:sreg_32 = SI_IF_BREAK %505, %143, implicit-def dead $scc
%508:sreg_32 = S_ANDN2_B32 %140, $exec_lo, implicit-def dead $scc
@@ -814,7 +814,7 @@ body: |
bb.38:
successors: %bb.14(0x40000000), %bb.10(0x40000000)
- SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%234:sreg_32 = S_MOV_B32 0
%244:sreg_32 = SI_IF %142, %bb.10, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.14
@@ -4975,7 +4975,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6431,7 +6431,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12331,7 +12331,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14092,7 +14092,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16469,7 +16469,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18452,7 +18452,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20204,7 +20204,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24327,7 +24327,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24411,7 +24411,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -24486,7 +24486,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -25296,7 +25296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26005,7 +26005,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.34": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -26846,7 +26846,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -27282,7 +27282,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.38": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -32423,7 +32423,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %96, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -34153,7 +34153,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %171, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40308,7 +40308,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %243, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -42189,7 +42189,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %253, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -44829,7 +44829,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %281, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -47086,7 +47086,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %388, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -49129,7 +49129,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %363, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53604,7 +53604,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %487, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53688,7 +53688,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %461, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -53763,7 +53763,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %442, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -54733,7 +54733,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %350, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -55582,7 +55582,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.34": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %59, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56475,7 +56475,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %213, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -56967,7 +56967,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.38": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %144, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
index f35dff5b8eab8..c7150e147d76e 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/if_else_with_loops_nested_in_2_outer_loops.mir
@@ -180,7 +180,7 @@ body: |
%81:vgpr_32 = PHI %82, %bb.11, %83, %bb.14
%84:vgpr_32 = PHI %85, %bb.11, %86, %bb.14
- SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.15
bb.6:
@@ -218,7 +218,7 @@ body: |
bb.10:
successors: %bb.14(0x04000000), %bb.6(0x7c000000)
- SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%101:sreg_32 = S_ADD_I32 %95, 1, implicit-def dead $scc
%93:sreg_32 = S_ADD_I32 %92, %34, implicit-def dead $scc
%91:sreg_32 = S_ADD_I32 %90, %65, implicit-def dead $scc
@@ -254,13 +254,13 @@ body: |
bb.13:
successors: %bb.11(0x80000000)
- SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.11
bb.14:
successors: %bb.5(0x80000000)
- SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.15:
@@ -328,7 +328,7 @@ body: |
%164:vgpr_32 = PHI %139, %bb.15, %163, %bb.16
%165:vgpr_32 = PHI %131, %bb.15, %153, %bb.16
- SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:sreg_32 = S_ADD_I32 %65, 1, implicit-def dead $scc
%64:sreg_32 = S_ADD_I32 %63, 10, implicit-def dead $scc
%166:sreg_32 = V_CMP_GE_U32_e64 %66, %3, implicit $exec
@@ -339,7 +339,7 @@ body: |
bb.18:
successors: %bb.19(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%35:sreg_32 = S_ADD_I32 %34, 1, implicit-def dead $scc
%167:vgpr_32 = V_ADD_U32_e64 %35, %49, 0, implicit $exec
%168:vgpr_32 = V_ASHRREV_I32_e64 31, %165, implicit $exec
@@ -357,7 +357,7 @@ body: |
S_BRANCH %bb.19
bb.19:
- SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%176:vgpr_32 = V_ADD_U32_e64 %164, %49, 0, implicit $exec
%177:sreg_32_xm0 = V_READFIRSTLANE_B32 %176, implicit $exec
$sgpr0 = COPY %177
@@ -3233,7 +3233,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3997,7 +3997,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.10": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4986,7 +4986,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5071,7 +5071,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7161,7 +7161,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7386,7 +7386,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7835,7 +7835,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.19": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11087,7 +11087,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %87, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11976,7 +11976,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.10": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13145,7 +13145,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %107, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13242,7 +13242,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %89, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15895,7 +15895,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %142, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16207,7 +16207,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %62, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16883,7 +16883,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.19": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
index c5a44b5d7c854..08a866634750d 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/inner_cfg_in_2_nested_loops.mir
@@ -175,7 +175,7 @@ body: |
bb.7:
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%22:sreg_32 = S_ADD_I32 %21, 1, implicit-def dead $scc
%83:sreg_32 = V_CMP_GE_U32_e64 %22, %1, implicit $exec
%20:sreg_32 = SI_IF_BREAK %83, %19, implicit-def dead $scc
@@ -184,7 +184,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%85:vgpr_32 = V_ADD3_U32_e64 %23, %84, %17, implicit $exec
%86:vgpr_32 = V_ADD3_U32_e64 %85, %26, 100, implicit $exec
%87:sreg_32_xm0 = V_READFIRSTLANE_B32 %86, implicit $exec
@@ -2972,7 +2972,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.7": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3154,7 +3154,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6448,7 +6448,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.7": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %28, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6685,7 +6685,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
index 465454e5aeaf1..913c04284b897 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/loop_nested_in_3_outer_loops_complex_cfg.mir
@@ -199,7 +199,7 @@ body: |
%38:vgpr_32 = PHI %95, %bb.4, %106, %bb.9
%107:vgpr_32 = PHI %97, %bb.4, %108, %bb.9
%109:vgpr_32 = PHI %99, %bb.4, %110, %bb.9
- SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.10
bb.7:
@@ -228,7 +228,7 @@ body: |
bb.9:
successors: %bb.6(0x80000000)
- SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%106:vgpr_32 = V_MUL_LO_U32_e64 %110, %117, implicit $exec
%108:vgpr_32 = COPY %102, implicit $exec
S_BRANCH %bb.6
@@ -290,7 +290,7 @@ body: |
%68:vgpr_32 = PHI %107, %bb.10, %69, %bb.11
%162:vgpr_32 = PHI %137, %bb.10, %161, %bb.11
%163:vgpr_32 = PHI %129, %bb.10, %151, %bb.11
- SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%92:sreg_32 = S_ADD_I32 %91, 1, implicit-def dead $scc
%164:sreg_32 = V_CMP_GE_U32_e64 %92, %2, implicit $exec
%90:sreg_32 = SI_IF_BREAK %164, %89, implicit-def dead $scc
@@ -300,7 +300,7 @@ body: |
bb.13:
successors: %bb.14(0x04000000), %bb.2(0x7c000000)
- SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:sreg_32 = S_ADD_I32 %65, 1, implicit-def dead $scc
%165:sreg_32 = V_CMP_GE_U32_e64 %66, %3, implicit $exec
%64:sreg_32 = SI_IF_BREAK %165, %63, implicit-def dead $scc
@@ -310,7 +310,7 @@ body: |
bb.14:
successors: %bb.15(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%36:sreg_32 = S_ADD_I32 %35, 1, implicit-def dead $scc
%166:vgpr_32 = V_ADD_U32_e64 %36, %52, 0, implicit $exec
%167:vgpr_32 = V_ASHRREV_I32_e64 31, %163, implicit $exec
@@ -327,7 +327,7 @@ body: |
S_BRANCH %bb.15
bb.15:
- SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%175:vgpr_32 = V_ADD_U32_e64 %162, %52, 0, implicit $exec
%176:sreg_32_xm0 = V_READFIRSTLANE_B32 %175, implicit $exec
$sgpr0 = COPY %176
@@ -4266,7 +4266,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5010,7 +5010,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.9": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7043,7 +7043,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7245,7 +7245,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7429,7 +7429,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7841,7 +7841,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.15": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -12325,7 +12325,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %101, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13212,7 +13212,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.9": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %112, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15787,7 +15787,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %140, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16085,7 +16085,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %90, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16359,7 +16359,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17017,7 +17017,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.15": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %34, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
index 20932f9c0331d..00ea6dd596bf5 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/nested-loops-with-side-exits-a.mir
@@ -154,7 +154,7 @@ body: |
%46:sreg_32 = PHI undef %70:sreg_32, %bb.1, %71, %bb.14
%72:vgpr_32 = PHI undef %73:vgpr_32, %bb.1, %74, %bb.14
%75:vgpr_32 = PHI undef %73:vgpr_32, %bb.1, %76, %bb.14
- SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%44:sreg_32 = SI_IF_BREAK %36, %43, implicit-def dead $scc
%77:vgpr_32 = COPY %45, implicit $exec
%78:sreg_32 = S_ANDN2_B32 %31, $exec_lo, implicit-def dead $scc
@@ -195,7 +195,7 @@ body: |
%90:sreg_32 = PHI %96, %bb.4, %102, %bb.17
%94:sreg_32 = PHI undef %103:sreg_32, %bb.4, %104, %bb.17
%74:vgpr_32 = PHI %95, %bb.4, %25, %bb.17
- SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%105:sreg_32 = S_XOR_B32 %90, -1, implicit-def dead $scc
%92:sreg_32 = SI_IF_BREAK %87, %91, implicit-def dead $scc
%106:sreg_32 = S_ANDN2_B32 %82, $exec_lo, implicit-def dead $scc
@@ -248,12 +248,12 @@ body: |
successors: %bb.9(0x80000000)
%119:vgpr_32 = PHI %75, %bb.16, %72, %bb.11
- SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.9
bb.13:
%126:vgpr_32 = PHI %117, %bb.9, %124, %bb.10
- SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%127:vgpr_32 = V_ADD3_U32_e64 %24, %126, 100, implicit $exec
%128:sreg_32_xm0 = V_READFIRSTLANE_B32 %127, implicit $exec
$sgpr0 = COPY %128
@@ -266,7 +266,7 @@ body: |
%131:sreg_32 = PHI %130, %bb.18, %115, %bb.8
%71:sreg_32 = PHI undef %132:sreg_32, %bb.18, %112, %bb.8
%76:vgpr_32 = PHI undef %133:vgpr_32, %bb.18, %113, %bb.8
- SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%135:sreg_32 = S_ANDN2_B32 %61, $exec_lo, implicit-def dead $scc
%136:sreg_32 = S_AND_B32 %131, $exec_lo, implicit-def dead $scc
%69:sreg_32 = S_OR_B32 %135, %136, implicit-def dead $scc
@@ -280,7 +280,7 @@ body: |
bb.15:
successors: %bb.16(0x40000000), %bb.9(0x40000000)
- SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%140:sreg_32 = S_XOR_B32 %30, -1, implicit-def dead $scc
%123:sreg_32 = SI_IF %140, %bb.9, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.16
@@ -297,7 +297,7 @@ body: |
%141:sreg_32 = PHI %99, %bb.5, %111, %bb.7
%142:sreg_32 = PHI %99, %bb.5, %110, %bb.7
%104:sreg_32 = PHI undef %143:sreg_32, %bb.5, %108, %bb.7
- SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%144:sreg_32 = S_ANDN2_B32 %96, $exec_lo, implicit-def dead $scc
%145:sreg_32 = S_AND_B32 %142, $exec_lo, implicit-def dead $scc
%102:sreg_32 = S_OR_B32 %144, %145, implicit-def dead $scc
@@ -309,7 +309,7 @@ body: |
bb.18:
successors: %bb.8(0x40000000), %bb.14(0x40000000)
- SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%130:sreg_32 = S_MOV_B32 -1
%134:sreg_32 = SI_IF %84, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.8
@@ -2294,7 +2294,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3351,7 +3351,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4336,7 +4336,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4411,7 +4411,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4647,7 +4647,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5003,7 +5003,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.15": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5296,7 +5296,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5622,7 +5622,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -7972,7 +7972,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9219,7 +9219,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %98, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10359,7 +10359,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %125, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10434,7 +10434,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %122, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10702,7 +10702,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %134, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11138,7 +11138,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.15": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %44, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11449,7 +11449,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %100, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11823,7 +11823,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %92, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
index fbcf447271740..778c7c137eaae 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/sequence_2_loops.mir
@@ -115,7 +115,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%51:vgpr_32 = V_ADD_U32_e64 -2, %50, 0, implicit $exec
%52:vgpr_32 = V_MUL_LO_U32_e64 %30, %51, implicit $exec
%53:vgpr_32 = V_MUL_LO_U32_e64 100, %27, implicit $exec
@@ -136,7 +136,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %8, %59, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%62:vgpr_32 = V_ADD3_U32_e64 %59, %61, %29, implicit $exec
%63:vgpr_32 = V_ADD3_U32_e64 %62, %53, %41, implicit $exec
@@ -1774,7 +1774,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2205,7 +2205,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4192,7 +4192,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4635,7 +4635,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %56, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
index 4338457e42221..820eb937c53e3 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/simple-loop-3blocks.mir
@@ -63,7 +63,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DS_WRITE_B32_gfx9 %1, %8, 0, 0, implicit $exec :: (store (s32) into %ir.p, addrspace 3)
S_ENDPGM 0
...
@@ -618,7 +618,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1273,7 +1273,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
index a7713d7b1d91f..d139ec03fe611 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_basic_case.mir
@@ -97,7 +97,7 @@ body: |
bb.4:
%45:vreg_64 = PHI %8, %bb.1, %29, %bb.2
- SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%46:vgpr_32, %47:sreg_32_xm0_xexec = V_ADD_CO_U32_e64 %45.sub0, %1, 0, implicit $exec
%48:vgpr_32, dead $sgpr_null = V_ADDC_U32_e64 %45.sub1, %0, %47, 0, implicit $exec
%49:sreg_32_xm0 = V_READFIRSTLANE_B32 %46, implicit $exec
@@ -1368,7 +1368,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2862,7 +2862,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
index 69b4a4ba60ee9..9381db82edbb2 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_do_not_spill_restore_inside_loop.mir
@@ -121,7 +121,7 @@ body: |
%34:vgpr_32 = PHI %54, %bb.2, %57, %bb.3
%65:vgpr_32 = PHI %46, %bb.2, %27, %bb.3
%38:vgpr_32 = PHI %48, %bb.2, %56, %bb.3
- SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%32:sreg_32 = SI_IF_BREAK %64, %31, implicit-def dead $scc
SI_LOOP %32, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -129,7 +129,7 @@ body: |
bb.6:
successors: %bb.7(0x80000000)
- SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%66:vgpr_32 = V_MUL_LO_U32_e64 %38, %27, implicit $exec
GLOBAL_STORE_DWORD %16, %66, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%67:sreg_32 = S_MOV_B32 0
@@ -148,7 +148,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%75:vgpr_32 = V_ADD_U32_e64 %72, %74, 0, implicit $exec
%76:vgpr_32 = V_ADD_U32_e64 %75, %38, 0, implicit $exec
%77:vgpr_32 = V_ADD3_U32_e64 %76, %66, %75, implicit $exec
@@ -2276,7 +2276,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2421,7 +2421,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2815,7 +2815,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5371,7 +5371,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %55, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5492,7 +5492,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5880,7 +5880,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
index 97206f60d7adf..e1cf748656c42 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_common_dominator.mir
@@ -70,7 +70,7 @@ body: |
successors: %bb.5(0x40000000), %bb.3(0x40000000)
%25:vgpr_32 = PHI %22, %bb.0, %24, %bb.1
- SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:vgpr_32 = DS_READ_U8_gfx9 %2, 0, 0, implicit $exec :: (load (s8) from %ir.p2, addrspace 3)
%27:vgpr_32 = DS_READ_U8_gfx9 %2, 1, 0, implicit $exec :: (load (s8) from %ir.p2 + 1, addrspace 3)
%28:vgpr_32 = DS_READ_U8_gfx9 %2, 2, 0, implicit $exec :: (load (s8) from %ir.p2 + 2, addrspace 3)
@@ -108,7 +108,7 @@ body: |
successors: %bb.12(0x40000000), %bb.7(0x40000000)
%44:vgpr_32 = PHI %34, %bb.3, %43, %bb.4
- SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%45:vgpr_32 = V_LSHL_OR_B32_e64 %40, 16, %39, implicit $exec
%46:vgpr_32 = DS_READ_B32_gfx9 %2, 12, 0, implicit $exec :: (load (s32) from %ir.gep2, align 8, addrspace 3)
%47:vgpr_32 = V_ADD_U32_e64 %46, %44, 0, implicit $exec
@@ -186,11 +186,11 @@ body: |
successors: %bb.14(0x80000000)
%92:vgpr_32 = PHI %62, %bb.9, %70, %bb.10
- SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.14:
%93:vgpr_32 = PHI %51, %bb.7, %92, %bb.13
- SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%94:vgpr_32 = V_ADD3_U32_e64 %32, %93, 100, implicit $exec
%95:sreg_32_xm0 = V_READFIRSTLANE_B32 %94, implicit $exec
$sgpr0 = COPY %95
@@ -899,7 +899,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1596,7 +1596,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2953,7 +2953,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3001,7 +3001,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3902,7 +3902,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %23, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4599,7 +4599,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5956,7 +5956,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %69, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6004,7 +6004,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
index f2c85920feb3b..7e03a9f5d7052 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader1.mir
@@ -97,7 +97,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%52:vreg_64 = REG_SEQUENCE %23, %subreg.sub0, undef %53:vgpr_32, %subreg.sub1
%54:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %16, %33, %52, 0, implicit $exec
GLOBAL_STORE_DWORD %11, %54.sub0, 0, 0, implicit $exec :: (store (s32) into %ir.p5, addrspace 1)
@@ -1712,7 +1712,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3453,7 +3453,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
index 434b3ec6c24bb..0b5cb308ca1f2 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader3.mir
@@ -102,7 +102,7 @@ body: |
successors: %bb.5(0x80000000)
%44:vgpr_32 = PHI %39, %bb.1, %43, %bb.2
- SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%45:vreg_64 = REG_SEQUENCE %9, %subreg.sub0, %8, %subreg.sub1
%46:vreg_64 = REG_SEQUENCE %11, %subreg.sub0, %10, %subreg.sub1
%47:vreg_64 = REG_SEQUENCE %13, %subreg.sub0, %12, %subreg.sub1
@@ -181,7 +181,7 @@ body: |
%64:vgpr_32 = PHI %78, %bb.7, %106, %bb.8
%107:vgpr_32 = PHI %94, %bb.7, %105, %bb.8
%88:vgpr_32 = PHI %94, %bb.7, %85, %bb.8
- SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %47, %88, 0, 0, implicit $exec :: (store (s32) into %ir.p9, addrspace 1)
%108:vgpr_32 = GLOBAL_LOAD_DWORD %46, 0, 0, implicit $exec :: (load (s32) from %ir.p10, addrspace 1)
%109:vgpr_32 = V_MUL_LO_U32_e64 %107, %64, implicit $exec
@@ -196,7 +196,7 @@ body: |
bb.10:
successors: %bb.11(0x04000000), %bb.6(0x7c000000)
- SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%86:vgpr_32 = V_ADD_U32_e64 1, %85, 0, implicit $exec
%112:vgpr_32 = GLOBAL_LOAD_DWORD %45, 0, 0, implicit $exec :: (load (s32) from %ir.p11, addrspace 1)
%113:vgpr_32 = V_ADD_U32_e64 %86, %107, 0, implicit $exec
@@ -210,7 +210,7 @@ body: |
bb.11:
successors: %bb.12(0x80000000)
- SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%115:vreg_64 = REG_SEQUENCE %64, %subreg.sub0, undef %116:vgpr_32, %subreg.sub1
%117:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %64, %107, %115, 0, implicit $exec
%118:vgpr_32 = COPY %117.sub0
@@ -248,7 +248,7 @@ body: |
bb.13:
successors: %bb.14(0x04000000), %bb.5(0x7c000000)
- SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%141:vgpr_32 = V_ADD_U32_e64 %135, %1, 0, implicit $exec
GLOBAL_STORE_DWORD %50, %141, 0, 0, implicit $exec :: (store (s32) into %ir.p5, addrspace 1)
%62:vgpr_32 = V_ADD_U32_e64 1, %61, 0, implicit $exec
@@ -261,7 +261,7 @@ body: |
bb.14:
successors: %bb.15(0x40000000), %bb.16(0x40000000)
- SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%143:vgpr_32 = V_MUL_LO_U32_e64 100, %44, implicit $exec
%144:sreg_32 = V_CMP_GT_U32_e64 %143, %141, implicit $exec
%145:sreg_32 = SI_IF %144, %bb.16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -276,7 +276,7 @@ body: |
successors: %bb.17(0x80000000)
%146:vgpr_32 = PHI %143, %bb.14, %141, %bb.15
- SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%147:sreg_32 = S_MOV_B32 0
bb.17:
@@ -291,7 +291,7 @@ body: |
S_BRANCH %bb.18
bb.18:
- SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%153:vgpr_32 = V_ADD_U32_e64 100, %37, 0, implicit $exec
%154:vgpr_32 = V_MUL_LO_U32_e64 %143, %150, implicit $exec
%155:vgpr_32 = V_MUL_LO_U32_e64 %153, %154, implicit $exec
@@ -2356,7 +2356,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5241,7 +5241,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5866,7 +5866,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.10": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -6413,7 +6413,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.11": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8038,7 +8038,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.13": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8447,7 +8447,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8629,7 +8629,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -8859,7 +8859,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.18": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11228,7 +11228,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %42, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -13991,7 +13991,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %103, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14508,7 +14508,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.10": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %91, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -14950,7 +14950,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.11": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %82, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16112,7 +16112,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.13": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %121, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16372,7 +16372,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %58, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16554,7 +16554,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %145, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -16778,7 +16778,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.18": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %149, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
index 231fcdea8cfde..26896997f786e 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_emit_restore_in_loop_preheader4.mir
@@ -104,7 +104,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%58:vgpr_32 = V_ADD_U32_e64 -2, %57, 0, implicit $exec
%59:vgpr_32 = V_MUL_LO_U32_e64 %37, %58, implicit $exec
%60:vgpr_32 = V_MUL_LO_U32_e64 100, %34, implicit $exec
@@ -125,7 +125,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %10, %67, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%69:vgpr_32 = V_ADD_U32_e64 %67, %66, 0, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %69, %36, %60, implicit $exec
@@ -2157,7 +2157,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2604,7 +2604,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4793,7 +4793,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5232,7 +5232,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
index 2d3b9bf2c0c1e..6103b28c345f4 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_keep_spilled_reg_live.mir
@@ -87,7 +87,7 @@ body: |
bb.4:
%39:vreg_64 = PHI %7, %bb.1, %21, %bb.2
- SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%40:sreg_32_xm0 = V_READFIRSTLANE_B32 %39.sub0, implicit $exec
%41:sreg_32_xm0 = V_READFIRSTLANE_B32 %39.sub1, implicit $exec
$sgpr0 = COPY %40
@@ -1046,7 +1046,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2169,7 +2169,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %16, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
index cb14edadc7a0e..7407fd3d7542b 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills1.mir
@@ -144,7 +144,7 @@ body: |
%76:vgpr_32 = PHI %34, %bb.1, %49, %bb.2
%77:vgpr_32 = PHI %40, %bb.1, %50, %bb.2
%78:vgpr_32 = PHI %37, %bb.1, %48, %bb.2
- SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%79:vreg_64 = REG_SEQUENCE %25, %subreg.sub0, undef %80:vgpr_32, %subreg.sub1
%81:vreg_64, $sgpr_null = V_MAD_U64_U32_e64 %75, %27, %79, 0, implicit $exec
%82:vreg_64 = REG_SEQUENCE %76, %subreg.sub0, undef %83:vgpr_32, %subreg.sub1
@@ -410,7 +410,7 @@ body: |
bb.8:
successors: %bb.11(0x40000000), %bb.9(0x40000000)
- SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%321:sreg_32 = V_CMP_GT_U32_e64 %27, %304, implicit $exec
%322:sreg_32 = S_MOV_B32 0
%323:sreg_32 = SI_IF %321, %bb.9, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -450,7 +450,7 @@ body: |
successors: %bb.13(0x40000000), %bb.14(0x40000000)
%341:sreg_32 = PHI %324, %bb.9, %337, %bb.10
- SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%342:vreg_64 = REG_SEQUENCE %1, %subreg.sub0, %0, %subreg.sub1
%343:vreg_64 = REG_SEQUENCE %3, %subreg.sub0, %2, %subreg.sub1
%344:sreg_32 = SI_IF %341, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -463,7 +463,7 @@ body: |
GLOBAL_STORE_DWORD %20, %345, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
bb.14:
- SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%346:vgpr_32 = GLOBAL_LOAD_UBYTE %342, 0, 0, implicit $exec :: (load (s8) from %ir.p10, addrspace 1)
%347:vgpr_32 = GLOBAL_LOAD_UBYTE %342, 1, 0, implicit $exec :: (load (s8) from %ir.p10 + 1, addrspace 1)
%348:vgpr_32 = V_LSHL_OR_B32_e64 %347, 8, %346, implicit $exec
@@ -3743,7 +3743,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -18969,7 +18969,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19748,7 +19748,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -19940,7 +19940,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.14": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -23810,7 +23810,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %46, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39036,7 +39036,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %320, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -39815,7 +39815,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %334, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40007,7 +40007,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.14": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %344, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
index 772ac74291933..f4cfb571a117a 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills2.mir
@@ -401,7 +401,7 @@ body: |
%332:vgpr_32 = PHI %58, %bb.0, %104, %bb.1
%333:vgpr_32 = PHI %65, %bb.0, %325, %bb.1
%334:vgpr_32 = PHI %81, %bb.0, %106, %bb.1
- SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%335:sreg_32 = V_CMP_GT_U32_e64 %34, %333, implicit $exec
%336:sreg_32 = S_MOV_B32 0
%337:sreg_32 = SI_IF %335, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -439,7 +439,7 @@ body: |
successors: %bb.7(0x40000000), %bb.8(0x40000000)
%351:sreg_32 = PHI %338, %bb.3, %348, %bb.4
- SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%352:vgpr_32 = V_LSHL_OR_B32_e64 %345, 16, %344, implicit $exec
%353:sreg_32 = SI_IF %351, %bb.8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
@@ -451,7 +451,7 @@ body: |
GLOBAL_STORE_DWORD %29, %354, 0, 0, implicit $exec :: (store (s32) into %ir.p1, addrspace 1)
bb.8:
- SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%355:vgpr_32 = V_ADD_U32_e64 %352, %82, 0, implicit $exec
GLOBAL_STORE_DWORD %28, %355, 0, 0, implicit $exec :: (store (s32) into %ir.p9, addrspace 1)
S_ENDPGM 0
@@ -19821,7 +19821,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20540,7 +20540,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20708,7 +20708,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -40513,7 +40513,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %84, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41232,7 +41232,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %346, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -41400,7 +41400,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %353, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
index 324976cec20e5..a7b06da340f3b 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_multiple_spills3.mir
@@ -100,7 +100,7 @@ body: |
successors: %bb.3(0x40000000), %bb.14(0x40000000)
%43:vgpr_32 = PHI %40, %bb.0, %42, %bb.1
- SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%44:sreg_32 = SI_IF %25, %bb.14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
@@ -291,7 +291,7 @@ body: |
%216:vgpr_32 = PHI %217, %bb.14, %218, %bb.13
%219:vgpr_32 = PHI %220, %bb.14, %204, %bb.13
%221:vgpr_32 = PHI %222, %bb.14, %223, %bb.13
- SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%225:vreg_64 = REG_SEQUENCE %2, %subreg.sub0, %1, %subreg.sub1
%226:vreg_64 = REG_SEQUENCE %4, %subreg.sub0, %3, %subreg.sub1
S_BRANCH %bb.17
@@ -300,7 +300,7 @@ body: |
successors: %bb.12(0x40000000), %bb.10(0x40000000)
%227:vgpr_32 = PHI %147, %bb.5, %169, %bb.6
- SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%228:vgpr_32 = V_ADD_U32_e64 100, %102, 0, implicit $exec
%229:vgpr_32 = COPY %155.sub1
%230:vgpr_32 = COPY %155.sub2
@@ -407,7 +407,7 @@ body: |
%315:vgpr_32 = PHI %231, %bb.10, %264, %bb.11
%316:vgpr_32 = PHI %232, %bb.10, %264, %bb.11
%317:vgpr_32 = PHI %136, %bb.10, %264, %bb.11
- SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%223:vgpr_32 = V_ADD_U32_e64 %308, %227, 0, implicit $exec
%215:vgpr_32 = V_SUB_U32_e64 %43, %318, 0, implicit $exec
%319:vgpr_32 = V_ADD_U32_e64 %223, %215, 0, implicit $exec
@@ -476,7 +476,7 @@ body: |
bb.16:
successors: %bb.14(0x80000000)
- SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%364:vgpr_32 = V_ADD_U32_e64 %48, %363, 0, implicit $exec
%365:vgpr_32 = V_ADD_U32_e64 %52, %363, 0, implicit $exec
%366:sreg_64 = SI_PC_ADD_REL_OFFSET target-flags(amdgpu-gotprel32-lo) @array7, target-flags(amdgpu-gotprel32-hi) @array7, implicit-def dead $scc
@@ -2178,7 +2178,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -10889,7 +10889,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -11076,7 +11076,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -15259,7 +15259,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -17558,7 +17558,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.16": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -20530,7 +20530,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %41, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29241,7 +29241,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %224, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -29428,7 +29428,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %161, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -33611,7 +33611,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %263, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -35904,7 +35904,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.16": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %358, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
index f8e354de9aa0c..2b3f3de5d5a23 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_in_common_dominator_and_optimize_restores.mir
@@ -163,7 +163,7 @@ body: |
%109:vgpr_32 = PHI %31, %bb.0, %24, %bb.1
%110:vgpr_32 = PHI %39, %bb.0, %1, %bb.1
%111:vgpr_32 = PHI %35, %bb.0, %72, %bb.1
- SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%112:vgpr_32 = V_ADD_U32_e64 %97, %56, 0, implicit $exec
%113:vgpr_32 = V_MUL_LO_U32_e64 %112, %49, implicit $exec
%114:vgpr_32 = V_SUB_U32_e64 %113, %39, 0, implicit $exec
@@ -4296,7 +4296,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -9224,7 +9224,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %65, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
index 0d595ede9c992..e740457b46c6f 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_livethrough_reg.mir
@@ -88,7 +88,7 @@ body: |
%43:sreg_32 = PHI %44, %bb.5, %45, %bb.6
%46:vgpr_32 = PHI undef %47:vgpr_32, %bb.5, %48, %bb.6
- SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%50:sreg_32 = SI_IF %43, %bb.7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
@@ -105,7 +105,7 @@ body: |
%52:sreg_32 = PHI %27, %bb.1, %42, %bb.2
%44:sreg_32 = PHI %17, %bb.1, %39, %bb.2
%22:vreg_64 = PHI %20, %bb.1, %36, %bb.2
- SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%49:sreg_32 = SI_IF %52, %bb.3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -123,7 +123,7 @@ body: |
successors: %bb.8(0x04000000), %bb.1(0x7c000000)
%26:vgpr_32 = PHI %46, %bb.3, %51, %bb.4
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%24:sreg_32 = S_ADD_I32 %23, 1, implicit-def dead $scc
%56:sreg_32 = V_CMP_GE_U32_e64 %24, %0, implicit $exec
%19:sreg_32 = SI_IF_BREAK %56, %18, implicit-def dead $scc
@@ -131,7 +131,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
GLOBAL_STORE_DWORD %7, %26, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
S_ENDPGM 0
...
@@ -1312,7 +1312,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1596,7 +1596,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -1923,7 +1923,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2101,7 +2101,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3371,7 +3371,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %49, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3621,7 +3621,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %29, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -3908,7 +3908,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4056,7 +4056,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
index eb605289042d5..93338b3e5abe2 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/test_ers_spill_loop_value_in_exit_block.mir
@@ -71,7 +71,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%27:vgpr_32 = V_ASHRREV_I32_e64 31, %25, implicit $exec
%28:vreg_64 = REG_SEQUENCE %25, %subreg.sub0, %27, %subreg.sub1
%29:vreg_64 = nsw V_LSHLREV_B64_pseudo_e64 2, %28, implicit $exec
@@ -933,7 +933,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.2": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2733,7 +2733,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.2": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
index 904a7971321ea..abd655422748f 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/triple-nested-loops.mir
@@ -141,7 +141,7 @@ body: |
bb.4:
successors: %bb.5(0x04000000), %bb.2(0x7c000000)
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%65:vgpr_32 = V_ADD_U32_e64 %59, %64, 0, implicit $exec
%37:vgpr_32 = V_MUL_LO_U32_e64 %29, %65, implicit $exec
%35:sreg_32 = S_ADD_I32 %34, 2, implicit-def dead $scc
@@ -153,7 +153,7 @@ body: |
bb.5:
successors: %bb.6(0x04000000), %bb.1(0x7c000000)
- SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%30:vgpr_32 = V_SUB_U32_e64 %44, %64, 0, implicit $exec
%28:sreg_32 = S_ADD_I32 %27, 1, implicit-def dead $scc
%67:sreg_32 = V_CMP_GE_U32_e64 %28, %1, implicit $exec
@@ -162,7 +162,7 @@ body: |
S_BRANCH %bb.6
bb.6:
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%68:vgpr_32 = V_MAX_U32_e64 1, %1, implicit $exec
%69:vgpr_32 = V_ADD_U32_e64 %68, %23, 0, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %36, %29, %65, implicit $exec
@@ -2339,7 +2339,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.4": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2598,7 +2598,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.5": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2799,7 +2799,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5393,7 +5393,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.4": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5698,7 +5698,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.5": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %33, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -5949,7 +5949,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
index d25842ccbf55b..5413abbe8c1ee 100644
--- a/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
+++ b/llvm/test/CodeGen/AMDGPU/NextUseAnalysis/two-sequential-loops.mir
@@ -140,7 +140,7 @@ body: |
%28:vgpr_32 = PHI %40, %bb.2, %52, %bb.3
%60:vgpr_32 = PHI %42, %bb.2, %21, %bb.3
%32:vgpr_32 = PHI %44, %bb.2, %51, %bb.3
- SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:sreg_32 = SI_IF_BREAK %59, %25, implicit-def dead $scc
SI_LOOP %26, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
@@ -148,7 +148,7 @@ body: |
bb.6:
successors: %bb.7(0x80000000)
- SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%61:vgpr_32 = V_MUL_LO_U32_e64 %32, %31, implicit $exec
GLOBAL_STORE_DWORD %10, %61, 0, 0, implicit $exec :: (store (s32) into %ir.p3, addrspace 1)
%62:sreg_32 = S_MOV_B32 0
@@ -167,7 +167,7 @@ body: |
S_BRANCH %bb.8
bb.8:
- SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%70:vgpr_32 = V_ADD3_U32_e64 %67, %69, %32, implicit $exec
%71:vgpr_32 = V_ADD3_U32_e64 %70, %61, %60, implicit $exec
%72:vgpr_32 = V_ADD3_U32_e64 %21, %71, 100, implicit $exec
@@ -1931,7 +1931,7 @@ body: |
# CHECK-GX-JSON: }
# CHECK-GX-JSON-NEXT: },
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2044,7 +2044,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.6": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -2400,7 +2400,7 @@ body: |
# CHECK-GX-JSON-NEXT: ],
# CHECK-GX-JSON-NEXT: "bb.8": [
# CHECK-GX-JSON-NEXT: {
-# CHECK-GX-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-GX-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-GX-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-GX-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4488,7 +4488,7 @@ body: |
# CHECK-CP-JSON: }
# CHECK-CP-JSON-NEXT: },
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %50, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4625,7 +4625,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.6": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %26, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
@@ -4993,7 +4993,7 @@ body: |
# CHECK-CP-JSON-NEXT: ],
# CHECK-CP-JSON-NEXT: "bb.8": [
# CHECK-CP-JSON-NEXT: {
-# CHECK-CP-JSON-NEXT: "instr": "SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
+# CHECK-CP-JSON-NEXT: "instr": "bb-prolog SI_END_CF %64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec",
# CHECK-CP-JSON-NEXT: "id": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "head-len": {{[0-9]+}},
# CHECK-CP-JSON-NEXT: "tail-len": {{[0-9]+}},
diff --git a/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir b/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
index 5eb8c608708de..0800fb3fc9fd8 100644
--- a/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
+++ b/llvm/test/CodeGen/AMDGPU/SRSRC-GIT-clobber-check.mir
@@ -29,7 +29,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $sgpr0
- $exec_lo = S_MOV_B32 -1
+ $exec_lo = bb-prolog S_MOV_B32 -1
renamable $vgpr0 = V_MBCNT_LO_U32_B32_e64 -1, 0, implicit $exec
renamable $sgpr0 = S_BFE_U32 killed renamable $sgpr0, 589836, implicit-def dead $scc
renamable $vcc_lo = V_CMP_GT_U32_e64 killed $sgpr0, killed $vgpr0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll b/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
index 7694e30a612af..5f4dd2a8b850c 100644
--- a/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
+++ b/llvm/test/CodeGen/AMDGPU/bb-prolog-spill-during-regalloc.ll
@@ -27,10 +27,10 @@ define i32 @prolog_spill(i32 %arg0, i32 %arg1, i32 %arg2) {
; REGALLOC-NEXT: bb.1.Flow:
; REGALLOC-NEXT: successors: %bb.2(0x40000000), %bb.4(0x40000000)
; REGALLOC-NEXT: {{ $}}
- ; REGALLOC-NEXT: $vgpr63 = SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
- ; REGALLOC-NEXT: $sgpr4 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 0, implicit-def $sgpr4_sgpr5
- ; REGALLOC-NEXT: $sgpr5 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 1
- ; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = S_OR_SAVEEXEC_B64 killed renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; REGALLOC-NEXT: $vgpr63 = bb-prolog SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
+ ; REGALLOC-NEXT: $sgpr4 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 0, implicit-def $sgpr4_sgpr5
+ ; REGALLOC-NEXT: $sgpr5 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 1
+ ; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def dead $scc, implicit $exec
; REGALLOC-NEXT: $vgpr0 = SI_SPILL_V32_RESTORE %stack.6, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.6, addrspace 5)
; REGALLOC-NEXT: SI_SPILL_V32_SAVE killed $vgpr0, %stack.5, $sgpr32, 0, implicit $exec :: (store (s32) into %stack.5, addrspace 5)
; REGALLOC-NEXT: renamable $sgpr4_sgpr5 = S_AND_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
@@ -60,10 +60,10 @@ define i32 @prolog_spill(i32 %arg0, i32 %arg1, i32 %arg2) {
; REGALLOC-NEXT: S_BRANCH %bb.1
; REGALLOC-NEXT: {{ $}}
; REGALLOC-NEXT: bb.4.bb.3:
- ; REGALLOC-NEXT: $vgpr63 = SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
- ; REGALLOC-NEXT: $sgpr4 = SI_RESTORE_S32_FROM_VGPR $vgpr63, 2, implicit-def $sgpr4_sgpr5
- ; REGALLOC-NEXT: $sgpr5 = SI_RESTORE_S32_FROM_VGPR killed $vgpr63, 3
- ; REGALLOC-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; REGALLOC-NEXT: $vgpr63 = bb-prolog SI_SPILL_WWM_V32_RESTORE %stack.2, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.2, addrspace 5)
+ ; REGALLOC-NEXT: $sgpr4 = bb-prolog SI_RESTORE_S32_FROM_VGPR $vgpr63, 2, implicit-def $sgpr4_sgpr5
+ ; REGALLOC-NEXT: $sgpr5 = bb-prolog SI_RESTORE_S32_FROM_VGPR killed $vgpr63, 3
+ ; REGALLOC-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; REGALLOC-NEXT: $vgpr0 = SI_SPILL_V32_RESTORE %stack.5, $sgpr32, 0, implicit $exec :: (load (s32) from %stack.5, addrspace 5)
; REGALLOC-NEXT: renamable $vgpr0 = V_LSHL_ADD_U32_e64 killed $vgpr0, 2, $vgpr0, implicit $exec
; REGALLOC-NEXT: SI_RETURN implicit killed $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir b/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
index 875e2fcc671e6..c6def72467cab 100644
--- a/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
+++ b/llvm/test/CodeGen/AMDGPU/block-should-not-be-in-alive-blocks.mir
@@ -61,7 +61,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.7(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 killed [[S_XOR_B32_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 killed [[S_XOR_B32_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: [[COPY4:%[0-9]+]]:vgpr_32 = COPY killed [[COPY2]]
; CHECK-NEXT: [[S_AND_B32_1:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def dead $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_1]], implicit-def dead $scc
@@ -75,7 +75,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed [[S_AND_B32_1]], implicit-def dead $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed [[S_AND_B32_1]], implicit-def dead $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.2(0x40000000), %bb.5(0x40000000)
@@ -128,7 +128,7 @@ body: |
S_BRANCH %bb.5
bb.7:
- SI_END_CF %14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %14, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll b/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
index 41b3bfdf523e8..604abf7e27e0b 100644
--- a/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
+++ b/llvm/test/CodeGen/AMDGPU/branch-folding-implicit-def-subreg.ll
@@ -133,7 +133,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.9(0x40000000), %bb.10(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr18_sgpr19, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr18_sgpr19, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr18_sgpr19, implicit-def dead $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr40_sgpr41, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.10, implicit $exec
@@ -150,7 +150,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.11(0x40000000), %bb.12(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr46_sgpr47, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.12, implicit $exec
@@ -167,7 +167,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.13(0x40000000), %bb.14(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr44_sgpr45, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr12_sgpr13 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.14, implicit $exec
@@ -184,7 +184,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.15(0x40000000), %bb.16(0x40000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr42_sgpr43, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr0_vgpr1:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: $sgpr12_sgpr13 = S_AND_SAVEEXEC_B64 $sgpr42_sgpr43, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr36_sgpr37 = S_XOR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.16, implicit $exec
@@ -208,7 +208,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.17(0x40000000), %bb.18(0x40000000)
; GFX90A-NEXT: liveins: $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr36_sgpr37, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr36_sgpr37, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr66_sgpr67, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.18, implicit $exec
@@ -225,7 +225,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.19(0x40000000), %bb.20(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr64_sgpr65, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.20, implicit $exec
@@ -242,7 +242,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.21(0x40000000), %bb.22(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr54_sgpr55, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.22, implicit $exec
@@ -259,7 +259,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.23(0x40000000), %bb.24(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr52_sgpr53, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.24, implicit $exec
@@ -276,7 +276,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.25(0x40000000), %bb.26(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr50_sgpr51, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.26, implicit $exec
@@ -293,7 +293,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.27(0x40000000), %bb.28(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr48_sgpr49, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr48_sgpr49, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECZ %bb.28, implicit $exec
@@ -310,7 +310,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.34(0x40000000), %bb.29(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr38_sgpr39, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX90A-NEXT: renamable $sgpr4_sgpr5 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: S_CBRANCH_EXECNZ %bb.34, implicit $exec
@@ -319,7 +319,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.30(0x40000000), %bb.31(0x40000000)
; GFX90A-NEXT: liveins: $sgpr4_sgpr5, $sgpr34_sgpr35, $sgpr68_sgpr69, $vgpr40_vgpr41:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX90A-NEXT: dead renamable $sgpr4_sgpr5 = S_AND_B64 killed renamable $sgpr34_sgpr35, $exec, implicit-def $scc
; GFX90A-NEXT: renamable $sgpr4 = S_CSELECT_B32 1, 0, implicit killed $scc
; GFX90A-NEXT: S_CMP_LG_U32 killed renamable $sgpr4, 1, implicit-def $scc
@@ -404,7 +404,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.6(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr24_sgpr25, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr24_sgpr25, implicit-def dead $scc
; GFX90A-NEXT: S_BRANCH %bb.6
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.37.bb27:
@@ -443,7 +443,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.36(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr38_sgpr39, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr38_sgpr39, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr38_sgpr39 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr48_sgpr49 = S_AND_B64 killed renamable $sgpr42_sgpr43, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr50_sgpr51 = S_AND_B64 killed renamable $sgpr66_sgpr67, $exec, implicit-def dead $scc
@@ -495,7 +495,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.38(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr68_sgpr69, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr40_sgpr41, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr40_sgpr41, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr42_sgpr43 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr66_sgpr67 = S_AND_B64 killed renamable $sgpr44_sgpr45, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr52_sgpr53 = S_AND_B64 killed renamable $sgpr68_sgpr69, $exec, implicit-def dead $scc
@@ -545,7 +545,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.40(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr42_sgpr43, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr42_sgpr43, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr44_sgpr45 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr68_sgpr69 = S_AND_B64 killed renamable $sgpr46_sgpr47, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_AND_B64 killed renamable $sgpr54_sgpr55, $exec, implicit-def dead $scc
@@ -643,7 +643,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.42(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr58_sgpr59, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr66_sgpr67, $sgpr68_sgpr69, $sgpr70_sgpr71, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr44_sgpr45, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr44_sgpr45, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr46_sgpr47 = S_XOR_B64 $exec, -1, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_AND_B64 killed renamable $sgpr62_sgpr63, $exec, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr64_sgpr65 = S_AND_B64 killed renamable $sgpr64_sgpr65, $exec, implicit-def dead $scc
@@ -758,7 +758,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.45(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr58_sgpr59, $sgpr60_sgpr61, $sgpr62_sgpr63, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr62_sgpr63, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr62_sgpr63, implicit-def dead $scc
; GFX90A-NEXT: S_BRANCH %bb.45
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.56.bb90:
@@ -862,7 +862,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: successors: %bb.61(0x80000000)
; GFX90A-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr17, $sgpr23, $vgpr30, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19:0x0000000000000003, $sgpr24_sgpr25, $sgpr28_sgpr29, $sgpr30_sgpr31, $sgpr34_sgpr35, $sgpr36_sgpr37, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr62_sgpr63, $sgpr64_sgpr65, $sgpr20_sgpr21_sgpr22_sgpr23:0x000000000000003C, $sgpr24_sgpr25_sgpr26_sgpr27:0x00000000000000F0, $vgpr0_vgpr1:0x000000000000000F, $vgpr2_vgpr3:0x0000000000000003, $vgpr4_vgpr5:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr10_vgpr11:0x0000000000000003, $vgpr12_vgpr13:0x0000000000000003, $vgpr14_vgpr15:0x0000000000000003, $vgpr16_vgpr17:0x000000000000000F, $vgpr18_vgpr19:0x0000000000000003, $vgpr20_vgpr21:0x0000000000000003, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr54_sgpr55, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr54_sgpr55, implicit-def dead $scc
; GFX90A-NEXT: renamable $sgpr54_sgpr55 = S_MOV_B64 0
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.61.Flow30:
@@ -908,7 +908,7 @@ define amdgpu_kernel void @f1(ptr addrspace(1) %arg, ptr addrspace(1) %arg1, i64
; GFX90A-NEXT: liveins: $sgpr14, $sgpr16, $sgpr17, $vgpr31, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x000000000000000F, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr34_sgpr35, $sgpr38_sgpr39, $sgpr40_sgpr41, $sgpr42_sgpr43, $sgpr44_sgpr45, $sgpr46_sgpr47, $sgpr48_sgpr49, $sgpr50_sgpr51, $sgpr52_sgpr53, $sgpr54_sgpr55, $sgpr64_sgpr65, $sgpr66_sgpr67, $vgpr0_vgpr1:0x000000000000000F, $vgpr6_vgpr7:0x000000000000000F, $vgpr8_vgpr9:0x000000000000000F, $vgpr40_vgpr41:0x000000000000000F, $vgpr42_vgpr43:0x000000000000000F, $vgpr44_vgpr45:0x000000000000000F, $vgpr46_vgpr47:0x000000000000000F, $vgpr54_vgpr55:0x000000000000000F, $vgpr56_vgpr57:0x000000000000000F, $vgpr58_vgpr59:0x000000000000000F, $vgpr60_vgpr61:0x000000000000000F, $vgpr62_vgpr63:0x000000000000000F, $sgpr0_sgpr1_sgpr2_sgpr3
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: $sgpr12_sgpr13 = S_ANDN2_SAVEEXEC_B64 $sgpr12_sgpr13, implicit-def $exec, implicit-def $scc, implicit $exec
- ; GFX90A-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
+ ; GFX90A-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr12_sgpr13, implicit-def dead $scc
; GFX90A-NEXT: {{ $}}
; GFX90A-NEXT: bb.66.Flow14:
; GFX90A-NEXT: successors: %bb.8(0x80000000)
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir b/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
index 4b5b94d359c63..68b3fdd1c5430 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-extend-pruned-subrange.mir
@@ -53,7 +53,7 @@ body: |
bb.4:
successors: %bb.5
- $exec = S_OR_B64 $exec, %20, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %20, implicit-def $scc
bb.5:
successors: %bb.7, %bb.6
@@ -90,7 +90,7 @@ body: |
bb.10:
successors: %bb.9
- $exec = S_OR_B64 $exec, %29, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %29, implicit-def $scc
%36:vgpr_32 = COPY %28
%37:sreg_64_xexec = V_CMP_NE_U32_e64 0, %36, implicit $exec
%38:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, %37, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
index 58994a9acf9aa..ead8128439f3d 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-copymi-not-live.mir
@@ -88,7 +88,7 @@ body: |
bb.10:
successors: %bb.12(0x80000000)
- $exec = S_OR_B64 $exec, killed %22, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %22, implicit-def $scc
S_BRANCH %bb.12
bb.11:
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
index 88717d21a05be..c19569053e105 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subranges-another-prune-error.mir
@@ -91,7 +91,7 @@ body: |
bb.5:
successors: %bb.9(0x80000000)
- $exec = S_OR_B64 $exec, %46, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %46, implicit-def $scc
%47:vgpr_32 = COPY killed %48
%49:vgpr_32 = COPY killed %50
%51:vreg_128 = COPY killed %52
@@ -111,7 +111,7 @@ body: |
bb.6:
successors: %bb.7(0x40000000), %bb.13(0x40000000)
- $exec = S_OR_B64 $exec, killed %60, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %60, implicit-def $scc
%61:sreg_64 = V_CMP_NE_U32_e64 0, killed %62, implicit $exec
%63:vreg_128 = COPY killed %64
%65:vgpr_32 = COPY killed %66
@@ -140,7 +140,7 @@ body: |
bb.9:
successors: %bb.6(0x04000000), %bb.4(0x7c000000)
- $exec = S_OR_B64 $exec, %45, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %45, implicit-def $scc
%62:vgpr_32 = COPY killed %42
%66:vgpr_32 = COPY killed %41
%76:vgpr_32 = COPY killed %40
@@ -182,7 +182,7 @@ body: |
bb.12:
successors: %bb.15(0x40000000), %bb.5(0x40000000)
- $exec = S_OR_B64 $exec, killed %85, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %85, implicit-def $scc
%87:sreg_64 = V_CMP_LT_U32_e64 11, killed %28, implicit $exec
%88:vgpr_32 = V_MOV_B32_e32 -1, implicit $exec
%56:sreg_64 = COPY %24
@@ -199,7 +199,7 @@ body: |
bb.13:
successors: %bb.14(0x40000000), %bb.16(0x40000000)
- $exec = S_OR_B64 $exec, killed %67, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %67, implicit-def $scc
%91:vgpr_32 = COPY killed %65
%92:vreg_128 = COPY killed %63
%93:sreg_64 = V_CMP_NE_U32_e64 0, killed %91, implicit $exec
@@ -232,7 +232,7 @@ body: |
bb.16:
successors: %bb.3(0x80000000)
- $exec = S_OR_B64 $exec, killed %95, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %95, implicit-def $scc
%103:vreg_128 = COPY killed %94
%104:vreg_128 = COPY killed %103
%5:vreg_128 = COPY killed %104
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir b/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
index 3beb6b6f02d36..bd2a4469021bb 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-subregjoin-fullcopy.mir
@@ -84,7 +84,7 @@ body: |
bb.7:
successors: %bb.6(0x40000000), %bb.8(0x40000000)
- $exec = S_OR_B64 $exec, %24, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %24, implicit-def $scc
%25:vreg_128 = COPY killed %21
%26:vreg_128 = COPY killed %20
%27:sreg_64 = COPY killed %19
@@ -97,7 +97,7 @@ body: |
bb.8:
successors: %bb.5(0x80000000)
- $exec = S_OR_B64 $exec, killed %28, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %28, implicit-def $scc
%29:vreg_128 = COPY killed %26
%1:vreg_128 = COPY killed %29
S_BRANCH %bb.5
diff --git a/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir b/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
index 481befdb0e187..83cf664af3233 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescer-with-subregs-bad-identical.mir
@@ -90,7 +90,7 @@ body: |
bb.7:
successors: %bb.6(0x80000000)
- $exec = S_OR_B64 $exec, killed %23, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %23, implicit-def $scc
%30:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
%15:vreg_128 = COPY %13
%16:vgpr_32 = COPY killed %30
@@ -157,7 +157,7 @@ body: |
bb.16:
successors: %bb.17(0x80000000)
- $exec = S_OR_B64 $exec, killed %50, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %50, implicit-def $scc
S_BRANCH %bb.17
bb.17:
diff --git a/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir b/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
index 25b2105bacd2c..85aac1fd627b8 100644
--- a/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
+++ b/llvm/test/CodeGen/AMDGPU/coalescing-with-subregs-in-loop-bug.mir
@@ -89,7 +89,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- $exec = S_OR_B64 $exec, killed %1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed %1, implicit-def $scc
%33:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
undef %32.sub0:vreg_128 = COPY killed %31.sub0
%32.sub2:vreg_128 = COPY %33
diff --git a/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll b/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
index dbed069faef1a..3f8aa97b637bd 100644
--- a/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
+++ b/llvm/test/CodeGen/AMDGPU/codegen-prepare-addrspacecast-non-null.ll
@@ -351,10 +351,9 @@ define void @recursive_phis(i1 %cond, ptr addrspace(5) %ptr) {
; GISEL-ASM: ; %bb.0: ; %entry
; GISEL-ASM-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; GISEL-ASM-NEXT: v_and_b32_e32 v0, 1, v0
+; GISEL-ASM-NEXT: s_lshr_b32 s6, s32, 6
; GISEL-ASM-NEXT: s_mov_b64 s[4:5], exec
; GISEL-ASM-NEXT: v_cmp_ne_u32_e32 vcc, 0, v0
-; GISEL-ASM-NEXT: s_lshr_b32 s6, s32, 6
-; GISEL-ASM-NEXT: s_xor_b64 s[4:5], vcc, s[4:5]
; GISEL-ASM-NEXT: v_mov_b32_e32 v0, s6
; GISEL-ASM-NEXT: s_and_saveexec_b64 s[6:7], vcc
; GISEL-ASM-NEXT: ; %bb.1: ; %then
@@ -362,19 +361,20 @@ define void @recursive_phis(i1 %cond, ptr addrspace(5) %ptr) {
; GISEL-ASM-NEXT: ; %bb.2: ; %finallyendcf.split
; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[6:7]
; GISEL-ASM-NEXT: s_mov_b64 s[8:9], src_private_base
-; GISEL-ASM-NEXT: s_mov_b64 s[6:7], 0
+; GISEL-ASM-NEXT: s_xor_b64 s[6:7], vcc, s[4:5]
+; GISEL-ASM-NEXT: s_mov_b64 s[4:5], 0
; GISEL-ASM-NEXT: v_mov_b32_e32 v1, s9
; GISEL-ASM-NEXT: v_mov_b32_e32 v2, 7
; GISEL-ASM-NEXT: .LBB11_3: ; %finally
; GISEL-ASM-NEXT: ; =>This Inner Loop Header: Depth=1
-; GISEL-ASM-NEXT: s_and_b64 s[8:9], exec, s[4:5]
-; GISEL-ASM-NEXT: s_or_b64 s[6:7], s[8:9], s[6:7]
+; GISEL-ASM-NEXT: s_and_b64 s[8:9], exec, s[6:7]
+; GISEL-ASM-NEXT: s_or_b64 s[4:5], s[8:9], s[4:5]
; GISEL-ASM-NEXT: flat_store_dword v[0:1], v2
; GISEL-ASM-NEXT: s_waitcnt vmcnt(0)
-; GISEL-ASM-NEXT: s_andn2_b64 exec, exec, s[6:7]
+; GISEL-ASM-NEXT: s_andn2_b64 exec, exec, s[4:5]
; GISEL-ASM-NEXT: s_cbranch_execnz .LBB11_3
; GISEL-ASM-NEXT: ; %bb.4: ; %end
-; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[6:7]
+; GISEL-ASM-NEXT: s_or_b64 exec, exec, s[4:5]
; GISEL-ASM-NEXT: s_waitcnt lgkmcnt(0)
; GISEL-ASM-NEXT: s_setpc_b64 s[30:31]
entry:
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
index b7f36c5dcd542..a0b00585e359c 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf-broken.mir
@@ -26,10 +26,10 @@ body: |
; GXN-NEXT: bb.1:
; GXN-NEXT: successors: %bb.2(0x80000000)
; GXN-NEXT: {{ $}}
- ; GXN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def $scc
+ ; GXN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def $scc
; GXN-NEXT: {{ $}}
; GXN-NEXT: bb.2:
- ; GXN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GXN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GXN-NEXT: [[DEF:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; GXN-NEXT: [[DEF1:%[0-9]+]]:vgpr_32 = IMPLICIT_DEF
; GXN-NEXT: DS_WRITE_B32 [[DEF]], [[DEF1]], 0, 0, implicit $m0, implicit $exec :: (store (s32), addrspace 3)
@@ -43,10 +43,10 @@ body: |
%2:sgpr_64 = COPY $exec
bb.1:
- $exec = S_OR_B64 $exec, %1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %1, implicit-def $scc
bb.2:
- $exec = S_OR_B64 $exec, %2, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def $scc
%5:vgpr_32 = IMPLICIT_DEF
%6:vgpr_32 = IMPLICIT_DEF
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll b/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
index 873d28b54337b..6fa995f4ff282 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf.ll
@@ -69,6 +69,7 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB0_4
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -76,14 +77,12 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: ; kill: def $sgpr0 killed $sgpr0 def $sgpr0_sgpr1
; GCN-O0-NEXT: s_mov_b32 s1, s2
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[0:1]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v0
; GCN-O0-NEXT: v_mov_b32_e32 v1, v0
; GCN-O0-NEXT: v_mov_b32_e32 v2, v3
@@ -102,6 +101,8 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB0_3
; GCN-O0-NEXT: ; %bb.2: ; %bb.inner.then
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -109,9 +110,7 @@ define amdgpu_kernel void @simple_nested_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -240,6 +239,7 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB1_3
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -247,14 +247,12 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: ; kill: def $sgpr0 killed $sgpr0 def $sgpr0_sgpr1
; GCN-O0-NEXT: s_mov_b32 s1, s2
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[0:1]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v0
; GCN-O0-NEXT: v_mov_b32_e32 v1, v0
; GCN-O0-NEXT: v_mov_b32_e32 v2, v3
@@ -273,6 +271,8 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB1_4
; GCN-O0-NEXT: ; %bb.2: ; %bb.inner.then
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -280,9 +280,7 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -316,9 +314,9 @@ define amdgpu_kernel void @uncollapsable_nested_if(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: v_readlane_b32 s2, v4, 4
; GCN-O0-NEXT: v_readlane_b32 s3, v4, 5
; GCN-O0-NEXT: s_or_b64 exec, exec, s[2:3]
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 2
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
@@ -456,17 +454,18 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB2_6
; GCN-O0-NEXT: ; %bb.1: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[6:7]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s0, 2
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_cmp_ne_u32_e64 s[0:1], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[2:3], exec
; GCN-O0-NEXT: s_and_b64 s[0:1], s[2:3], s[0:1]
; GCN-O0-NEXT: s_xor_b64 s[2:3], s[0:1], s[2:3]
+; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_writelane_b32 v4, s2, 4
; GCN-O0-NEXT: v_writelane_b32 v4, s3, 5
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
@@ -493,6 +492,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_xor_b64 exec, exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB2_5
; GCN-O0-NEXT: ; %bb.3: ; %bb.then
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -500,9 +500,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -518,6 +516,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: buffer_store_dword v0, v[1:2], s[0:3], 0 addr64
; GCN-O0-NEXT: s_branch .LBB2_5
; GCN-O0-NEXT: .LBB2_4: ; %bb.else
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v4, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -525,9 +524,7 @@ define amdgpu_kernel void @nested_if_if_else(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v4, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v4, 1
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: v_mov_b32_e32 v0, 2
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_add_i32_e64 v1, s[2:3], v1, v0
; GCN-O0-NEXT: v_ashrrev_i32_e64 v3, 31, v1
; GCN-O0-NEXT: ; kill: def $vgpr1 killed $vgpr1 def $vgpr1_vgpr2 killed $exec
@@ -725,13 +722,13 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: s_xor_b64 exec, exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB3_8
; GCN-O0-NEXT: ; %bb.2: ; %bb.outer.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v6, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[8:9]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s0, 0xf000
; GCN-O0-NEXT: s_mov_b32 s2, 0
; GCN-O0-NEXT: s_mov_b32 s4, s2
@@ -741,11 +738,12 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: ; kill: def $sgpr0_sgpr1 killed $sgpr0_sgpr1 def $sgpr0_sgpr1_sgpr2_sgpr3
; GCN-O0-NEXT: s_mov_b64 s[2:3], s[4:5]
; GCN-O0-NEXT: v_mov_b32_e32 v1, 1
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: buffer_store_dword v1, v[2:3], s[0:3], 0 addr64 offset:4
; GCN-O0-NEXT: s_mov_b32 s0, 2
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[2:3], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[0:1], exec
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_writelane_b32 v6, s0, 4
; GCN-O0-NEXT: v_writelane_b32 v6, s1, 5
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
@@ -771,13 +769,13 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: buffer_store_dword v0, v[1:2], s[0:3], 0 addr64 offset:8
; GCN-O0-NEXT: s_branch .LBB3_7
; GCN-O0-NEXT: .LBB3_4: ; %bb.outer.else
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v6, off, s[12:15], 0 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[8:9]
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s1, 0xf000
; GCN-O0-NEXT: s_mov_b32 s0, 0
; GCN-O0-NEXT: s_mov_b32 s2, s0
@@ -787,10 +785,11 @@ define amdgpu_kernel void @nested_if_else_if(ptr addrspace(1) nocapture %arg) {
; GCN-O0-NEXT: ; kill: def $sgpr4_sgpr5 killed $sgpr4_sgpr5 def $sgpr4_sgpr5_sgpr6_sgpr7
; GCN-O0-NEXT: s_mov_b64 s[6:7], s[2:3]
; GCN-O0-NEXT: v_mov_b32_e32 v1, 3
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: buffer_store_dword v1, v[2:3], s[4:7], 0 addr64 offset:12
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[2:3], v0, s0
; GCN-O0-NEXT: s_mov_b64 s[0:1], exec
+; GCN-O0-NEXT: s_waitcnt vmcnt(1)
; GCN-O0-NEXT: v_writelane_b32 v6, s0, 6
; GCN-O0-NEXT: v_writelane_b32 v6, s1, 7
; GCN-O0-NEXT: s_or_saveexec_b64 s[8:9], -1
@@ -930,6 +929,7 @@ define amdgpu_kernel void @s_endpgm_unsafe_barrier(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_mov_b64 exec, s[0:1]
; GCN-O0-NEXT: s_cbranch_execz .LBB4_2
; GCN-O0-NEXT: ; %bb.1: ; %bb.then
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[6:7], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v3, off, s[12:15], 0 ; 4-byte Folded Reload
@@ -937,14 +937,12 @@ define amdgpu_kernel void @s_endpgm_unsafe_barrier(ptr addrspace(1) nocapture %a
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s0, v3, 0
; GCN-O0-NEXT: v_readlane_b32 s1, v3, 1
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[12:15], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s2, 0xf000
; GCN-O0-NEXT: s_mov_b32 s4, 0
; GCN-O0-NEXT: ; kill: def $sgpr4 killed $sgpr4 def $sgpr4_sgpr5
; GCN-O0-NEXT: s_mov_b32 s5, s2
; GCN-O0-NEXT: ; kill: def $sgpr0_sgpr1 killed $sgpr0_sgpr1 def $sgpr0_sgpr1_sgpr2_sgpr3
; GCN-O0-NEXT: s_mov_b64 s[2:3], s[4:5]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_ashrrev_i32_e64 v2, 31, v0
; GCN-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GCN-O0-NEXT: v_mov_b32_e32 v1, v2
@@ -1071,6 +1069,8 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: .LBB5_1: ; %bb1
; GCN-O0-NEXT: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: s_waitcnt expcnt(1)
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
; GCN-O0-NEXT: s_waitcnt expcnt(0)
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
@@ -1082,9 +1082,7 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: v_readlane_b32 s7, v7, 1
; GCN-O0-NEXT: v_writelane_b32 v7, s6, 4
; GCN-O0-NEXT: v_writelane_b32 v7, s7, 5
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b32 s4, 0x207
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_cmp_lt_i32_e64 s[4:5], v0, s4
; GCN-O0-NEXT: s_or_b64 s[4:5], s[4:5], s[8:9]
; GCN-O0-NEXT: v_writelane_b32 v7, s4, 6
@@ -1280,14 +1278,14 @@ define void @scc_liveness(i32 %arg) local_unnamed_addr #0 {
; GCN-O0-NEXT: v_readlane_b32 s8, v7, 16
; GCN-O0-NEXT: v_readlane_b32 s9, v7, 17
; GCN-O0-NEXT: s_or_b64 exec, exec, s[8:9]
-; GCN-O0-NEXT: v_readlane_b32 s6, v7, 4
-; GCN-O0-NEXT: v_readlane_b32 s7, v7, 5
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 14
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 15
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
+; GCN-O0-NEXT: v_readlane_b32 s6, v7, 4
+; GCN-O0-NEXT: v_readlane_b32 s7, v7, 5
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 14
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 15
; GCN-O0-NEXT: s_and_b64 s[4:5], exec, s[4:5]
; GCN-O0-NEXT: s_or_b64 s[4:5], s[4:5], s[6:7]
; GCN-O0-NEXT: s_mov_b64 s[6:7], 0
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
index 5c62c7886020b..75224269c4b93 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf.mir
@@ -30,7 +30,7 @@ body: |
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: DBG_VALUE
; GCN-NEXT: S_ENDPGM 0
bb.0:
@@ -46,12 +46,12 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DBG_VALUE
bb.4:
DBG_VALUE
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -87,7 +87,7 @@ body: |
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
%0:sreg_64 = SI_IF undef %1:sreg_64, %bb.4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -100,12 +100,12 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -143,7 +143,7 @@ body: |
; GCN-NEXT: DBG_VALUE
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -158,13 +158,13 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
DBG_VALUE
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -204,7 +204,7 @@ body: |
; GCN-NEXT: KILL [[DEF]]
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -226,7 +226,7 @@ body: |
SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -268,7 +268,7 @@ body: |
; GCN-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY [[S_BREV_B32_]]
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -283,14 +283,14 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%4:sgpr_32 = IMPLICIT_DEF
%5:sgpr_32 = S_BREV_B32 %4
KILL %4
%6:sgpr_32 = COPY %5
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -325,11 +325,11 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
; GCN-NEXT: [[S_BREV_B64_:%[0-9]+]]:sreg_64 = S_BREV_B64 $exec
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -344,11 +344,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%4:sreg_64 = S_BREV_B64 $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -384,11 +384,11 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
; GCN-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY [[DEF]].sub2
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.4
@@ -404,11 +404,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:vgpr_32 = COPY %2.sub2
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -442,7 +442,7 @@ body: |
; GCN-NEXT: S_BRANCH %bb.5
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
@@ -462,11 +462,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.5:
@@ -495,7 +495,7 @@ body: |
; GCN-NEXT: bb.1:
; GCN-NEXT: successors: %bb.1(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[V_CMP_LT_U32_e64_]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[V_CMP_LT_U32_e64_]], implicit-def dead $scc
; GCN-NEXT: S_BRANCH %bb.1
bb.0:
successors: %bb.1
@@ -507,7 +507,7 @@ body: |
bb.1:
successors: %bb.1
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.1
...
@@ -538,7 +538,7 @@ body: |
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x40000000), %bb.6(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
; GCN-NEXT: [[S_AND_B64_1:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def dead $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_1]], implicit-def dead $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -557,10 +557,10 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY1]], implicit-def dead $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.2
@@ -586,10 +586,10 @@ body: |
bb.5:
successors: %bb.6
- SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.6:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -622,12 +622,12 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, %2, implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, %2, implicit-def dead $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
@@ -652,10 +652,10 @@ body: |
S_BRANCH %bb.6
bb.3:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
@@ -698,7 +698,7 @@ body: |
; GCN-NEXT: bb.4:
; GCN-NEXT: successors: %bb.5(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.5:
; GCN-NEXT: S_ENDPGM 0
@@ -715,11 +715,11 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.5:
S_ENDPGM 0
@@ -765,7 +765,7 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
; GCN-NEXT: successors: %bb.4(0x80000000)
@@ -784,7 +784,7 @@ body: |
bb.2:
bb.3:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
@@ -792,7 +792,7 @@ body: |
bb.5:
- SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.6:
S_BRANCH %bb.4
@@ -863,7 +863,7 @@ body: |
; GCN-NEXT: bb.7:
; GCN-NEXT: successors: %bb.8(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def dead $scc
; GCN-NEXT: S_BRANCH %bb.8
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.8:
@@ -896,14 +896,14 @@ body: |
; GCN-NEXT: bb.12:
; GCN-NEXT: successors: %bb.10(0x40000000), %bb.14(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
; GCN-NEXT: [[S_AND_B64_5:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def dead $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_5]], implicit-def dead $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.14, implicit $exec
; GCN-NEXT: S_BRANCH %bb.10
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.14:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1, %bb.14
@@ -945,19 +945,19 @@ body: |
bb.5:
successors: %bb.7
- SI_END_CF %11:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.7
bb.6:
successors: %bb.14
- SI_END_CF %5:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.14
bb.7:
successors: %bb.8
- SI_END_CF %8:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.8
bb.8:
@@ -992,12 +992,12 @@ body: |
bb.13:
successors: %bb.6
- SI_END_CF %15:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %15:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.6
bb.14:
- SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %2:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir b/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
index afa46f7877892..abc51f015fc12 100644
--- a/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
+++ b/llvm/test/CodeGen/AMDGPU/collapse-endcf2.mir
@@ -64,12 +64,12 @@ body: |
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY4]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY4]], implicit-def $scc
; GCN-NEXT: [[DEF:%[0-9]+]]:sreg_64 = IMPLICIT_DEF
; GCN-NEXT: dead [[SI_CALL:%[0-9]+]]:sreg_64 = SI_CALL [[DEF]], @func, csr_amdgpu
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GCN-NEXT: [[V_MOV_B32_e32_1:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 3, implicit $exec
; GCN-NEXT: [[V_MOV_B32_e32_2:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
; GCN-NEXT: $m0 = S_MOV_B32 -1
@@ -114,12 +114,12 @@ body: |
BUFFER_STORE_DWORD_ADDR64 %14, %8, %5, 0, 4, 0, 0, implicit $exec :: (store (s32), addrspace 1)
bb.3:
- $exec = S_OR_B64 $exec, %12, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %12, implicit-def $scc
%20:sreg_64 = IMPLICIT_DEF
%21:sreg_64 = SI_CALL %20, @func, csr_amdgpu
bb.4:
- $exec = S_OR_B64 $exec, %3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %3, implicit-def $scc
%17:vgpr_32 = V_MOV_B32_e32 3, implicit $exec
%18:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
$m0 = S_MOV_B32 -1
diff --git a/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir b/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
index 5a2b055a110f8..3e5453cb36728 100644
--- a/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
+++ b/llvm/test/CodeGen/AMDGPU/constant-fold-imm-immreg.mir
@@ -492,7 +492,7 @@ body: |
bb.3:
liveins: $vcc
- SI_END_CF %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit $vcc
...
diff --git a/llvm/test/CodeGen/AMDGPU/convergent.mir b/llvm/test/CodeGen/AMDGPU/convergent.mir
index 008d243e84309..4a3a8342cacf3 100644
--- a/llvm/test/CodeGen/AMDGPU/convergent.mir
+++ b/llvm/test/CodeGen/AMDGPU/convergent.mir
@@ -45,7 +45,7 @@ body: |
%5:vreg_256 = COPY %3.sub1:vreg_256
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -81,7 +81,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -118,7 +118,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -153,7 +153,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -190,7 +190,7 @@ body: |
bb.1:
%vcopy:vgpr_32 = COPY %vdst.sub0
bb.2:
- SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %sdst:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -233,7 +233,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -271,7 +271,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -309,7 +309,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -347,7 +347,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -385,7 +385,7 @@ body: |
%5:vgpr_32 = COPY %3:vgpr_32
bb.2:
- SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -421,7 +421,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -457,7 +457,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -493,7 +493,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -534,7 +534,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -570,7 +570,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -606,7 +606,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -642,7 +642,7 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
---
@@ -678,6 +678,6 @@ body: |
%4:vgpr_32 = COPY %2:vgpr_32
bb.2:
- SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3:sreg_32, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll b/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
index 4ffff7bd28ad4..c1dd3679e859c 100644
--- a/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
+++ b/llvm/test/CodeGen/AMDGPU/cross-block-use-is-not-abi-copy.ll
@@ -149,8 +149,8 @@ define { i32, half } @call_split_type_used_outside_block_struct() #0 {
; GCN-NEXT: s_addc_u32 s17, s17, func_struct at rel32@hi+12
; GCN-NEXT: s_swappc_b64 s[30:31], s[16:17]
; GCN-NEXT: v_readlane_b32 s30, v40, 0
-; GCN-NEXT: v_readlane_b32 s31, v40, 1
; GCN-NEXT: v_mov_b32_e32 v1, v4
+; GCN-NEXT: v_readlane_b32 s31, v40, 1
; GCN-NEXT: s_mov_b32 s32, s33
; GCN-NEXT: v_readlane_b32 s4, v40, 2
; GCN-NEXT: s_or_saveexec_b64 s[6:7], -1
diff --git a/llvm/test/CodeGen/AMDGPU/div_i128.ll b/llvm/test/CodeGen/AMDGPU/div_i128.ll
index 5ff48c8c20c3e..ff086c237c25a 100644
--- a/llvm/test/CodeGen/AMDGPU/div_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/div_i128.ll
@@ -636,12 +636,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB0_4
; GFX9-O0-NEXT: .LBB0_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
@@ -666,8 +660,13 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:276 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -694,7 +693,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -704,7 +702,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -713,11 +710,9 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -852,9 +847,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB0_6
; GFX9-O0-NEXT: s_branch .LBB0_1
; GFX9-O0-NEXT: .LBB0_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
@@ -871,9 +863,12 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -947,6 +942,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -978,9 +974,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:212 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB0_6
; GFX9-O0-NEXT: .LBB0_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
@@ -989,14 +982,17 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -1106,6 +1102,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -1739,12 +1736,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_branch .LBB0_3
; GFX9-G-O0-NEXT: .LBB0_5: ; %udiv-do-while
; GFX9-G-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-G-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-G-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-G-O0-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
@@ -1769,12 +1760,16 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:280 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-G-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-G-O0-NEXT: s_mov_b64 s[8:9], 1
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(18)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v2
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v3
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v15, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v4
; GFX9-G-O0-NEXT: s_mov_b32 s11, 1
@@ -1810,10 +1805,8 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_lshrrev_b32_e64 v15, v2, v3
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, v1
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v27
; GFX9-G-O0-NEXT: v_mov_b32_e32 v2, v28
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v23, v29
; GFX9-G-O0-NEXT: v_mov_b32_e32 v24, v30
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v1
@@ -1838,7 +1831,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:152 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:156 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:160 ; 4-byte Folded Spill
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_sub_co_u32_e64 v13, s[12:13], v13, v4
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v12, s[12:13], v12, v9, s[12:13]
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v10, s[12:13], v10, v7, s[12:13]
@@ -1933,9 +1925,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_cbranch_execnz .LBB0_5
; GFX9-G-O0-NEXT: s_branch .LBB0_1
; GFX9-G-O0-NEXT: .LBB0_6: ; %udiv-preheader
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
@@ -1953,12 +1942,15 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b32 s4, 64
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v22, v17
; GFX9-G-O0-NEXT: v_mov_b32_e32 v23, v16
; GFX9-G-O0-NEXT: ; kill: def $vgpr20 killed $vgpr20 def $vgpr20_vgpr21 killed $exec
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v4
; GFX9-G-O0-NEXT: s_mov_b32 s5, 0xffffffc0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s5
@@ -1970,7 +1962,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_cmp_lt_u32_e64 s[4:5], v18, v6
; GFX9-G-O0-NEXT: v_mov_b32_e32 v6, s6
; GFX9-G-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v18, v6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-G-O0-NEXT: v_lshrrev_b64 v[6:7], v18, v[20:21]
; GFX9-G-O0-NEXT: v_lshrrev_b64 v[25:26], v18, v[22:23]
; GFX9-G-O0-NEXT: v_lshlrev_b64 v[23:24], v5, v[20:21]
@@ -2019,6 +2011,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:276 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], s[8:9]
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], s[8:9]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s8, 8
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s9, 9
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -2050,9 +2043,6 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:224 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_branch .LBB0_5
; GFX9-G-O0-NEXT: .LBB0_7: ; %udiv-bb1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
@@ -2061,15 +2051,18 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
; GFX9-G-O0-NEXT: s_mov_b32 s7, 1
; GFX9-G-O0-NEXT: s_mov_b32 s6, 0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s7
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_add_co_u32_e64 v4, s[8:9], v2, v4
; GFX9-G-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:324 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: v_mov_b32_e32 v7, s6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v5, s[8:9], v5, v7, s[8:9]
; GFX9-G-O0-NEXT: v_mov_b32_e32 v7, s6
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v7, s[8:9], v6, v7, s[8:9]
@@ -2164,6 +2157,7 @@ define i128 @v_sdiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-G-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-G-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(17)
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-G-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -2738,12 +2732,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB1_4
; GFX9-O0-NEXT: .LBB1_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
@@ -2768,8 +2756,13 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -2796,7 +2789,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -2806,7 +2798,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -2815,11 +2806,9 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -2954,9 +2943,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB1_6
; GFX9-O0-NEXT: s_branch .LBB1_1
; GFX9-O0-NEXT: .LBB1_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
@@ -2973,9 +2959,12 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -3049,6 +3038,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:252 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -3080,9 +3070,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:188 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB1_6
; GFX9-O0-NEXT: .LBB1_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
@@ -3091,14 +3078,17 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -3208,6 +3198,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -3755,12 +3746,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_branch .LBB1_3
; GFX9-G-O0-NEXT: .LBB1_5: ; %udiv-do-while
; GFX9-G-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-G-O0-NEXT: v_readlane_b32 s6, v32, 8
-; GFX9-G-O0-NEXT: v_readlane_b32 s7, v32, 9
; GFX9-G-O0-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:180 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
@@ -3785,12 +3770,16 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:256 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: v_readlane_b32 s6, v32, 8
+; GFX9-G-O0-NEXT: v_readlane_b32 s7, v32, 9
; GFX9-G-O0-NEXT: s_mov_b64 s[8:9], 1
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(18)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v2
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v3
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v4
; GFX9-G-O0-NEXT: s_mov_b32 s11, 1
@@ -3826,10 +3815,8 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: v_lshrrev_b32_e64 v13, v2, v3
; GFX9-G-O0-NEXT: v_mov_b32_e32 v12, v0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, v1
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v1, v28
; GFX9-G-O0-NEXT: v_mov_b32_e32 v2, v29
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v30
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v31
; GFX9-G-O0-NEXT: v_mov_b32_e32 v0, v1
@@ -3854,7 +3841,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_sub_co_u32_e64 v11, s[12:13], v11, v4
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v10, s[12:13], v10, v9, s[12:13]
; GFX9-G-O0-NEXT: v_subb_co_u32_e64 v8, s[12:13], v8, v7, s[12:13]
@@ -3957,9 +3943,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_cbranch_execnz .LBB1_5
; GFX9-G-O0-NEXT: s_branch .LBB1_1
; GFX9-G-O0-NEXT: .LBB1_6: ; %udiv-preheader
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:260 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
@@ -3977,11 +3960,14 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b32 s4, 64
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v15, v5
; GFX9-G-O0-NEXT: v_mov_b32_e32 v14, v4
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-G-O0-NEXT: v_mov_b32_e32 v21, v7
; GFX9-G-O0-NEXT: v_mov_b32_e32 v20, v6
; GFX9-G-O0-NEXT: s_mov_b32 s5, 0xffffffc0
@@ -4048,6 +4034,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:244 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], s[8:9]
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], s[8:9]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s8, 8
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s9, 9
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
@@ -4079,9 +4066,6 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:192 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: s_branch .LBB1_5
; GFX9-G-O0-NEXT: .LBB1_7: ; %udiv-bb1
-; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
-; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:52 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
@@ -4090,15 +4074,18 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-G-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
+; GFX9-G-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-G-O0-NEXT: s_mov_b64 exec, s[18:19]
; GFX9-G-O0-NEXT: s_mov_b64 s[4:5], 0
; GFX9-G-O0-NEXT: s_mov_b32 s7, 1
; GFX9-G-O0-NEXT: s_mov_b32 s6, 0
; GFX9-G-O0-NEXT: v_mov_b32_e32 v4, s7
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(3)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-G-O0-NEXT: v_add_co_u32_e64 v4, s[8:9], v1, v4
; GFX9-G-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:292 ; 4-byte Folded Spill
; GFX9-G-O0-NEXT: v_mov_b32_e32 v5, s6
-; GFX9-G-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v5, s[8:9], v3, v5, s[8:9]
; GFX9-G-O0-NEXT: v_mov_b32_e32 v3, s6
; GFX9-G-O0-NEXT: v_addc_co_u32_e64 v7, s[8:9], v2, v3, s[8:9]
@@ -4193,6 +4180,7 @@ define i128 @v_udiv_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-G-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-G-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-G-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-G-O0-NEXT: s_waitcnt vmcnt(17)
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s6, 6
; GFX9-G-O0-NEXT: v_writelane_b32 v32, s7, 7
; GFX9-G-O0-NEXT: s_or_saveexec_b64 s[18:19], -1
diff --git a/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir b/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
index 41127424dd892..477e7233d9e01 100644
--- a/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
+++ b/llvm/test/CodeGen/AMDGPU/do-not-fold-copy.mir
@@ -48,7 +48,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%8:vgpr_32 = COPY %7
%9:sreg_64_xexec = IMPLICIT_DEF
%10:vgpr_32 = V_SET_INACTIVE_B32 0, %8, 0, 0, killed %9, implicit $exec
@@ -114,7 +114,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:vgpr_32 = V_ADD_U32_e64 %9, 1, 0, implicit $exec
$vgpr0 = COPY %11
SI_RETURN implicit $vgpr0
@@ -177,7 +177,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:vgpr_32 = COPY %9
%12:vgpr_32 = V_ADD_U32_e64 %11, 1, 0, implicit $exec
$vgpr0 = COPY %12
diff --git a/llvm/test/CodeGen/AMDGPU/dpp_combine.mir b/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
index fe9d6864c5d52..0d061c7d42dce 100644
--- a/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
+++ b/llvm/test/CodeGen/AMDGPU/dpp_combine.mir
@@ -435,7 +435,7 @@ body: |
successors: %bb.2
bb.2:
- SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
...
# GCN-LABEL: name: old_in_diff_bb
diff --git a/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir b/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
index 321414a81bdc8..5a62c9ce1cbdc 100644
--- a/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
+++ b/llvm/test/CodeGen/AMDGPU/dpp_combine_gfx11.mir
@@ -393,7 +393,7 @@ body: |
successors: %bb.2
bb.2:
- SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %8, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
...
# GCN-LABEL: name: old_in_diff_bb
diff --git a/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir b/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
index 3433534698dd4..49f59cb7ec41d 100644
--- a/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
+++ b/llvm/test/CodeGen/AMDGPU/extend-wwm-virt-reg-liveness.mir
@@ -176,7 +176,7 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: liveins: $sgpr6_sgpr7
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
; GCN-NEXT: SI_RETURN
bb.0:
liveins: $sgpr4, $sgpr10_sgpr11
@@ -208,7 +208,7 @@ body: |
S_BRANCH %bb.3
bb.5:
liveins: $sgpr6_sgpr7
- $exec = S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, $sgpr6_sgpr7, implicit-def $scc
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll b/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
index aabed8e2814c0..5c8ac8dcc14d4 100644
--- a/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
+++ b/llvm/test/CodeGen/AMDGPU/extract_vector_dynelt.ll
@@ -2963,12 +2963,6 @@ define float @float32_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB20_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[46:47], -1
-; GCN-O0-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[46:47]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v63, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v63, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
@@ -3002,7 +2996,12 @@ define float @float32_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:184 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:188 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:192 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[46:47], -1
+; GCN-O0-NEXT: buffer_load_dword v63, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[46:47]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v63, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v63, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3375,12 +3374,6 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB21_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
-; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v34, 35
-; GCN-O0-NEXT: v_readlane_b32 s5, v34, 36
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -3414,7 +3407,12 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
+; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v34, 35
+; GCN-O0-NEXT: v_readlane_b32 s5, v34, 36
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3547,12 +3545,6 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB21_4: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
-; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v34, 39
-; GCN-O0-NEXT: v_readlane_b32 s5, v34, 40
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:152 ; 4-byte Folded Reload
@@ -3586,7 +3578,12 @@ define double @double16_extelt_vec(i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:136 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[78:79], -1
+; GCN-O0-NEXT: buffer_load_dword v34, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[78:79]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v34, 39
+; GCN-O0-NEXT: v_readlane_b32 s5, v34, 40
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v32
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v32
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3687,17 +3684,16 @@ define i32 @extract_dyn_i32_3(<3 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB22_1: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 2
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 3
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v3
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v3
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3799,17 +3795,16 @@ define float @extract_dyn_float_3(<3 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB24_1: ; =>This Inner Loop Header: Depth=1
+; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
+; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 ; 4-byte Folded Reload
; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readlane_b32 s4, v4, 2
; GCN-O0-NEXT: v_readlane_b32 s5, v4, 3
-; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
-; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v3
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v3
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -3927,19 +3922,18 @@ define i32 @extract_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB26_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v5
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v5
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4061,19 +4055,18 @@ define float @extract_dyn_float_5(<5 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB28_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v6, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v6, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v5
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v5
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4209,12 +4202,6 @@ define i32 @extract_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB30_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4222,7 +4209,12 @@ define i32 @extract_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4356,12 +4348,6 @@ define float @extract_dyn_float_6(<6 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB32_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4369,7 +4355,12 @@ define float @extract_dyn_float_6(<6 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v7, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v7, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4519,12 +4510,6 @@ define i32 @extract_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB34_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4533,7 +4518,12 @@ define i32 @extract_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -4675,12 +4665,6 @@ define float @extract_dyn_float_7(<7 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: ; implicit-def: $vgpr0
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB36_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -4689,7 +4673,12 @@ define float @extract_dyn_float_7(<7 x float> inreg %arg, i32 %idx) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v8, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v8, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir b/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
index 6584abbb5716c..f1507dd420740 100644
--- a/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
+++ b/llvm/test/CodeGen/AMDGPU/fold-operands-s-add-copy-to-vgpr.mir
@@ -571,7 +571,7 @@ body: |
; CHECK-NEXT: SI_WATERFALL_LOOP %bb.1, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec = S_MOV_B64 [[S_MOV_B64_]]
+ ; CHECK-NEXT: $exec = bb-prolog S_MOV_B64 [[S_MOV_B64_]]
; CHECK-NEXT: SI_RETURN
bb.0:
liveins: $vgpr0_vgpr1
@@ -595,7 +595,7 @@ body: |
SI_WATERFALL_LOOP %bb.1, implicit $exec
bb.2:
- $exec = S_MOV_B64 %3
+ $exec = bb-prolog S_MOV_B64 %3
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir b/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
index 9297b44695c68..aa49187d4916e 100644
--- a/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/fold-restore-undef-use.mir
@@ -29,7 +29,7 @@ body: |
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.5(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr4_sgpr5 = S_OR_SAVEEXEC_B64 undef renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr4_sgpr5 = bb-prolog S_OR_SAVEEXEC_B64 undef renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: $exec = S_XOR_B64_term $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; CHECK-NEXT: {{ $}}
@@ -67,7 +67,7 @@ body: |
S_CBRANCH_EXECZ %bb.3, implicit $exec
bb.1:
- %1:sreg_64 = S_OR_SAVEEXEC_B64 undef %2:sreg_64, implicit-def $exec, implicit-def $scc, implicit $exec
+ %1:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 undef %2:sreg_64, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %1, implicit-def $scc
S_CBRANCH_EXECZ %bb.5, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir b/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
index 94c22b1aa8664..f73316636e7b2 100644
--- a/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
+++ b/llvm/test/CodeGen/AMDGPU/greedy-alloc-fail-sgpr1024-spill.mir
@@ -94,7 +94,7 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr4_sgpr5, $sgpr64_sgpr65:0x000000000000000F, $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99:0x0000000000000003
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 renamable $sgpr4_sgpr5, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: renamable $sgpr36 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr37 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr38 = COPY renamable $sgpr68
@@ -126,7 +126,7 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr64_sgpr65:0x0000000000000003
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: dead renamable $sgpr4 = S_LSHL_B32 killed renamable $sgpr64, 1, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.0, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.0, align 4, addrspace 5)
; CHECK-NEXT: dead [[COPY2:%[0-9]+]]:vreg_1024 = COPY killed renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
@@ -191,7 +191,7 @@ body: |
%12:vreg_1024 = V_INDIRECT_REG_WRITE_GPR_IDX_B32_V32 %12, 0, %13, 3, implicit-def $m0, implicit $m0, implicit $exec
bb.3:
- %14:sreg_64 = S_OR_SAVEEXEC_B64 %9, implicit-def $exec, implicit-def $scc, implicit $exec
+ %14:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %9, implicit-def $exec, implicit-def $scc, implicit $exec
undef %15.sub0:sgpr_1024 = COPY %7.sub0
%15.sub1:sgpr_1024 = COPY %7.sub0
%15.sub2:sgpr_1024 = COPY %7.sub0
@@ -220,7 +220,7 @@ body: |
S_BRANCH %bb.1
bb.4:
- $exec = S_OR_B64 $exec, %11, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %11, implicit-def $scc
%17:sreg_32 = S_LSHL_B32 %6.sub0, 1, implicit-def dead $scc
%16:vreg_1024 = COPY %7
diff --git a/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir b/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
index caa4c87eacc62..030dc20accdf8 100644
--- a/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/greedy-broken-ssa-verifier-error.mir
@@ -33,7 +33,7 @@ body: |
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.3:
- $exec_lo = S_OR_B32 $exec_lo, undef $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, undef $sgpr4, implicit-def $scc
$vgpr0 = COPY %0
S_SETPC_B64_return undef $sgpr30_sgpr31, implicit $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll b/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
index 127f4996b7676..e339bf976b5e7 100644
--- a/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
+++ b/llvm/test/CodeGen/AMDGPU/indirect-addressing-si.ll
@@ -1497,13 +1497,7 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB5_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 7
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -1514,14 +1508,26 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: buffer_load_dword v7, off, s[20:23], 0 offset:32 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[20:23], 0 offset:36 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[20:23], 0 offset:40 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[20:23], 0 offset:44 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[20:23], 0 offset:48 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[20:23], 0 offset:52 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[20:23], 0 offset:56 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[20:23], 0 offset:60 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 7
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -1546,6 +1552,7 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: v_readlane_b32 s1, v31, 5
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
+; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
@@ -1554,8 +1561,6 @@ define amdgpu_kernel void @extract_neg_offset_vgpr(ptr addrspace(1) %out) {
; NOOPT-NEXT: v_readlane_b32 s1, v31, 1
; NOOPT-NEXT: v_readlane_b32 s2, v31, 2
; NOOPT-NEXT: v_readlane_b32 s3, v31, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:72 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v0, off, s[0:3], 0
; NOOPT-NEXT: s_endpgm
;
@@ -3888,13 +3893,6 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_store_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB14_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -3920,7 +3918,12 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -3976,17 +3979,6 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: v_readlane_b32 s1, v31, 8
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
-; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
-; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
-; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
-; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:148 ; 4-byte Folded Reload
@@ -4003,22 +3995,29 @@ define amdgpu_kernel void @insert_neg_offset_vgpr(ptr addrspace(1) %in, ptr addr
; NOOPT-NEXT: buffer_load_dword v28, off, s[20:23], 0 offset:192 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v29, off, s[20:23], 0 offset:196 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[20:23], 0 offset:200 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
+; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
+; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
+; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
+; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: v_mov_b32_e32 v4, v18
; NOOPT-NEXT: v_mov_b32_e32 v5, v17
; NOOPT-NEXT: v_mov_b32_e32 v6, v16
; NOOPT-NEXT: v_mov_b32_e32 v0, v15
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v22
; NOOPT-NEXT: v_mov_b32_e32 v2, v21
; NOOPT-NEXT: v_mov_b32_e32 v3, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v12, v26
; NOOPT-NEXT: v_mov_b32_e32 v13, v25
; NOOPT-NEXT: v_mov_b32_e32 v14, v24
; NOOPT-NEXT: v_mov_b32_e32 v8, v23
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v9, v30
; NOOPT-NEXT: v_mov_b32_e32 v10, v29
; NOOPT-NEXT: v_mov_b32_e32 v11, v28
@@ -4377,13 +4376,6 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_store_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB15_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: buffer_load_dword v0, off, s[20:23], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[20:23], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[20:23], 0 offset:12 ; 4-byte Folded Reload
@@ -4409,7 +4401,12 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:68 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 9
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 10
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -4465,17 +4462,6 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: v_readlane_b32 s1, v31, 8
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
-; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
-; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
-; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
-; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
-; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
-; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: buffer_load_dword v15, off, s[20:23], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[20:23], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[20:23], 0 offset:148 ; 4-byte Folded Reload
@@ -4492,22 +4478,29 @@ define amdgpu_kernel void @insert_neg_inline_offset_vgpr(ptr addrspace(1) %in, p
; NOOPT-NEXT: buffer_load_dword v28, off, s[20:23], 0 offset:192 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v29, off, s[20:23], 0 offset:196 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[20:23], 0 offset:200 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[16:17], -1
+; NOOPT-NEXT: buffer_load_dword v31, off, s[20:23], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[16:17]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v31, 3
+; NOOPT-NEXT: v_readlane_b32 s1, v31, 4
+; NOOPT-NEXT: v_readlane_b32 s2, v31, 5
+; NOOPT-NEXT: v_readlane_b32 s3, v31, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v31, 0
+; NOOPT-NEXT: v_readlane_b32 s5, v31, 1
+; NOOPT-NEXT: v_readlane_b32 s6, v31, 2
; NOOPT-NEXT: v_mov_b32_e32 v4, v18
; NOOPT-NEXT: v_mov_b32_e32 v5, v17
; NOOPT-NEXT: v_mov_b32_e32 v6, v16
; NOOPT-NEXT: v_mov_b32_e32 v0, v15
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v22
; NOOPT-NEXT: v_mov_b32_e32 v2, v21
; NOOPT-NEXT: v_mov_b32_e32 v3, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v12, v26
; NOOPT-NEXT: v_mov_b32_e32 v13, v25
; NOOPT-NEXT: v_mov_b32_e32 v14, v24
; NOOPT-NEXT: v_mov_b32_e32 v8, v23
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v9, v30
; NOOPT-NEXT: v_mov_b32_e32 v10, v29
; NOOPT-NEXT: v_mov_b32_e32 v11, v28
@@ -4932,13 +4925,7 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB16_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v17, 23
-; NOOPT-NEXT: v_readlane_b32 s1, v17, 24
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:12 ; 4-byte Folded Reload
@@ -4949,14 +4936,26 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: buffer_load_dword v7, off, s[36:39], 0 offset:32 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[36:39], 0 offset:36 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[36:39], 0 offset:40 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[36:39], 0 offset:44 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[36:39], 0 offset:48 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[36:39], 0 offset:52 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[36:39], 0 offset:56 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[36:39], 0 offset:60 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[36:39], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[36:39], 0 offset:72 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v17, 23
+; NOOPT-NEXT: v_readlane_b32 s1, v17, 24
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5047,13 +5046,7 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: ; implicit-def: $vgpr0
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB16_4: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v17, 28
-; NOOPT-NEXT: v_readlane_b32 s1, v17, 29
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:92 ; 4-byte Folded Reload
@@ -5064,14 +5057,26 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: buffer_load_dword v7, off, s[36:39], 0 offset:112 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v8, off, s[36:39], 0 offset:116 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v9, off, s[36:39], 0 offset:120 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(6)
; NOOPT-NEXT: buffer_load_dword v10, off, s[36:39], 0 offset:124 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(5)
; NOOPT-NEXT: buffer_load_dword v11, off, s[36:39], 0 offset:128 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(4)
; NOOPT-NEXT: buffer_load_dword v12, off, s[36:39], 0 offset:132 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(3)
; NOOPT-NEXT: buffer_load_dword v13, off, s[36:39], 0 offset:136 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(2)
; NOOPT-NEXT: buffer_load_dword v14, off, s[36:39], 0 offset:140 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[36:39], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[36:39], 0 offset:68 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v17, 28
+; NOOPT-NEXT: v_readlane_b32 s1, v17, 29
; NOOPT-NEXT: v_readfirstlane_b32 s2, v16
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v16
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5096,6 +5101,9 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: v_readlane_b32 s1, v17, 27
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.6:
+; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:76 ; 4-byte Folded Reload
+; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:148 ; 4-byte Folded Reload
+; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:80 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[28:29], -1
; NOOPT-NEXT: buffer_load_dword v17, off, s[36:39], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[28:29]
@@ -5105,10 +5113,6 @@ define amdgpu_kernel void @extract_vgpr_offset_multiple_in_block(ptr addrspace(1
; NOOPT-NEXT: v_readlane_b32 s5, v17, 1
; NOOPT-NEXT: v_readlane_b32 s6, v17, 2
; NOOPT-NEXT: v_readlane_b32 s7, v17, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[36:39], 0 offset:76 ; 4-byte Folded Reload
-; NOOPT-NEXT: buffer_load_dword v1, off, s[36:39], 0 offset:148 ; 4-byte Folded Reload
-; NOOPT-NEXT: buffer_load_dword v2, off, s[36:39], 0 offset:80 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v2, off, s[4:7], 0
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v1, off, s[4:7], 0
@@ -5668,13 +5672,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_store_dword v15, off, s[28:31], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB17_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 11
-; NOOPT-NEXT: v_readlane_b32 s1, v32, 12
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:12 ; 4-byte Folded Reload
@@ -5700,7 +5697,12 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:80 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 11
+; NOOPT-NEXT: v_readlane_b32 s1, v32, 12
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5756,9 +5758,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: v_readlane_b32 s1, v32, 10
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:92 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:96 ; 4-byte Folded Reload
@@ -5775,10 +5774,13 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v13, off, s[28:31], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v14, off, s[28:31], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:148 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: v_mov_b32_e32 v16, 63
; NOOPT-NEXT: buffer_store_dword v16, off, s[28:31], 0 offset:216 ; 4-byte Folded Spill
; NOOPT-NEXT: s_mov_b64 s[0:1], exec
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
+; NOOPT-NEXT: s_waitcnt vmcnt(1)
; NOOPT-NEXT: v_writelane_b32 v32, s0, 13
; NOOPT-NEXT: v_writelane_b32 v32, s1, 14
; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
@@ -5787,32 +5789,21 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_store_dword v0, off, s[28:31], 0 offset:152 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v1, off, s[28:31], 0 offset:156 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v2, off, s[28:31], 0 offset:160 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v3, off, s[28:31], 0 offset:164 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v4, off, s[28:31], 0 offset:168 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v5, off, s[28:31], 0 offset:172 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v6, off, s[28:31], 0 offset:176 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v7, off, s[28:31], 0 offset:180 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v8, off, s[28:31], 0 offset:184 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v9, off, s[28:31], 0 offset:188 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v10, off, s[28:31], 0 offset:192 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v11, off, s[28:31], 0 offset:196 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v12, off, s[28:31], 0 offset:200 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v13, off, s[28:31], 0 offset:204 ; 4-byte Folded Spill
; NOOPT-NEXT: buffer_store_dword v14, off, s[28:31], 0 offset:208 ; 4-byte Folded Spill
-; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v15, off, s[28:31], 0 offset:212 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB17_4: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 15
-; NOOPT-NEXT: v_readlane_b32 s1, v32, 16
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:152 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[28:31], 0 offset:156 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[28:31], 0 offset:160 ; 4-byte Folded Reload
@@ -5838,7 +5829,12 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v15, off, s[28:31], 0 offset:212 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:216 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:76 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 15
+; NOOPT-NEXT: v_readlane_b32 s1, v32, 16
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -5894,19 +5890,6 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: v_readlane_b32 s1, v32, 14
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.6:
-; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
-; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v32, 6
-; NOOPT-NEXT: v_readlane_b32 s4, v32, 2
-; NOOPT-NEXT: v_readlane_b32 s5, v32, 3
-; NOOPT-NEXT: v_readlane_b32 s6, v32, 4
-; NOOPT-NEXT: v_readlane_b32 s7, v32, 5
-; NOOPT-NEXT: v_readlane_b32 s2, v32, 7
-; NOOPT-NEXT: v_readlane_b32 s3, v32, 8
-; NOOPT-NEXT: v_readlane_b32 s8, v32, 0
-; NOOPT-NEXT: v_readlane_b32 s9, v32, 1
; NOOPT-NEXT: buffer_load_dword v0, off, s[28:31], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[28:31], 0 offset:220 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[28:31], 0 offset:224 ; 4-byte Folded Reload
@@ -5924,22 +5907,31 @@ define amdgpu_kernel void @insert_vgpr_offset_multiple_in_block(ptr addrspace(1)
; NOOPT-NEXT: buffer_load_dword v29, off, s[28:31], 0 offset:272 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v30, off, s[28:31], 0 offset:276 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v31, off, s[28:31], 0 offset:280 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[26:27], -1
+; NOOPT-NEXT: buffer_load_dword v32, off, s[28:31], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[26:27]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v32, 6
+; NOOPT-NEXT: v_readlane_b32 s4, v32, 2
+; NOOPT-NEXT: v_readlane_b32 s5, v32, 3
+; NOOPT-NEXT: v_readlane_b32 s6, v32, 4
+; NOOPT-NEXT: v_readlane_b32 s7, v32, 5
+; NOOPT-NEXT: v_readlane_b32 s2, v32, 7
+; NOOPT-NEXT: v_readlane_b32 s3, v32, 8
+; NOOPT-NEXT: v_readlane_b32 s8, v32, 0
+; NOOPT-NEXT: v_readlane_b32 s9, v32, 1
; NOOPT-NEXT: v_mov_b32_e32 v5, v19
; NOOPT-NEXT: v_mov_b32_e32 v6, v18
; NOOPT-NEXT: v_mov_b32_e32 v7, v17
; NOOPT-NEXT: v_mov_b32_e32 v1, v16
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v2, v23
; NOOPT-NEXT: v_mov_b32_e32 v3, v22
; NOOPT-NEXT: v_mov_b32_e32 v4, v21
; NOOPT-NEXT: v_mov_b32_e32 v8, v20
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v13, v27
; NOOPT-NEXT: v_mov_b32_e32 v14, v26
; NOOPT-NEXT: v_mov_b32_e32 v15, v25
; NOOPT-NEXT: v_mov_b32_e32 v9, v24
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v10, v31
; NOOPT-NEXT: v_mov_b32_e32 v11, v30
; NOOPT-NEXT: v_mov_b32_e32 v12, v29
@@ -8786,17 +8778,15 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: .LBB26_1: ; %bb2
; NOOPT-NEXT: ; =>This Loop Header: Depth=1
; NOOPT-NEXT: ; Child Loop BB26_3 Depth 2
+; NOOPT-NEXT: s_waitcnt expcnt(0)
+; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: s_waitcnt expcnt(1)
; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_readlane_b32 s3, v18, 0
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 s[0:1], -1
; NOOPT-NEXT: ; implicit-def: $sgpr2
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_cmp_lt_i32_e64 s[4:5], v0, s3
; NOOPT-NEXT: s_xor_b64 s[4:5], s[4:5], s[0:1]
; NOOPT-NEXT: s_and_b64 vcc, exec, s[4:5]
@@ -8874,13 +8864,6 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB26_3: ; Parent Loop BB26_1 Depth=1
; NOOPT-NEXT: ; => This Inner Loop Header: Depth=2
-; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v18, 6
-; NOOPT-NEXT: v_readlane_b32 s1, v18, 7
; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:12 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[24:27], 0 offset:16 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[24:27], 0 offset:20 ; 4-byte Folded Reload
@@ -8906,7 +8889,12 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: buffer_load_dword v15, off, s[24:27], 0 offset:72 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[24:27], 0 offset:76 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[24:27], 0 offset:80 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
+; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v18, 6
+; NOOPT-NEXT: v_readlane_b32 s1, v18, 7
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -8962,9 +8950,6 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: v_readlane_b32 s1, v18, 5
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.5: ; in Loop: Header=BB26_1 Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
-; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:84 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[24:27], 0 offset:88 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[24:27], 0 offset:92 ; 4-byte Folded Reload
@@ -8981,9 +8966,13 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: buffer_load_dword v13, off, s[24:27], 0 offset:136 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v14, off, s[24:27], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v15, off, s[24:27], 0 offset:144 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
+; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: s_mov_b64 s[0:1], 0
; NOOPT-NEXT: s_waitcnt vmcnt(14)
; NOOPT-NEXT: buffer_store_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Spill
+; NOOPT-NEXT: s_waitcnt vmcnt(1)
; NOOPT-NEXT: v_writelane_b32 v18, s0, 2
; NOOPT-NEXT: v_writelane_b32 v18, s1, 3
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -8991,6 +8980,8 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: s_mov_b64 exec, s[20:21]
; NOOPT-NEXT: .LBB26_6: ; %Flow
; NOOPT-NEXT: ; in Loop: Header=BB26_1 Depth=1
+; NOOPT-NEXT: s_waitcnt expcnt(1)
+; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: s_or_saveexec_b64 s[20:21], -1
; NOOPT-NEXT: s_waitcnt expcnt(0)
; NOOPT-NEXT: buffer_load_dword v18, off, s[24:27], 0 ; 4-byte Folded Reload
@@ -8998,11 +8989,9 @@ define amdgpu_kernel void @broken_phi_bb(i32 %arg, i32 %arg1) {
; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_readlane_b32 s0, v18, 2
; NOOPT-NEXT: v_readlane_b32 s1, v18, 3
-; NOOPT-NEXT: buffer_load_dword v0, off, s[24:27], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: s_mov_b64 s[2:3], -1
; NOOPT-NEXT: s_xor_b64 s[0:1], s[0:1], s[2:3]
; NOOPT-NEXT: s_and_b64 vcc, exec, s[0:1]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: buffer_store_dword v0, off, s[24:27], 0 offset:4 ; 4-byte Folded Spill
; NOOPT-NEXT: s_cbranch_vccnz .LBB26_1
; NOOPT-NEXT: ; %bb.7: ; %bb8
@@ -9363,13 +9352,6 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_store_dword v15, off, s[16:19], 0 offset:64 ; 4-byte Folded Spill
; NOOPT-NEXT: ; implicit-def: $sgpr0_sgpr1
; NOOPT-NEXT: .LBB27_1: ; =>This Inner Loop Header: Depth=1
-; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
-; NOOPT-NEXT: s_waitcnt expcnt(0)
-; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v33, 11
-; NOOPT-NEXT: v_readlane_b32 s1, v33, 12
; NOOPT-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v1, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v2, off, s[16:19], 0 offset:12 ; 4-byte Folded Reload
@@ -9395,7 +9377,12 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_load_dword v15, off, s[16:19], 0 offset:64 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v16, off, s[16:19], 0 offset:144 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[16:19], 0 offset:132 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
+; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v33, 11
+; NOOPT-NEXT: v_readlane_b32 s1, v33, 12
; NOOPT-NEXT: v_readfirstlane_b32 s2, v17
; NOOPT-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v17
; NOOPT-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -9451,16 +9438,6 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: v_readlane_b32 s1, v33, 10
; NOOPT-NEXT: s_mov_b64 exec, s[0:1]
; NOOPT-NEXT: ; %bb.3:
-; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
-; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
-; NOOPT-NEXT: v_readlane_b32 s0, v33, 5
-; NOOPT-NEXT: v_readlane_b32 s1, v33, 6
-; NOOPT-NEXT: v_readlane_b32 s2, v33, 7
-; NOOPT-NEXT: v_readlane_b32 s3, v33, 8
-; NOOPT-NEXT: v_readlane_b32 s8, v33, 3
-; NOOPT-NEXT: v_readlane_b32 s9, v33, 4
; NOOPT-NEXT: buffer_load_dword v4, off, s[16:19], 0 offset:136 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v5, off, s[16:19], 0 offset:140 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v17, off, s[16:19], 0 offset:148 ; 4-byte Folded Reload
@@ -9479,22 +9456,28 @@ define amdgpu_cs void @insert_or_disj_index(ptr addrspace(1) %out, ptr addrspace
; NOOPT-NEXT: buffer_load_dword v30, off, s[16:19], 0 offset:200 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v31, off, s[16:19], 0 offset:204 ; 4-byte Folded Reload
; NOOPT-NEXT: buffer_load_dword v32, off, s[16:19], 0 offset:208 ; 4-byte Folded Reload
-; NOOPT-NEXT: s_waitcnt vmcnt(12)
+; NOOPT-NEXT: s_or_saveexec_b64 s[12:13], -1
+; NOOPT-NEXT: buffer_load_dword v33, off, s[16:19], 0 ; 4-byte Folded Reload
+; NOOPT-NEXT: s_mov_b64 exec, s[12:13]
+; NOOPT-NEXT: s_waitcnt vmcnt(0)
+; NOOPT-NEXT: v_readlane_b32 s0, v33, 5
+; NOOPT-NEXT: v_readlane_b32 s1, v33, 6
+; NOOPT-NEXT: v_readlane_b32 s2, v33, 7
+; NOOPT-NEXT: v_readlane_b32 s3, v33, 8
+; NOOPT-NEXT: v_readlane_b32 s8, v33, 3
+; NOOPT-NEXT: v_readlane_b32 s9, v33, 4
; NOOPT-NEXT: v_mov_b32_e32 v6, v20
; NOOPT-NEXT: v_mov_b32_e32 v7, v19
; NOOPT-NEXT: v_mov_b32_e32 v8, v18
; NOOPT-NEXT: v_mov_b32_e32 v0, v17
-; NOOPT-NEXT: s_waitcnt vmcnt(8)
; NOOPT-NEXT: v_mov_b32_e32 v1, v24
; NOOPT-NEXT: v_mov_b32_e32 v2, v23
; NOOPT-NEXT: v_mov_b32_e32 v3, v22
; NOOPT-NEXT: v_mov_b32_e32 v9, v21
-; NOOPT-NEXT: s_waitcnt vmcnt(4)
; NOOPT-NEXT: v_mov_b32_e32 v14, v28
; NOOPT-NEXT: v_mov_b32_e32 v15, v27
; NOOPT-NEXT: v_mov_b32_e32 v16, v26
; NOOPT-NEXT: v_mov_b32_e32 v10, v25
-; NOOPT-NEXT: s_waitcnt vmcnt(0)
; NOOPT-NEXT: v_mov_b32_e32 v11, v32
; NOOPT-NEXT: v_mov_b32_e32 v12, v31
; NOOPT-NEXT: v_mov_b32_e32 v13, v30
diff --git a/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir b/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
index 3081a2c1d1fe1..8773de86bf44b 100644
--- a/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
+++ b/llvm/test/CodeGen/AMDGPU/insert-skip-from-vcc.mir
@@ -167,7 +167,7 @@ body: |
...
---
# GCN-LABEL: name: and_execz_mov_exec_vccz
-# GCN: $exec = S_MOV_B64 -1
+# GCN: $exec = bb-prolog S_MOV_B64 -1
# GCN-NEXT: S_ENDPGM 0
name: and_execz_mov_exec_vccz
body: |
@@ -178,14 +178,14 @@ body: |
S_NOP 0
bb.2:
- $exec = S_MOV_B64 -1
+ $exec = bb-prolog S_MOV_B64 -1
$vcc = S_AND_B64 $exec, $exec, implicit-def dead $scc
S_CBRANCH_VCCZ %bb.1, implicit killed $vcc
S_ENDPGM 0
...
---
# GCN-LABEL: name: and_execz_mov_exec_vccnz
-# GCN: $exec = S_MOV_B64 -1
+# GCN: $exec = bb-prolog S_MOV_B64 -1
# GCN-NEXT: S_BRANCH %bb.1{{$}}
name: and_execz_mov_exec_vccnz
body: |
@@ -196,7 +196,7 @@ body: |
S_NOP 0
bb.2:
- $exec = S_MOV_B64 -1
+ $exec = bb-prolog S_MOV_B64 -1
$vcc = S_AND_B64 $exec, $exec, implicit-def dead $scc
S_CBRANCH_VCCNZ %bb.1, implicit killed $vcc
S_ENDPGM 0
@@ -626,7 +626,7 @@ body: |
S_NOP 0
bb.2:
- V_CMPX_EQ_U32_e32 0, killed $vgpr0, implicit-def $vcc, implicit-def $exec, implicit $exec
+ bb-prolog V_CMPX_EQ_U32_e32 0, killed $vgpr0, implicit-def $vcc, implicit-def $exec, implicit $exec
$vcc = S_AND_B64 $exec, $vcc, implicit-def dead $scc
S_CBRANCH_VCCZ %bb.1, implicit killed $vcc
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll b/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
index 02802330fe785..ac72c821fa4c7 100644
--- a/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
+++ b/llvm/test/CodeGen/AMDGPU/insert-waitcnts-merge.ll
@@ -64,7 +64,7 @@ define amdgpu_kernel void @widget(ptr addrspace(1) %arg, i1 %arg1) #0 {
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9, $vgpr0_vgpr1:0x000000000000000C, $vgpr2_vgpr3, $sgpr20_sgpr21_sgpr22_sgpr23
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr8_sgpr9, implicit-def dead $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr8_sgpr9, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr8_sgpr9 = S_AND_B64 $exec, renamable $sgpr6_sgpr7, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr4_sgpr5 = S_OR_B64 killed renamable $sgpr8_sgpr9, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; CHECK-NEXT: $exec = S_ANDN2_B64 $exec, renamable $sgpr4_sgpr5, implicit-def dead $scc
@@ -74,7 +74,7 @@ define amdgpu_kernel void @widget(ptr addrspace(1) %arg, i1 %arg1) #0 {
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5, $vgpr0_vgpr1:0x000000000000000C, $vgpr2_vgpr3, $sgpr20_sgpr21_sgpr22_sgpr23
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; CHECK-NEXT: renamable $vgpr4 = GLOBAL_LOAD_DWORD renamable $vgpr2_vgpr3, 0, 0, implicit $exec :: (load (s32) from `ptr addrspace(1) null`, addrspace 1)
; CHECK-NEXT: renamable $vgpr0 = BUFFER_LOAD_DWORD_OFFSET $sgpr20_sgpr21_sgpr22_sgpr23, 0, 0, 0, 0, implicit $exec :: (load (s32) from `ptr addrspace(5) null`, addrspace 5)
; CHECK-NEXT: S_WAITCNT .Vmcnt_1
diff --git a/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll b/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
index e756e485a75c8..9a1121f4fac4c 100644
--- a/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
+++ b/llvm/test/CodeGen/AMDGPU/insert_vector_dynelt.ll
@@ -8205,12 +8205,6 @@ define amdgpu_ps <32 x float> @float32_inselt_vec(<32 x float> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v31, off, s[8:11], 0 offset:128 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr0_sgpr1
; GCN-O0-NEXT: .LBB22_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[4:5], -1
-; GCN-O0-NEXT: buffer_load_dword v63, off, s[8:11], 0 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s0, v63, 2
-; GCN-O0-NEXT: v_readlane_b32 s1, v63, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[8:11], 0 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[8:11], 0 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[8:11], 0 offset:12 ; 4-byte Folded Reload
@@ -8245,7 +8239,12 @@ define amdgpu_ps <32 x float> @float32_inselt_vec(<32 x float> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v31, off, s[8:11], 0 offset:128 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v32, off, s[8:11], 0 offset:132 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v33, off, s[8:11], 0 offset:264 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[4:5], -1
+; GCN-O0-NEXT: buffer_load_dword v63, off, s[8:11], 0 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s0, v63, 2
+; GCN-O0-NEXT: v_readlane_b32 s1, v63, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s2, v33
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[0:1], s2, v33
; GCN-O0-NEXT: s_and_saveexec_b64 s[0:1], s[0:1]
@@ -8534,12 +8533,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB23_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 4
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 5
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -8558,7 +8551,12 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 4
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 5
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v17
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v17
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8613,12 +8611,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: v_readlane_b32 s5, v32, 3
; GCN-O0-NEXT: s_mov_b64 exec, s[4:5]
; GCN-O0-NEXT: ; %bb.3:
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 0
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 1
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:140 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:144 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:148 ; 4-byte Folded Reload
@@ -8636,8 +8628,13 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:200 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b32 s6, 1
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 0
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 1
+; GCN-O0-NEXT: s_mov_b32 s6, 1
; GCN-O0-NEXT: v_add_u32_e64 v16, s[6:7], v16, s6
; GCN-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:272 ; 4-byte Folded Spill
; GCN-O0-NEXT: s_mov_b32 s4, s5
@@ -8667,12 +8664,6 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_store_dword v15, off, s[0:3], s32 offset:264 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB23_4: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v32, 8
-; GCN-O0-NEXT: v_readlane_b32 s5, v32, 9
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:204 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:208 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:212 ; 4-byte Folded Reload
@@ -8691,7 +8682,12 @@ define <8 x double> @double8_inselt_vec(<8 x double> %vec, i32 %sel) {
; GCN-O0-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:264 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:268 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:272 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v32, 8
+; GCN-O0-NEXT: v_readlane_b32 s5, v32, 9
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v17
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v17
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8847,18 +8843,17 @@ define <3 x i32> @insert_dyn_i32_3(<3 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB24_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v4
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v4
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -8988,18 +8983,17 @@ define <3 x float> @insert_dyn_float_3(<3 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB26_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
-; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[10:11], -1
+; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[10:11]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v5, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v5, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v4
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v4
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9141,12 +9135,6 @@ define <5 x i32> @insert_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB28_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9154,7 +9142,12 @@ define <5 x i32> @insert_dyn_i32_5(<5 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9320,12 +9313,6 @@ define <5 x float> @insert_dyn_float_5(<5 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB30_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9333,7 +9320,12 @@ define <5 x float> @insert_dyn_float_5(<5 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v9, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v9, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v9, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v6
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v6
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9505,12 +9497,6 @@ define <6 x i32> @insert_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB32_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9519,7 +9505,12 @@ define <6 x i32> @insert_dyn_i32_6(<6 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9703,12 +9694,6 @@ define <6 x float> @insert_dyn_float_6(<6 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB34_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
-; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9717,7 +9702,12 @@ define <6 x float> @insert_dyn_float_6(<6 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[12:13], -1
+; GCN-O0-NEXT: buffer_load_dword v11, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[12:13]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v11, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v11, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v7
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v7
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -9907,12 +9897,6 @@ define <7 x i32> @insert_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB36_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -9922,7 +9906,12 @@ define <7 x i32> @insert_dyn_i32_7(<7 x i32> inreg %arg, i32 %idx, i32 %val) {
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v8
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v8
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -10124,12 +10113,6 @@ define <7 x float> @insert_dyn_float_7(<7 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_store_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Spill
; GCN-O0-NEXT: ; implicit-def: $sgpr4_sgpr5
; GCN-O0-NEXT: .LBB38_1: ; =>This Inner Loop Header: Depth=1
-; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
-; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
-; GCN-O0-NEXT: s_waitcnt vmcnt(0)
-; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
-; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
@@ -10139,7 +10122,12 @@ define <7 x float> @insert_dyn_float_7(<7 x float> inreg %arg, i32 %idx, float %
; GCN-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GCN-O0-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_or_saveexec_b64 s[14:15], -1
+; GCN-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; GCN-O0-NEXT: s_mov_b64 exec, s[14:15]
; GCN-O0-NEXT: s_waitcnt vmcnt(0)
+; GCN-O0-NEXT: v_readlane_b32 s4, v13, 2
+; GCN-O0-NEXT: v_readlane_b32 s5, v13, 3
; GCN-O0-NEXT: v_readfirstlane_b32 s6, v8
; GCN-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], s6, v8
; GCN-O0-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/licm-wwm.mir b/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
index 1ef1bc27095b5..9b19d1b937470 100644
--- a/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
+++ b/llvm/test/CodeGen/AMDGPU/licm-wwm.mir
@@ -13,7 +13,7 @@ body: |
; GCN: bb.0:
; GCN-NEXT: successors: %bb.1(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_32 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[V_MOV_B32_e32_:%[0-9]+]]:vgpr_32 = V_MOV_B32_e32 1, implicit $exec
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; GCN-NEXT: S_BRANCH %bb.1
@@ -21,7 +21,7 @@ body: |
; GCN-NEXT: bb.1:
; GCN-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[ENTER_STRICT_WWM1:%[0-9]+]]:sreg_32 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: [[ENTER_STRICT_WWM1:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: [[V_READFIRSTLANE_B32_:%[0-9]+]]:sreg_32_xm0 = V_READFIRSTLANE_B32 [[V_MOV_B32_e32_]], implicit $exec
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM [[ENTER_STRICT_WWM1]]
; GCN-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY [[V_READFIRSTLANE_B32_]]
diff --git a/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir b/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
index 0c9c5ad84c092..61fe8593ae7c8 100644
--- a/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
+++ b/llvm/test/CodeGen/AMDGPU/live-interval-bug-in-rename-independent-subregs.mir
@@ -34,7 +34,7 @@ body: |
; REG_ALLOC-NEXT: successors: %bb.3(0x40000000), %bb.5(0x40000000)
; REG_ALLOC-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr4_sgpr5, $sgpr6_sgpr7, $vgpr5_vgpr6_vgpr7_vgpr8:0x0000000000000003, $vgpr6_vgpr7_vgpr8_vgpr9:0x0000000000000003
; REG_ALLOC-NEXT: {{ $}}
- ; REG_ALLOC-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; REG_ALLOC-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
; REG_ALLOC-NEXT: $exec = S_XOR_B64_term $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; REG_ALLOC-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; REG_ALLOC-NEXT: S_BRANCH %bb.3
@@ -72,7 +72,7 @@ body: |
; REG_ALLOC-NEXT: successors: %bb.4(0x80000000)
; REG_ALLOC-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr2_sgpr3, $sgpr4_sgpr5
; REG_ALLOC-NEXT: {{ $}}
- ; REG_ALLOC-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; REG_ALLOC-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; REG_ALLOC-NEXT: S_BRANCH %bb.4
;
; DEAD_INST_DEL-LABEL: name: _amdgpu_cs_main
@@ -103,7 +103,7 @@ body: |
; DEAD_INST_DEL-NEXT: successors: %bb.3(0x40000000), %bb.5(0x40000000)
; DEAD_INST_DEL-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr4_sgpr5, $sgpr6_sgpr7, $vgpr5_vgpr6_vgpr7_vgpr8:0x0000000000000003, $vgpr6_vgpr7_vgpr8_vgpr9:0x0000000000000003
; DEAD_INST_DEL-NEXT: {{ $}}
- ; DEAD_INST_DEL-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; DEAD_INST_DEL-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
; DEAD_INST_DEL-NEXT: $exec = S_XOR_B64_term $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; DEAD_INST_DEL-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; DEAD_INST_DEL-NEXT: S_BRANCH %bb.3
@@ -141,7 +141,7 @@ body: |
; DEAD_INST_DEL-NEXT: successors: %bb.4(0x80000000)
; DEAD_INST_DEL-NEXT: liveins: $sgpr0, $vgpr0, $vgpr10, $sgpr2_sgpr3, $sgpr4_sgpr5
; DEAD_INST_DEL-NEXT: {{ $}}
- ; DEAD_INST_DEL-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; DEAD_INST_DEL-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; DEAD_INST_DEL-NEXT: S_BRANCH %bb.4
bb.0:
successors: %bb.2(0x40000000), %bb.1(0x40000000)
@@ -174,7 +174,7 @@ body: |
bb.1:
successors: %bb.3(0x40000000), %bb.5(0x40000000)
- %17:sreg_64 = S_OR_SAVEEXEC_B64 %15, implicit-def $exec, implicit-def $scc, implicit $exec
+ %17:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %15, implicit-def $exec, implicit-def $scc, implicit $exec
%18:sreg_64_xexec = S_AND_B64 $exec, %17, implicit-def $scc
$exec = S_XOR_B64_term $exec, %18, implicit-def $scc
S_CBRANCH_EXECZ %bb.5, implicit $exec
@@ -207,6 +207,6 @@ body: |
bb.5:
successors: %bb.4(0x80000000)
- $exec = S_OR_B64 $exec, %18, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %18, implicit-def $scc
S_BRANCH %bb.4
...
diff --git a/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir b/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
index a23fb912893b1..b9c0aa8883417 100644
--- a/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
+++ b/llvm/test/CodeGen/AMDGPU/loop-header-align-gfx950.mir
@@ -36,7 +36,7 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
S_SETPC_B64_return undef $sgpr30_sgpr31
...
@@ -69,6 +69,6 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
S_SETPC_B64_return undef $sgpr30_sgpr31
...
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
index 1a07124d88491..2da76e20647de 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-intervals.mir
@@ -24,7 +24,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %3, implicit-def dead $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %3, implicit-def dead $scc
; CHECK-NEXT: S_ENDPGM 0
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
@@ -33,7 +33,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: [[S_AND_B32_1:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def dead $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_1]], implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
@@ -53,7 +53,7 @@ body: |
S_BRANCH %bb.2
bb.1:
- SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.2:
@@ -100,7 +100,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_OR_B32_]], implicit-def dead $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_OR_B32_]], implicit-def dead $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1(0x80000000)
@@ -121,7 +121,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF killed %2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -158,7 +158,7 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY1]], implicit-def dead $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY1]], implicit-def dead $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.1(0x40000000), %bb.4(0x40000000)
@@ -183,10 +183,10 @@ body: |
bb.3:
successors: %bb.4(0x80000000)
- SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -233,7 +233,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.4(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY1]], implicit-def dead $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY1]], implicit-def dead $scc
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.6:
@@ -293,7 +293,7 @@ body: |
bb.3:
successors: %bb.5(0x80000000)
- SI_END_CF killed %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.5
bb.4:
@@ -302,7 +302,7 @@ body: |
bb.5:
successors: %bb.4(0x80000000)
- SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %0, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.4
bb.6:
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
index 67b971ed23f4f..8fa01b36a1dde 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-live-variables-update.mir
@@ -32,7 +32,7 @@ body: |
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.3(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_64_xexec = COPY killed [[S_MOV_B64_term]]
+ ; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_64_xexec = bb-prolog COPY killed [[S_MOV_B64_term]]
; CHECK-NEXT: $exec = S_OR_B64_term $exec, killed [[COPY4]], implicit-def dead $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -72,7 +72,7 @@ body: |
%4:sreg_64_xexec = PHI %5, %bb.2, %3, %bb.0
%6:vgpr_32 = PHI %7, %bb.2, %1, %bb.0
- SI_END_CF killed %4, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %4, implicit-def $exec, implicit-def dead $scc, implicit $exec
%8:vgpr_32 = nsw V_ADD_U32_e32 1, killed %6, implicit $exec
bb.2:
diff --git a/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir b/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
index c9be9238c2824..a36b3156fecea 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-control-flow-other-terminators.mir
@@ -110,7 +110,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $sgpr4_sgpr5
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 %2, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %2, implicit-def $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY killed $vgpr0
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sreg_64_xexec = COPY $sgpr4_sgpr5
; CHECK-NEXT: [[V_CMP_EQ_U32_e64_:%[0-9]+]]:sreg_64_xexec = V_CMP_EQ_U32_e64 0, [[COPY]], implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir b/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
index 87e1b4a4bb604..65d7fa32ebfd5 100644
--- a/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
+++ b/llvm/test/CodeGen/AMDGPU/lower-i1-copies-clear-kills.mir
@@ -120,7 +120,7 @@ body: |
%24:vgpr_32 = PHI %17, %bb.1, %21, %bb.2
%25:vreg_1 = PHI %7, %bb.1, %23, %bb.2
- SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %20, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%26:sreg_32 = S_MOV_B32 -1
%27:sreg_32 = IMPLICIT_DEF
%28:sreg_32 = COPY %25
@@ -143,7 +143,7 @@ body: |
%18:vgpr_32 = PHI %29, %bb.3, %24, %bb.4
%16:sreg_32 = PHI %27, %bb.3, %33, %bb.4
%36:vreg_1 = PHI %30, %bb.3, %35, %bb.4
- SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %31, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%37:sreg_32 = COPY %36
%14:sreg_32 = SI_IF_BREAK %37, %13, implicit-def dead $scc
SI_LOOP %14, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -151,7 +151,7 @@ body: |
bb.6:
%38:sreg_32 = PHI %14, %bb.5
- SI_END_CF %38, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %38, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
index 2abcb46977827..06778fba76f37 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-rematerialization-scoring.mir
@@ -95,7 +95,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -172,7 +172,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -260,7 +260,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -337,7 +337,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -416,7 +416,7 @@ body: |
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x7c000000), %bb.5(0x04000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
; CHECK-NEXT: %loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
; CHECK-NEXT: S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -501,7 +501,7 @@ body: |
bb.3:
successors: %bb.4(0x7c000000), %bb.5(0x04000000)
- $exec = S_OR_B64 $exec, %exec_save_if, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %exec_save_if, implicit-def $scc
%loop_counter:sreg_32 = S_ADD_I32 %loop_counter, -1, implicit-def dead $scc
S_CMP_LG_U32 %loop_counter, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.5, implicit killed $scc
diff --git a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
index 8c9e4a5a26c83..f02efdc9881a3 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-scheduler-sink-trivial-remats.mir
@@ -1080,7 +1080,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -1160,7 +1160,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -1237,7 +1237,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4186,7 +4186,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4267,7 +4267,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4344,7 +4344,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4433,7 +4433,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4520,7 +4520,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4603,7 +4603,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4698,7 +4698,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4791,7 +4791,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -4880,7 +4880,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -4981,7 +4981,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5080,7 +5080,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5175,7 +5175,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -5824,7 +5824,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -5965,7 +5965,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6102,7 +6102,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -6267,7 +6267,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6438,7 +6438,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -6605,7 +6605,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
@@ -6824,7 +6824,7 @@ body: |
; GFX908-NEXT: bb.3:
; GFX908-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-NEXT: {{ $}}
- ; GFX908-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -7061,7 +7061,7 @@ body: |
; GFX908-GCNTRACKERS-NEXT: bb.3:
; GFX908-GCNTRACKERS-NEXT: successors: %bb.5(0x04000000), %bb.4(0x7c000000)
; GFX908-GCNTRACKERS-NEXT: {{ $}}
- ; GFX908-GCNTRACKERS-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GFX908-GCNTRACKERS-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: undef [[S_MOV_B32_:%[0-9]+]].sub0:sreg_64 = S_ADD_I32 [[S_MOV_B32_]].sub0, -1, implicit-def dead $scc
; GFX908-GCNTRACKERS-NEXT: S_CMP_LG_U32 [[S_MOV_B32_]].sub0, 0, implicit-def $scc
; GFX908-GCNTRACKERS-NEXT: S_CBRANCH_SCC0 %bb.5, implicit killed $scc
@@ -7294,7 +7294,7 @@ body: |
bb.3:
successors: %bb.4(0x04000000), %bb.5(0x7c000000)
- $exec = S_OR_B64 $exec, %6, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %6, implicit-def $scc
%5.sub0:sreg_64 = S_ADD_I32 %5.sub0, -1, implicit-def dead $scc
S_CMP_LG_U32 %5.sub0, 0, implicit-def $scc
S_CBRANCH_SCC0 %bb.4, implicit killed $scc
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
index b1bcd89e06c28..fd6896d1da6f2 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-ignorable-exec-use.mir
@@ -80,7 +80,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %22, implicit %23
@@ -163,7 +163,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
%24:vgpr_32 = V_ADD_F32_e32 %14, %11, implicit $mode, implicit $exec
@@ -248,7 +248,7 @@ body: |
bb.2:
%22:vgpr_32 = PHI %3, %bb.0, %20, %bb.1
%23:vgpr_32 = PHI %4, %bb.0, %21, %bb.1
- SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %19, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %22, implicit %23
@@ -550,7 +550,7 @@ body: |
successors: %bb.4(0x40000000), %bb.6(0x40000000)
liveins: $vgpr0, $vgpr1, $vgpr2_vgpr3, $vcc
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_CBRANCH_EXECZ %bb.6, implicit $exec
bb.4:
@@ -703,7 +703,7 @@ body: |
successors: %bb.5(0x40000000), %bb.7(0x40000000)
liveins: $vgpr0, $vgpr1, $vgpr2_vgpr3, $vcc
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_CBRANCH_EXECZ %bb.7, implicit $exec
S_BRANCH %bb.5
@@ -795,7 +795,7 @@ body: |
successors: %bb.3(0x80000000)
%13:vgpr_32 = PHI %1, %bb.0, %12, %bb.1
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
S_ENDPGM 0, implicit %13
@@ -845,7 +845,7 @@ body: |
bb.2:
%7:vgpr_32 = PHI %1, %bb.0, %6, %bb.1
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %7
...
---
@@ -893,6 +893,6 @@ body: |
bb.2:
%7:vgpr_32 = PHI %1, %bb.0, %6, %bb.1
- SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %7
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
index 6b606fbf0847b..abb7f80f1e42f 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-lane-mask.mir
@@ -108,7 +108,7 @@ body: |
S_BRANCH %bb.4
bb.3:
- SI_END_CF %12, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %12, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
bb.4:
@@ -128,7 +128,7 @@ body: |
%10:sreg_32 = PHI %45, %bb.2, %46, %bb.4
%8:sreg_32 = PHI %39, %bb.2, %40, %bb.4
%9:vgpr_32 = PHI %36, %bb.2, %6, %bb.4
- SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%11:sreg_32 = SI_IF_BREAK %10, %2, implicit-def dead $scc
%12:sreg_32 = SI_IF_BREAK %8, %0, implicit-def dead $scc
SI_LOOP %11, %bb.2, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -138,7 +138,7 @@ body: |
successors: %bb.3(0x04000000), %bb.1(0x7c000000)
%13:vgpr_32 = PHI %9, %bb.5
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
SI_LOOP %12, %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.3
...
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
index 574bfaef981be..dde7fe76074be 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-loop-var-out-of-divergent-loop-swdev407790.mir
@@ -96,7 +96,7 @@ body: |
bb.5:
%7:vgpr_32 = PHI %0, %bb.4
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
INLINEASM &"", 1, implicit %5
S_BRANCH %bb.2
@@ -207,7 +207,7 @@ body: |
bb.5:
%7:vgpr_32 = PHI %0, %bb.4
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
INLINEASM &"", 1, implicit %5
S_BRANCH %bb.2
diff --git a/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir b/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
index c8afebf22e08d..70d6d2a3bd60d 100644
--- a/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
+++ b/llvm/test/CodeGen/AMDGPU/machine-sink-temporal-divergence-swdev407790.mir
@@ -32,7 +32,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -59,7 +59,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %9, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -95,7 +95,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -124,7 +124,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %11, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -158,7 +158,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: bb-prolog SI_END_CF [[SI_IF_BREAK]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -185,7 +185,7 @@ body: |
S_BRANCH %bb.2
bb.2:
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
FLAT_STORE_DWORD %3, %11, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
@@ -214,7 +214,7 @@ body: |
; CHECK-NEXT: SI_WATERFALL_LOOP %bb.1, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_MOV_B32 [[S_MOV_B32_]]
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_MOV_B32 [[S_MOV_B32_]]
; CHECK-NEXT: FLAT_STORE_DWORD [[COPY1]], [[V_ADD_U32_e64_]], 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
; CHECK-NEXT: SI_RETURN
bb.0:
@@ -236,7 +236,7 @@ body: |
SI_WATERFALL_LOOP %bb.1, implicit $exec
bb.2:
- $exec_lo = S_MOV_B32 %2
+ $exec_lo = bb-prolog S_MOV_B32 %2
FLAT_STORE_DWORD %1, %6, 0, 0, implicit $exec, implicit $flat_scr :: (store (s32))
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir b/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
index f171652bf7b55..3abf41009aa4d 100644
--- a/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
+++ b/llvm/test/CodeGen/AMDGPU/memory-legalizer-atomic-insert-end.mir
@@ -105,7 +105,7 @@ body: |
; CHECK-NEXT: bb.2.exit:
; CHECK-NEXT: liveins: $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: S_ENDPGM 0
bb.0 (%ir-block.0):
successors: %bb.1.atomic(0x40000000), %bb.2.exit(0x40000000)
@@ -140,7 +140,7 @@ body: |
bb.2.exit:
liveins: $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr2_sgpr3, implicit-def $scc
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/merge-m0.mir b/llvm/test/CodeGen/AMDGPU/merge-m0.mir
index a06ffafa7a046..11d87bd043749 100644
--- a/llvm/test/CodeGen/AMDGPU/merge-m0.mir
+++ b/llvm/test/CodeGen/AMDGPU/merge-m0.mir
@@ -204,7 +204,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
%0 = IMPLICIT_DEF
%1 = IMPLICIT_DEF
S_CBRANCH_VCCZ %bb.1, implicit undef $vcc
@@ -242,7 +242,7 @@ body: |
...
# GCN-LABEL: name: move-m0-with-prologue
-# GCN: $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+# GCN: $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
# GCN: $m0 = S_MOV_B32 -1
# GCN-NEXT: $vgpr0 = V_MOV_B32_e32 0, implicit $exec
# GCN-NEXT: DS_GWS_INIT $vgpr0, 0, implicit $m0, implicit $exec
@@ -252,7 +252,7 @@ body: |
bb.0:
liveins: $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$vgpr0 = V_MOV_B32_e32 0, implicit $exec
$m0 = S_MOV_B32 -1
DS_GWS_INIT $vgpr0, 0, implicit $m0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
index c97acc3d55de4..5f30a47e7dbf6 100644
--- a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
+++ b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands-non-ptr-intrinsics.ll
@@ -164,23 +164,23 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -190,6 +190,7 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v7, s8, 3
; W64-O0-NEXT: v_writelane_b32 v7, s9, 4
; W64-O0-NEXT: v_writelane_b32 v7, s10, 5
@@ -201,6 +202,7 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -212,15 +214,14 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s10, v7, 5
; W64-O0-NEXT: v_readlane_b32 s11, v7, 6
; W64-O0-NEXT: v_readlane_b32 s6, v7, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB0_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -228,7 +229,6 @@ define float @mubuf_vgpr(<4 x i32> %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s4, v7, 1
; W64-O0-NEXT: v_readlane_b32 s5, v7, 2
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_xor_saveexec_b64 s[4:5], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
@@ -534,23 +534,23 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -560,6 +560,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 3
; W64-O0-NEXT: v_writelane_b32 v17, s9, 4
; W64-O0-NEXT: v_writelane_b32 v17, s10, 5
@@ -571,6 +572,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB1_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -582,9 +584,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: v_readlane_b32 s10, v17, 5
; W64-O0-NEXT: v_readlane_b32 s11, v17, 6
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -605,23 +605,23 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_4: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -631,6 +631,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 11
; W64-O0-NEXT: v_writelane_b32 v17, s9, 12
; W64-O0-NEXT: v_writelane_b32 v17, s10, 13
@@ -642,6 +643,7 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.5: ; in Loop: Header=BB1_4 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -653,29 +655,26 @@ define void @mubuf_vgpr_adjacent_in_block(<4 x i32> %i, <4 x i32> %j, i32 %c, pt
; W64-O0-NEXT: v_readlane_b32 s10, v17, 13
; W64-O0-NEXT: v_readlane_b32 s11, v17, 14
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB1_4
; W64-O0-NEXT: ; %bb.6:
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
-; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
-; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
+; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
+; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: global_store_dword v[3:4], v5, off
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: global_store_dword v[0:1], v2, off
@@ -1041,23 +1040,23 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1067,6 +1066,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 4
; W64-O0-NEXT: v_writelane_b32 v13, s9, 5
; W64-O0-NEXT: v_writelane_b32 v13, s10, 6
@@ -1078,6 +1078,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB2_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1089,27 +1090,24 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s10, v13, 6
; W64-O0-NEXT: v_readlane_b32 s11, v13, 7
; W64-O0-NEXT: v_readlane_b32 s6, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
+; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
; W64-O0-NEXT: v_readlane_b32 s6, v13, 2
; W64-O0-NEXT: v_readlane_b32 s7, v13, 3
; W64-O0-NEXT: s_mov_b64 exec, s[6:7]
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
-; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b32 s5, 0x3ff
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_and_b32_e64 v1, v1, s5
; W64-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v1, s4
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
@@ -1139,23 +1137,23 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_5: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1165,6 +1163,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 15
; W64-O0-NEXT: v_writelane_b32 v13, s9, 16
; W64-O0-NEXT: v_writelane_b32 v13, s10, 17
@@ -1176,6 +1175,7 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.6: ; in Loop: Header=BB2_5 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1187,15 +1187,14 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s10, v13, 17
; W64-O0-NEXT: v_readlane_b32 s11, v13, 18
; W64-O0-NEXT: v_readlane_b32 s6, v13, 12
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_5
; W64-O0-NEXT: ; %bb.7:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1203,8 +1202,6 @@ define void @mubuf_vgpr_outside_entry(<4 x i32> %i, <4 x i32> %j, i32 %c, ptr ad
; W64-O0-NEXT: v_readlane_b32 s4, v13, 13
; W64-O0-NEXT: v_readlane_b32 s5, v13, 14
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; W64-O0-NEXT: .LBB2_8: ; %bb2
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
diff --git a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
index 831d876e280a2..ca2d277e2b9b5 100644
--- a/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
+++ b/llvm/test/CodeGen/AMDGPU/mubuf-legalize-operands.ll
@@ -170,23 +170,23 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB0_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -196,6 +196,7 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v7, s8, 3
; W64-O0-NEXT: v_writelane_b32 v7, s9, 4
; W64-O0-NEXT: v_writelane_b32 v7, s10, 5
@@ -207,6 +208,7 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: buffer_store_dword v7, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB0_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -218,15 +220,14 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s10, v7, 5
; W64-O0-NEXT: v_readlane_b32 s11, v7, 6
; W64-O0-NEXT: v_readlane_b32 s6, v7, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB0_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -234,7 +235,6 @@ define float @mubuf_vgpr(ptr addrspace(8) %i, i32 %c) #0 {
; W64-O0-NEXT: v_readlane_b32 s4, v7, 1
; W64-O0-NEXT: v_readlane_b32 s5, v7, 2
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: s_xor_saveexec_b64 s[4:5], -1
; W64-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
@@ -554,23 +554,23 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -580,6 +580,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 3
; W64-O0-NEXT: v_writelane_b32 v17, s9, 4
; W64-O0-NEXT: v_writelane_b32 v17, s10, 5
@@ -591,6 +592,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB1_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -602,9 +604,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: v_readlane_b32 s10, v17, 5
; W64-O0-NEXT: v_readlane_b32 s11, v17, 6
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
@@ -625,23 +625,23 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB1_4: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -651,6 +651,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v17, s8, 11
; W64-O0-NEXT: v_writelane_b32 v17, s9, 12
; W64-O0-NEXT: v_writelane_b32 v17, s10, 13
@@ -662,6 +663,7 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.5: ; in Loop: Header=BB1_4 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -673,29 +675,26 @@ define void @mubuf_vgpr_adjacent_in_block(ptr addrspace(8) %i, ptr addrspace(8)
; W64-O0-NEXT: v_readlane_b32 s10, v17, 13
; W64-O0-NEXT: v_readlane_b32 s11, v17, 14
; W64-O0-NEXT: v_readlane_b32 s6, v17, 0
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB1_4
; W64-O0-NEXT: ; %bb.6:
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
-; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
-; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:72 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:12 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v17, 9
+; W64-O0-NEXT: v_readlane_b32 s5, v17, 10
+; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: global_store_dword v[3:4], v5, off
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: global_store_dword v[0:1], v2, off
@@ -1072,23 +1071,23 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_1: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1098,6 +1097,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 4
; W64-O0-NEXT: v_writelane_b32 v13, s9, 5
; W64-O0-NEXT: v_writelane_b32 v13, s10, 6
@@ -1109,6 +1109,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.2: ; in Loop: Header=BB2_1 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1120,27 +1121,24 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s10, v13, 6
; W64-O0-NEXT: v_readlane_b32 s11, v13, 7
; W64-O0-NEXT: v_readlane_b32 s6, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_1
; W64-O0-NEXT: ; %bb.3:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
+; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
; W64-O0-NEXT: v_readlane_b32 s6, v13, 2
; W64-O0-NEXT: v_readlane_b32 s7, v13, 3
; W64-O0-NEXT: s_mov_b64 exec, s[6:7]
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 1
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b32 s5, 0x3ff
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_and_b32_e64 v1, v1, s5
; W64-O0-NEXT: v_cmp_eq_u32_e64 s[6:7], v1, s4
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -1154,16 +1152,15 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execz .LBB2_8
; W64-O0-NEXT: ; %bb.4: ; %bb1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: v_readlane_b32 s4, v13, 0
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:16 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:20 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: v_readlane_b32 s4, v13, 0
; W64-O0-NEXT: v_mov_b32_e32 v6, v4
; W64-O0-NEXT: v_mov_b32_e32 v0, v3
; W64-O0-NEXT: v_mov_b32_e32 v4, v2
@@ -1188,23 +1185,23 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: .LBB2_5: ; =>This Inner Loop Header: Depth=1
-; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
-; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; W64-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(3)
+; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
+; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
+; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
+; W64-O0-NEXT: s_waitcnt vmcnt(4)
; W64-O0-NEXT: v_readfirstlane_b32 s8, v0
-; W64-O0-NEXT: s_waitcnt vmcnt(2)
+; W64-O0-NEXT: s_waitcnt vmcnt(3)
; W64-O0-NEXT: v_readfirstlane_b32 s12, v1
; W64-O0-NEXT: s_mov_b32 s4, s8
; W64-O0-NEXT: s_mov_b32 s5, s12
; W64-O0-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
-; W64-O0-NEXT: s_waitcnt vmcnt(1)
+; W64-O0-NEXT: s_waitcnt vmcnt(2)
; W64-O0-NEXT: v_readfirstlane_b32 s7, v2
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
+; W64-O0-NEXT: s_waitcnt vmcnt(1)
; W64-O0-NEXT: v_readfirstlane_b32 s6, v3
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
@@ -1214,6 +1211,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: s_mov_b32 s9, s12
; W64-O0-NEXT: s_mov_b32 s10, s7
; W64-O0-NEXT: s_mov_b32 s11, s6
+; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: v_writelane_b32 v13, s8, 15
; W64-O0-NEXT: v_writelane_b32 v13, s9, 16
; W64-O0-NEXT: v_writelane_b32 v13, s10, 17
@@ -1225,6 +1223,7 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: buffer_store_dword v13, off, s[0:3], s32 ; 4-byte Folded Spill
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
; W64-O0-NEXT: ; %bb.6: ; in Loop: Header=BB2_5 Depth=1
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1236,15 +1235,14 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s10, v13, 17
; W64-O0-NEXT: v_readlane_b32 s11, v13, 18
; W64-O0-NEXT: v_readlane_b32 s6, v13, 12
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
-; W64-O0-NEXT: s_nop 2
+; W64-O0-NEXT: s_nop 4
; W64-O0-NEXT: buffer_load_format_x v0, v0, s[8:11], s6 idxen
; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; W64-O0-NEXT: s_xor_b64 exec, exec, s[4:5]
; W64-O0-NEXT: s_cbranch_execnz .LBB2_5
; W64-O0-NEXT: ; %bb.7:
+; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
; W64-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 ; 4-byte Folded Reload
; W64-O0-NEXT: s_mov_b64 exec, s[16:17]
@@ -1252,8 +1250,6 @@ define void @mubuf_vgpr_outside_entry(ptr addrspace(8) %i, ptr addrspace(8) %j,
; W64-O0-NEXT: v_readlane_b32 s4, v13, 13
; W64-O0-NEXT: v_readlane_b32 s5, v13, 14
; W64-O0-NEXT: s_mov_b64 exec, s[4:5]
-; W64-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
-; W64-O0-NEXT: s_waitcnt vmcnt(0)
; W64-O0-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; W64-O0-NEXT: .LBB2_8: ; %bb2
; W64-O0-NEXT: s_or_saveexec_b64 s[16:17], -1
diff --git a/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir b/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
index c2e24fcd6de06..177ad8ab6d308 100644
--- a/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
+++ b/llvm/test/CodeGen/AMDGPU/no-remat-indirect-mov.mir
@@ -66,7 +66,7 @@ body: |
; GFX9-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX9-NEXT: {{ $}}
; GFX9-NEXT: bb.2:
- ; GFX9-NEXT: $exec = S_MOV_B64 [[S_MOV_B64_]]
+ ; GFX9-NEXT: $exec = bb-prolog S_MOV_B64 [[S_MOV_B64_]]
; GFX9-NEXT: $sgpr30_sgpr31 = COPY [[COPY]]
; GFX9-NEXT: $vgpr0 = COPY [[V_MOV_B32_e32_]]
; GFX9-NEXT: S_SETPC_B64_return $sgpr30_sgpr31, implicit $vgpr0, implicit undef $vgpr1, implicit undef $vgpr2, implicit undef $vgpr3
@@ -109,7 +109,7 @@ body: |
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.2:
- $exec = S_MOV_B64 killed %20
+ $exec = bb-prolog S_MOV_B64 killed %20
$sgpr30_sgpr31 = COPY killed %17
$vgpr0 = COPY killed %24
S_SETPC_B64_return killed $sgpr30_sgpr31, implicit killed $vgpr0, implicit undef $vgpr1, implicit undef $vgpr2, implicit undef $vgpr3
diff --git a/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir b/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
index d5736fc9446c2..86481eb74a9c5 100644
--- a/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
+++ b/llvm/test/CodeGen/AMDGPU/opt-sgpr-to-vgpr-copy.mir
@@ -160,7 +160,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %29, killed %30, killed %28, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -310,7 +310,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %38, killed %39, killed %37, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
@@ -433,7 +433,7 @@ body: |
BUFFER_STORE_DWORD_ADDR64 killed %29, killed %30, killed %28, 0, 0, 0, 0, implicit $exec
bb.2.bb2:
- SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
index df83b74042ad2..e9e08ece38faf 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-alloc-failure.mir
@@ -283,11 +283,11 @@ body: |
S_BRANCH %bb.18
bb.18:
- $exec_lo = S_OR_B32 $exec_lo, %65, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %65, implicit-def $scc
S_ENDPGM 0
bb.19:
- $exec_lo = S_OR_B32 $exec_lo, %59, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %59, implicit-def $scc
S_BRANCH %bb.2
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
index 391f20b241c3b..12d1abdcc720f 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-def-after-use.mir
@@ -21,23 +21,23 @@ body: |
; GCN-NEXT: {{ $}}
; GCN-NEXT: [[S_AND_B32_:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[DEF]], implicit-def dead $scc
; GCN-NEXT: $vcc_lo = COPY [[S_AND_B32_]]
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.6, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.6, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.1
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x80000000)
; GCN-NEXT: {{ $}}
; GCN-NEXT: $vcc_lo = S_ANDN2_B32 $exec_lo, %8, implicit-def dead $scc
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.3, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.3, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.3
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.3:
; GCN-NEXT: successors: %bb.4(0x50000000), %bb.5(0x30000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ ; GCN-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
; GCN-NEXT: [[V_CMP_NE_U32_e64_:%[0-9]+]]:sreg_32_xm0_xexec = V_CMP_NE_U32_e64 1, %5, implicit $exec
; GCN-NEXT: $vcc_lo = S_AND_B32 $exec_lo, [[V_CMP_NE_U32_e64_]], implicit-def dead $scc
- ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.5, implicit $vcc
+ ; GCN-NEXT: S_CBRANCH_VCCNZ %bb.5, implicit $vcc_lo
; GCN-NEXT: S_BRANCH %bb.4
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.4:
@@ -81,7 +81,7 @@ body: |
bb.3:
successors: %bb.4(0x50000000), %bb.5(0x30000000)
- $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
%7:sreg_32_xm0_xexec = V_CMP_NE_U32_e64 1, %5, implicit $exec
$vcc_lo = S_AND_B32 $exec_lo, %7, implicit-def dead $scc
S_CBRANCH_VCCNZ %bb.5, implicit killed $vcc
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
index 2d0ec54c5940f..414587d3d34e0 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-mask-pre-ra-non-empty-but-used-interval.mir
@@ -136,7 +136,7 @@ body: |
S_BRANCH %bb.21
bb.20:
- $exec_lo = S_OR_B32 $exec_lo, %34, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %34, implicit-def $scc
S_BRANCH %bb.30
bb.21:
@@ -184,7 +184,7 @@ body: |
S_BRANCH %bb.31
bb.30:
- $exec_lo = S_OR_B32 $exec_lo, %32, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %32, implicit-def $scc
S_BRANCH %bb.10
bb.31:
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir b/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
index 1280f015358fe..b01ec7aa78a40 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-exec-masking-pre-ra.mir
@@ -32,8 +32,8 @@ body: |
; GCN-NEXT: bb.2:
; GCN-NEXT: successors: %bb.3(0x40000000), %bb.6(0x40000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
- ; GCN-NEXT: $exec = S_AND_B64 $exec, [[COPY]], implicit-def dead $scc
+ ; GCN-NEXT: [[S_OR_SAVEEXEC_B64_:%[0-9]+]]:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 [[S_XOR_B64_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: $exec = bb-prolog S_AND_B64 $exec, [[COPY]], implicit-def dead $scc
; GCN-NEXT: [[S_AND_B64_1:%[0-9]+]]:sreg_64 = S_AND_B64 $exec, [[S_OR_SAVEEXEC_B64_]], implicit-def $scc
; GCN-NEXT: $exec = S_XOR_B64_term $exec, [[S_AND_B64_1]], implicit-def $scc
; GCN-NEXT: S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -55,10 +55,10 @@ body: |
; GCN-NEXT: bb.5:
; GCN-NEXT: successors: %bb.6(0x80000000)
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[COPY2]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY2]], implicit-def $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.6:
- ; GCN-NEXT: $exec = S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[S_AND_B64_1]], implicit-def $scc
bb.0:
successors: %bb.1, %bb.2
liveins: $vgpr0
@@ -78,8 +78,8 @@ body: |
bb.2:
successors: %bb.3, %bb.6
- %6:sreg_64 = S_OR_SAVEEXEC_B64 %5, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_AND_B64 $exec, %0, implicit-def dead $scc
+ %6:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 %5, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_AND_B64 $exec, %0, implicit-def dead $scc
%7:sreg_64 = S_AND_B64 $exec, %6, implicit-def $scc
$exec = S_XOR_B64_term $exec, %7, implicit-def $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -98,10 +98,10 @@ body: |
bb.4:
bb.5:
- $exec = S_OR_B64 $exec, %9, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %9, implicit-def $scc
bb.6:
- $exec = S_OR_B64 $exec, %7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %7, implicit-def $scc
...
diff --git a/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir b/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
index f7e9e248c7698..499f457a570ea 100644
--- a/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
+++ b/llvm/test/CodeGen/AMDGPU/optimize-if-exec-masking.mir
@@ -156,7 +156,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -193,7 +193,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -230,7 +230,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -271,7 +271,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -309,7 +309,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1, $sgpr4_sgpr5_sgpr6_sgpr7
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr4_sgpr5_sgpr6_sgpr7, 0, 0, 0, 0, implicit $exec
@@ -351,7 +351,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr0 = S_MOV_B32 0
$sgpr1 = S_MOV_B32 1
$sgpr2 = S_MOV_B32 -1
@@ -392,7 +392,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -431,7 +431,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -468,7 +468,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -505,7 +505,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
@@ -544,7 +544,7 @@ body: |
bb.2.end:
liveins: $vgpr0, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
$sgpr3 = S_MOV_B32 61440
$sgpr2 = S_MOV_B32 -1
BUFFER_STORE_DWORD_OFFSET killed $vgpr0, $sgpr0_sgpr1_sgpr2_sgpr3, 0, 0, 0, 0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll b/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
index 21d723018c729..250803fda0039 100644
--- a/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
+++ b/llvm/test/CodeGen/AMDGPU/partial-sgpr-to-vgpr-spills.ll
@@ -231,12 +231,15 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: s_cbranch_vccnz .LBB0_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v0, off, s[92:95], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v2, off, s[92:95], 0 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v1, off, s[92:95], 0 offset:8 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
-; GCN-NEXT: s_waitcnt vmcnt(1)
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v0, off, s[92:95], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s8, v0, 56
; GCN-NEXT: v_readlane_b32 s9, v0, 57
; GCN-NEXT: v_readlane_b32 s10, v0, 58
@@ -301,7 +304,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: v_readlane_b32 s73, v0, 5
; GCN-NEXT: v_readlane_b32 s74, v0, 6
; GCN-NEXT: v_readlane_b32 s75, v0, 7
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s76, v1, 56
; GCN-NEXT: v_readlane_b32 s77, v1, 57
; GCN-NEXT: v_readlane_b32 s78, v1, 58
@@ -326,9 +328,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: v_readlane_b32 s5, v1, 5
; GCN-NEXT: v_readlane_b32 s6, v1, 6
; GCN-NEXT: v_readlane_b32 s7, v1, 7
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v2, off, s[92:95], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[0:7]
; GCN-NEXT: ;;#ASMEND
@@ -387,7 +386,6 @@ define amdgpu_kernel void @spill_sgprs_to_multiple_vgprs(ptr addrspace(1) %out,
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[0:7]
; GCN-NEXT: ;;#ASMEND
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v2, 0
; GCN-NEXT: v_readlane_b32 s1, v2, 1
; GCN-NEXT: v_readlane_b32 s2, v2, 2
@@ -601,12 +599,12 @@ define amdgpu_kernel void @split_sgpr_spill_2_vgprs(ptr addrspace(1) %out, i32 %
; GCN-NEXT: s_cbranch_vccnz .LBB1_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[28:29], -1
-; GCN-NEXT: buffer_load_dword v0, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v1, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[28:29]
; GCN-NEXT: s_or_saveexec_b64 s[28:29], -1
-; GCN-NEXT: buffer_load_dword v1, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
+; GCN-NEXT: buffer_load_dword v0, off, s[52:55], 0 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[28:29]
-; GCN-NEXT: s_waitcnt vmcnt(1)
+; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s16, v0, 8
; GCN-NEXT: v_readlane_b32 s17, v0, 9
; GCN-NEXT: v_readlane_b32 s20, v0, 0
@@ -617,7 +615,6 @@ define amdgpu_kernel void @split_sgpr_spill_2_vgprs(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s25, v0, 5
; GCN-NEXT: v_readlane_b32 s26, v0, 6
; GCN-NEXT: v_readlane_b32 s27, v0, 7
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s36, v1, 32
; GCN-NEXT: v_readlane_b32 s37, v1, 33
; GCN-NEXT: v_readlane_b32 s38, v1, 34
@@ -853,6 +850,9 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: s_cbranch_vccnz .LBB2_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v31, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -904,9 +904,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s29, v31, 13
; GCN-NEXT: v_readlane_b32 s30, v31, 14
; GCN-NEXT: v_readlane_b32 s31, v31, 15
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[16:31]
; GCN-NEXT: ;;#ASMEND
@@ -929,7 +926,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill(ptr addrspace(1) %out, i32 %
; GCN-NEXT: v_readlane_b32 s17, v31, 61
; GCN-NEXT: v_readlane_b32 s18, v31, 62
; GCN-NEXT: v_readlane_b32 s19, v31, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v32, 0
; GCN-NEXT: v_readlane_b32 s1, v32, 1
; GCN-NEXT: ;;#ASMSTART
@@ -1095,6 +1091,9 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: s_cbranch_vccnz .LBB3_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
+; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[34:35]
+; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
; GCN-NEXT: buffer_load_dword v31, off, s[52:55], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -1146,9 +1145,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: v_readlane_b32 s29, v31, 13
; GCN-NEXT: v_readlane_b32 s30, v31, 14
; GCN-NEXT: v_readlane_b32 s31, v31, 15
-; GCN-NEXT: s_or_saveexec_b64 s[34:35], -1
-; GCN-NEXT: buffer_load_dword v32, off, s[52:55], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[34:35]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; def v0
; GCN-NEXT: ;;#ASMEND
@@ -1174,7 +1170,6 @@ define amdgpu_kernel void @no_vgprs_last_sgpr_spill_live_v0(i32 %in) #1 {
; GCN-NEXT: v_readlane_b32 s17, v31, 61
; GCN-NEXT: v_readlane_b32 s18, v31, 62
; GCN-NEXT: v_readlane_b32 s19, v31, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s0, v32, 0
; GCN-NEXT: v_readlane_b32 s1, v32, 1
; GCN-NEXT: ;;#ASMSTART
diff --git a/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir b/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
index c2b9f34ea78b0..dde94836c5e95 100644
--- a/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
+++ b/llvm/test/CodeGen/AMDGPU/phi-elimination-end-cf.mir
@@ -8,8 +8,8 @@
# CHECK: [[IF_INPUT_REG:%[0-9]+]]:sreg_64 = S_MOV_B64_term killed [[IF_SOURCE0]], implicit $exec
# CHECK: bb.1:
-# CHECK: [[END_CF_ARG:%[0-9]+]]:sreg_64 = COPY killed [[IF_INPUT_REG]]
-# CHECK: SI_END_CF killed [[END_CF_ARG]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+# CHECK: [[END_CF_ARG:%[0-9]+]]:sreg_64 = bb-prolog COPY killed [[IF_INPUT_REG]]
+# CHECK: bb-prolog SI_END_CF killed [[END_CF_ARG]], implicit-def dead $exec, implicit-def dead $scc, implicit $exec
# CHECK: bb.2:
# CHECK: [[IF_SOURCE1:%[0-9]+]]:sreg_64 = SI_IF [[COND]], %bb.1, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
@@ -37,7 +37,7 @@ body: |
%24:sreg_64 = PHI %20, %bb.3, %22, %bb.0
%23:vgpr_32 = PHI %19, %bb.3, %18, %bb.0
- SI_END_CF %24, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %24, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%3:vgpr_32, dead %10:sreg_64 = nsw V_ADD_CO_U32_e64 1, %23, 0, implicit $exec
bb.3:
diff --git a/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir b/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
index ce2ede3fe215e..674b3a93f07c5 100644
--- a/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
+++ b/llvm/test/CodeGen/AMDGPU/postra-sink-update-dependency.mir
@@ -15,8 +15,6 @@ body: |
; CHECK-NEXT: liveins: $sgpr0, $sgpr3, $sgpr2
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vgpr1 = IMPLICIT_DEF
- ; CHECK-NEXT: renamable $sgpr4 = COPY $sgpr2
- ; CHECK-NEXT: renamable $sgpr2 = COPY $sgpr3
; CHECK-NEXT: $vgpr1 = SI_SPILL_S32_TO_VGPR $sgpr0, 0, $vgpr1
; CHECK-NEXT: $sgpr1 = S_AND_SAVEEXEC_B32 $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
@@ -27,8 +25,10 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.1(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr0, $sgpr2, $sgpr4, $vgpr1
+ ; CHECK-NEXT: liveins: $sgpr0, $sgpr2, $sgpr3, $vgpr1
; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: renamable $sgpr4 = COPY $sgpr2
+ ; CHECK-NEXT: renamable $sgpr2 = COPY $sgpr3
; CHECK-NEXT: $sgpr3 = SI_RESTORE_S32_FROM_VGPR $vgpr1, 0
; CHECK-NEXT: renamable $sgpr0_sgpr1 = S_GETPC_B64_pseudo
; CHECK-NEXT: renamable $sgpr5 = COPY $sgpr1
diff --git a/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir b/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
index 794d96dfa2365..a122e0abc6a6d 100644
--- a/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
+++ b/llvm/test/CodeGen/AMDGPU/ran-out-of-sgprs-allocation-failure.mir
@@ -41,10 +41,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $vgpr0, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9, $sgpr10_sgpr11
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: SI_SPILL_S32_SAVE $sgpr15, %stack.3, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.3, addrspace 5)
- ; CHECK-NEXT: SI_SPILL_S32_SAVE $sgpr14, %stack.4, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.4, addrspace 5)
; CHECK-NEXT: renamable $sgpr12_sgpr13 = V_CMP_GT_I32_e64 1, undef %18:vgpr_32, implicit $exec
- ; CHECK-NEXT: renamable $sgpr14_sgpr15 = V_CMP_EQ_U32_e64 0, undef %18:vgpr_32, implicit $exec
+ ; CHECK-NEXT: renamable $sgpr74_sgpr75 = V_CMP_EQ_U32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr72_sgpr73 = V_CMP_NE_U32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr18_sgpr19 = V_CMP_GT_I32_e64 0, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr52 = S_MOV_B32 0
@@ -52,12 +50,12 @@ body: |
; CHECK-NEXT: [[COPY:%[0-9]+]]:vreg_1024_align2 = COPY renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, implicit $exec
; CHECK-NEXT: renamable $sgpr22_sgpr23 = V_CMP_NE_U32_e64 1, undef %18:vgpr_32, implicit $exec
; CHECK-NEXT: renamable $sgpr53 = S_MOV_B32 1083786240
- ; CHECK-NEXT: SI_SPILL_S1024_SAVE renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.5, implicit $exec, implicit $sgpr32 :: (store (s1024) into %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: SI_SPILL_S1024_SAVE renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.4, implicit $exec, implicit $sgpr32 :: (store (s1024) into %stack.4, align 4, addrspace 5)
; CHECK-NEXT: S_BRANCH %bb.1
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.1:
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr22_sgpr23, implicit-def dead $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vreg_1024_align2 = COPY [[COPY]]
@@ -66,9 +64,9 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.5(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.5, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.4, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr68_sgpr69 = lr-split COPY killed renamable $sgpr52_sgpr53
; CHECK-NEXT: renamable $sgpr36 = COPY renamable $sgpr68
; CHECK-NEXT: renamable $sgpr37 = COPY renamable $sgpr68
@@ -108,28 +106,30 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
; CHECK-NEXT: successors: %bb.4(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr60 = lr-split COPY killed renamable $sgpr16
+ ; CHECK-NEXT: renamable $sgpr60 = lr-split COPY killed renamable $sgpr14
+ ; CHECK-NEXT: renamable $sgpr62 = lr-split COPY killed renamable $sgpr15
+ ; CHECK-NEXT: SI_SPILL_S32_SAVE killed renamable $sgpr16, %stack.3, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.3, addrspace 5)
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr4_sgpr5, 0, CustomRegMask($sgpr60,$sgpr62)
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
; CHECK-NEXT: successors: %bb.17(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr60
+ ; CHECK-NEXT: liveins: $sgpr60, $sgpr62
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
- ; CHECK-NEXT: $sgpr12 = SI_SPILL_S32_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.4, addrspace 5)
- ; CHECK-NEXT: $sgpr13 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
- ; CHECK-NEXT: $sgpr14 = COPY killed renamable $sgpr60
+ ; CHECK-NEXT: $sgpr12 = COPY killed renamable $sgpr60
+ ; CHECK-NEXT: $sgpr13 = COPY killed renamable $sgpr62
+ ; CHECK-NEXT: $sgpr14 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr4_sgpr5, 0, csr_amdgpu_noregs, implicit $sgpr12, implicit $sgpr13, implicit $sgpr14
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: S_BRANCH %bb.17
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.12(0x40000000), %bb.6(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $sgpr26_sgpr27 = S_AND_B64 renamable $sgpr72_sgpr73, undef renamable $sgpr64_sgpr65, implicit-def dead $scc
; CHECK-NEXT: renamable $sgpr24_sgpr25 = V_CMP_GT_I32_e64 0, undef %18:vgpr_32, implicit $exec
@@ -138,39 +138,39 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.6:
; CHECK-NEXT: successors: %bb.7(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: dead [[V_CNDMASK_B32_e64_:%[0-9]+]]:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, $sgpr18_sgpr19, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
; CHECK-NEXT: successors: %bb.8(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: renamable $sgpr34_sgpr35 = nofpexcept V_CMP_NLT_F64_e64 0, undef $sgpr4_sgpr5, 0, undef %29:vreg_64_align2, 0, implicit $mode, implicit $exec
; CHECK-NEXT: renamable $sgpr68_sgpr69 = nofpexcept V_CMP_NLT_F64_e64 0, 4607182418800017408, 0, undef %29:vreg_64_align2, 0, implicit $mode, implicit $exec
- ; CHECK-NEXT: dead [[V_INDIRECT_REG_READ_GPR_IDX_B32_V32_:%[0-9]+]]:vgpr_32 = V_INDIRECT_REG_READ_GPR_IDX_B32_V32 [[COPY1]], undef $sgpr4, 11, implicit-def $m0, implicit $m0, implicit $exec
+ ; CHECK-NEXT: dead [[V_INDIRECT_REG_READ_GPR_IDX_B32_V32_:%[0-9]+]]:vgpr_32 = V_INDIRECT_REG_READ_GPR_IDX_B32_V32 [[COPY1]], undef $sgpr14, 11, implicit-def $m0, implicit $m0, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.8:
; CHECK-NEXT: successors: %bb.10(0x40000000), %bb.9(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr34_sgpr35, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.10, implicit $vcc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.9:
; CHECK-NEXT: successors: %bb.10(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.5, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.5, align 4, addrspace 5)
+ ; CHECK-NEXT: renamable $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67 = SI_SPILL_S1024_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s1024) from %stack.4, align 4, addrspace 5)
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_64_align2 = COPY killed renamable $sgpr52_sgpr53, implicit $exec
; CHECK-NEXT: GLOBAL_STORE_DWORDX2_SADDR undef %18:vgpr_32, [[COPY2]], undef renamable $sgpr4_sgpr5, 0, 0, implicit $exec :: (store (s64), addrspace 1)
; CHECK-NEXT: [[V_CNDMASK_B32_e64_1:%[0-9]+]]:vgpr_32 = V_CNDMASK_B32_e64 0, 0, 0, 1, $sgpr12_sgpr13, implicit $exec
; CHECK-NEXT: dead renamable $sgpr40_sgpr41 = V_CMP_NE_U32_e64 1, [[V_CNDMASK_B32_e64_1]], implicit $exec
- ; CHECK-NEXT: renamable $sgpr70 = S_ADD_U32 renamable $sgpr8, 32, implicit-def dead $scc
+ ; CHECK-NEXT: renamable $sgpr82 = S_ADD_U32 renamable $sgpr8, 32, implicit-def dead $scc
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: SI_SPILL_S64_SAVE killed renamable $sgpr4_sgpr5, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s64) into %stack.0, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr36_sgpr37 = lr-split COPY killed renamable $sgpr12_sgpr13
- ; CHECK-NEXT: renamable $sgpr38_sgpr39 = lr-split COPY killed renamable $sgpr14_sgpr15
+ ; CHECK-NEXT: renamable $sgpr38_sgpr39 = lr-split COPY killed renamable $sgpr74_sgpr75
; CHECK-NEXT: renamable $sgpr48_sgpr49 = lr-split COPY killed renamable $sgpr72_sgpr73
; CHECK-NEXT: renamable $sgpr50_sgpr51 = lr-split COPY killed renamable $sgpr18_sgpr19
; CHECK-NEXT: renamable $sgpr52_sgpr53 = lr-split COPY killed renamable $sgpr20_sgpr21
@@ -181,22 +181,26 @@ body: |
; CHECK-NEXT: SI_SPILL_S64_SAVE killed renamable $sgpr10_sgpr11, %stack.2, implicit $exec, implicit $sgpr32 :: (store (s64) into %stack.2, align 4, addrspace 5)
; CHECK-NEXT: $sgpr10_sgpr11 = SI_SPILL_S64_RESTORE %stack.2, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.2, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr64_sgpr65 = lr-split COPY killed renamable $sgpr24_sgpr25
- ; CHECK-NEXT: $sgpr12 = SI_SPILL_S32_RESTORE %stack.4, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.4, addrspace 5)
- ; CHECK-NEXT: $sgpr13 = SI_SPILL_S32_RESTORE %stack.3, implicit $exec, implicit $sgpr32 :: (load (s32) from %stack.3, addrspace 5)
+ ; CHECK-NEXT: $sgpr12 = COPY renamable $sgpr14
+ ; CHECK-NEXT: $sgpr13 = COPY renamable $sgpr15
; CHECK-NEXT: renamable $sgpr66 = lr-split COPY killed renamable $sgpr8
; CHECK-NEXT: renamable $sgpr67 = lr-split COPY killed renamable $sgpr16
+ ; CHECK-NEXT: renamable $sgpr70 = lr-split COPY killed renamable $sgpr15
+ ; CHECK-NEXT: renamable $sgpr80 = lr-split COPY killed renamable $sgpr14
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr40_sgpr41, 0, csr_amdgpu_gfx90ainsts, implicit $sgpr4_sgpr5, implicit $sgpr6_sgpr7, implicit $sgpr10_sgpr11, implicit $sgpr12, implicit $sgpr13
; CHECK-NEXT: ADJCALLSTACKDOWN 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
; CHECK-NEXT: ADJCALLSTACKUP 0, 0, implicit-def dead $scc, implicit-def $sgpr32, implicit $sgpr32
- ; CHECK-NEXT: $sgpr8_sgpr9 = COPY renamable $sgpr70_sgpr71
+ ; CHECK-NEXT: $sgpr8_sgpr9 = COPY renamable $sgpr82_sgpr83
; CHECK-NEXT: dead $sgpr30_sgpr31 = SI_CALL undef renamable $sgpr40_sgpr41, 0, csr_amdgpu_gfx90ainsts, implicit $sgpr8_sgpr9
+ ; CHECK-NEXT: renamable $sgpr14 = lr-split COPY killed renamable $sgpr80
+ ; CHECK-NEXT: renamable $sgpr15 = lr-split COPY killed renamable $sgpr70
; CHECK-NEXT: renamable $sgpr16 = lr-split COPY killed renamable $sgpr67
; CHECK-NEXT: renamable $sgpr24_sgpr25 = lr-split COPY killed renamable $sgpr64_sgpr65
; CHECK-NEXT: renamable $sgpr22_sgpr23 = lr-split COPY killed renamable $sgpr54_sgpr55
; CHECK-NEXT: renamable $sgpr20_sgpr21 = lr-split COPY killed renamable $sgpr52_sgpr53
; CHECK-NEXT: renamable $sgpr18_sgpr19 = lr-split COPY killed renamable $sgpr50_sgpr51
; CHECK-NEXT: renamable $sgpr72_sgpr73 = lr-split COPY killed renamable $sgpr48_sgpr49
- ; CHECK-NEXT: renamable $sgpr14_sgpr15 = lr-split COPY killed renamable $sgpr38_sgpr39
+ ; CHECK-NEXT: renamable $sgpr74_sgpr75 = lr-split COPY killed renamable $sgpr38_sgpr39
; CHECK-NEXT: renamable $sgpr12_sgpr13 = lr-split COPY killed renamable $sgpr36_sgpr37
; CHECK-NEXT: renamable $sgpr4_sgpr5 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.0, align 4, addrspace 5)
; CHECK-NEXT: renamable $sgpr6_sgpr7 = SI_SPILL_S64_RESTORE %stack.1, implicit $exec, implicit $sgpr32 :: (load (s64) from %stack.1, align 4, addrspace 5)
@@ -209,28 +213,28 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.10:
; CHECK-NEXT: successors: %bb.8(0x40000000), %bb.12(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr34_sgpr35, $sgpr68_sgpr69, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.8, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.12
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.11:
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.1, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.17
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.12:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.13(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr24_sgpr25, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $exec = S_MOV_B64_term killed renamable $sgpr24_sgpr25
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.11, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.13:
; CHECK-NEXT: successors: %bb.15(0x40000000), %bb.14(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr20_sgpr21, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.15, implicit $vcc
@@ -238,18 +242,18 @@ body: |
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.14:
; CHECK-NEXT: successors: %bb.15(0x80000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.15:
; CHECK-NEXT: successors: %bb.11(0x40000000), %bb.16(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr14_sgpr15, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16, $sgpr4_sgpr5, $sgpr6_sgpr7, $sgpr8_sgpr9:0x0000000000000003, $sgpr10_sgpr11, $sgpr12_sgpr13, $sgpr18_sgpr19, $sgpr20_sgpr21, $sgpr22_sgpr23, $sgpr72_sgpr73, $sgpr74_sgpr75
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr14_sgpr15, implicit-def dead $scc
+ ; CHECK-NEXT: $vcc = S_AND_B64 $exec, renamable $sgpr74_sgpr75, implicit-def dead $scc
; CHECK-NEXT: S_CBRANCH_VCCNZ %bb.11, implicit $vcc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.16:
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.17(0x40000000)
- ; CHECK-NEXT: liveins: $sgpr16
+ ; CHECK-NEXT: liveins: $sgpr14, $sgpr15, $sgpr16
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; CHECK-NEXT: {{ $}}
diff --git a/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir b/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
index fe0dfd2eba37e..b831d2eac6165 100644
--- a/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
+++ b/llvm/test/CodeGen/AMDGPU/readlane_exec0.mir
@@ -29,5 +29,5 @@ body: |
bb.2:
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
...
diff --git a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
index 9b7411d98dc75..cc80d129acb46 100644
--- a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
+++ b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-interference.mir
@@ -4,12 +4,13 @@
# InlineSpiller tries to hoist the spill of %1 by storing its sibling %0 after
# the basic block prologue. Extending the PHI-defined %0 to that insertion point
# would overlap an interval already assigned to the same physical register.
-# Reject the hoist and spill %0 at the original copy instead.
+# Reject the hoist and spill %0 at the original copy instead. %3 is a COPY so it
+# is not rematerialized out of the prolog.
# CHECK-LABEL: name: hoist_spill_live_range_interference
# CHECK: bb.2:
# CHECK: SI_SPILL_WWM_V32_SAVE %0
-# CHECK-NEXT: $exec = S_AND_B64
+# CHECK-NEXT: $exec = bb-prolog S_AND_B64
---
name: hoist_spill_live_range_interference
@@ -35,10 +36,10 @@ body: |
%0:vgpr_32 = COPY %2
bb.2:
- %1:vgpr_32 = lr-split WWM_COPY %0:vgpr_32
- $exec = S_AND_B64 $exec, $exec, implicit-def $scc
- %3:vgpr_32 = IMPLICIT_DEF
- %4:vgpr_32 = IMPLICIT_DEF
+ %1:vgpr_32 = lr-split bb-prolog WWM_COPY %0:vgpr_32
+ $exec = bb-prolog S_AND_B64 $exec, $exec, implicit-def $scc
+ %3:vgpr_32 = bb-prolog COPY %2
+ %4:vgpr_32 = bb-prolog IMPLICIT_DEF
KILL %2:vgpr_32, %3:vgpr_32, %4:vgpr_32
KILL %1:vgpr_32, %2:vgpr_32
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
index 31b2e4820aca1..71b9241549175 100644
--- a/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
+++ b/llvm/test/CodeGen/AMDGPU/regalloc-hoist-spill-live-range-upd.mir
@@ -62,8 +62,8 @@ body: |
bb.2:
liveins: $sgpr0_sgpr1
- %1:vgpr_32 = lr-split WWM_COPY %0:vgpr_32
- $exec = S_AND_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ %1:vgpr_32 = lr-split bb-prolog WWM_COPY %0:vgpr_32
+ $exec = bb-prolog S_AND_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
KILL implicit-def $vgpr0, implicit-def $vgpr1, implicit-def $vgpr2, implicit-def $vgpr3, implicit-def $vgpr4, implicit-def $vgpr5, implicit-def $vgpr6, implicit-def $vgpr7, implicit-def $vgpr8, implicit-def $vgpr9, implicit-def $vgpr10, implicit-def $vgpr11, implicit-def $vgpr12, implicit-def $vgpr13, implicit-def $vgpr14, implicit-def $vgpr15, implicit-def $vgpr16, implicit-def $vgpr17, implicit-def $vgpr18, implicit-def $vgpr19, implicit-def $vgpr20, implicit-def $vgpr21, implicit-def $vgpr22, implicit-def $vgpr23, implicit-def $vgpr24, implicit-def $vgpr25, implicit-def $vgpr26, implicit-def $vgpr27, implicit-def $vgpr28, implicit-def $vgpr29, implicit-def $vgpr30, implicit-def $vgpr31, implicit-def $vgpr32, implicit-def $vgpr33, implicit-def $vgpr34, implicit-def $vgpr35, implicit-def $vgpr36, implicit-def $vgpr37, implicit-def $vgpr38, implicit-def $vgpr39, implicit-def $vgpr40, implicit-def $vgpr41, implicit-def $vgpr42, implicit-def $vgpr43, implicit-def $vgpr44, implicit-def $vgpr45, implicit-def $vgpr46, implicit-def $vgpr47, implicit-def $vgpr48, implicit-def $vgpr49, implicit-def $vgpr50, implicit-def $vgpr51, implicit-def $vgpr52, implicit-def $vgpr53, implicit-def $vgpr54, implicit-def $vgpr55, implicit-def $vgpr56, implicit-def $vgpr57, implicit-def $vgpr58, implicit-def $vgpr59, implicit-def $vgpr60, implicit-def $vgpr61, implicit-def $vgpr62, implicit-def $vgpr63, implicit-def $vgpr64, implicit-def $vgpr65, implicit-def $vgpr66, implicit-def $vgpr67, implicit-def $vgpr68, implicit-def $vgpr69, implicit-def $vgpr70, implicit-def $vgpr71, implicit-def $vgpr72, implicit-def $vgpr73, implicit-def $vgpr74, implicit-def $vgpr75, implicit-def $vgpr76, implicit-def $vgpr77, implicit-def $vgpr78, implicit-def $vgpr79, implicit-def $vgpr80, implicit-def $vgpr81, implicit-def $vgpr82, implicit-def $vgpr83, implicit-def $vgpr84, implicit-def $vgpr85, implicit-def $vgpr86, implicit-def $vgpr87, implicit-def $vgpr88, implicit-def $vgpr89, implicit-def $vgpr90, implicit-def $vgpr91, implicit-def $vgpr92, implicit-def $vgpr93, implicit-def $vgpr94, implicit-def $vgpr95, implicit-def $vgpr96, implicit-def $vgpr97, implicit-def $vgpr98, implicit-def $vgpr99, implicit-def $vgpr100, implicit-def $vgpr101, implicit-def $vgpr102, implicit-def $vgpr103, implicit-def $vgpr104, implicit-def $vgpr105, implicit-def $vgpr106, implicit-def $vgpr107, implicit-def $vgpr108, implicit-def $vgpr109, implicit-def $vgpr110, implicit-def $vgpr111, implicit-def $vgpr112, implicit-def $vgpr113, implicit-def $vgpr114, implicit-def $vgpr115, implicit-def $vgpr116, implicit-def $vgpr117, implicit-def $vgpr118, implicit-def $vgpr119, implicit-def $vgpr120, implicit-def $vgpr121, implicit-def $vgpr122, implicit-def $vgpr123, implicit-def $vgpr124, implicit-def $vgpr125, implicit-def $vgpr126, implicit-def $vgpr127, implicit-def $vgpr128, implicit-def $vgpr129, implicit-def $vgpr130, implicit-def $vgpr131, implicit-def $vgpr132, implicit-def $vgpr133, implicit-def $vgpr134, implicit-def $vgpr135, implicit-def $vgpr136, implicit-def $vgpr137, implicit-def $vgpr138, implicit-def $vgpr139, implicit-def $vgpr140, implicit-def $vgpr141, implicit-def $vgpr142, implicit-def $vgpr143, implicit-def $vgpr144, implicit-def $vgpr145, implicit-def $vgpr146, implicit-def $vgpr147, implicit-def $vgpr148, implicit-def $vgpr149, implicit-def $vgpr150, implicit-def $vgpr151, implicit-def $vgpr152, implicit-def $vgpr153, implicit-def $vgpr154, implicit-def $vgpr155, implicit-def $vgpr156, implicit-def $vgpr157, implicit-def $vgpr158, implicit-def $vgpr159, implicit-def $vgpr160, implicit-def $vgpr161, implicit-def $vgpr162, implicit-def $vgpr163, implicit-def $vgpr164, implicit-def $vgpr165, implicit-def $vgpr166, implicit-def $vgpr167, implicit-def $vgpr168, implicit-def $vgpr169, implicit-def $vgpr170, implicit-def $vgpr171, implicit-def $vgpr172, implicit-def $vgpr173, implicit-def $vgpr174, implicit-def $vgpr175, implicit-def $vgpr176, implicit-def $vgpr177, implicit-def $vgpr178, implicit-def $vgpr179, implicit-def $vgpr180, implicit-def $vgpr181, implicit-def $vgpr182, implicit-def $vgpr183, implicit-def $vgpr184, implicit-def $vgpr185, implicit-def $vgpr186, implicit-def $vgpr187, implicit-def $vgpr188, implicit-def $vgpr189, implicit-def $vgpr190, implicit-def $vgpr191, implicit-def $vgpr192, implicit-def $vgpr193, implicit-def $vgpr194, implicit-def $vgpr195, implicit-def $vgpr196, implicit-def $vgpr197, implicit-def $vgpr198, implicit-def $vgpr199, implicit-def $vgpr200, implicit-def $vgpr201, implicit-def $vgpr202, implicit-def $vgpr203, implicit-def $vgpr204, implicit-def $vgpr205, implicit-def $vgpr206, implicit-def $vgpr207, implicit-def $vgpr208, implicit-def $vgpr209, implicit-def $vgpr210, implicit-def $vgpr211, implicit-def $vgpr212, implicit-def $vgpr213, implicit-def $vgpr214, implicit-def $vgpr215, implicit-def $vgpr216, implicit-def $vgpr217, implicit-def $vgpr218, implicit-def $vgpr219, implicit-def $vgpr220, implicit-def $vgpr221, implicit-def $vgpr222, implicit-def $vgpr223, implicit-def $vgpr224, implicit-def $vgpr225, implicit-def $vgpr226, implicit-def $vgpr227, implicit-def $vgpr228, implicit-def $vgpr229, implicit-def $vgpr230, implicit-def $vgpr231, implicit-def $vgpr232, implicit-def $vgpr233, implicit-def $vgpr234, implicit-def $vgpr235, implicit-def $vgpr236, implicit-def $vgpr237, implicit-def $vgpr238, implicit-def $vgpr239, implicit-def $vgpr240, implicit-def $vgpr241, implicit-def $vgpr242, implicit-def $vgpr243, implicit-def $vgpr244, implicit-def $vgpr245, implicit-def $vgpr246, implicit-def $vgpr247, implicit-def $vgpr248, implicit-def $vgpr249, implicit-def $vgpr250, implicit-def $vgpr251, implicit-def $vgpr252, implicit-def $vgpr253, implicit-def $vgpr254, implicit-def $vgpr255
KILL %1:vgpr_32, %2:vgpr_32
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/rem_i128.ll b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
index a52d709ca28cc..72e94b6bbfc16 100644
--- a/llvm/test/CodeGen/AMDGPU/rem_i128.ll
+++ b/llvm/test/CodeGen/AMDGPU/rem_i128.ll
@@ -679,12 +679,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB0_4
; GFX9-O0-NEXT: .LBB0_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:244 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:248 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:252 ; 4-byte Folded Reload
@@ -709,8 +703,13 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:312 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:316 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:320 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -737,7 +736,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -747,7 +745,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -756,11 +753,9 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -895,9 +890,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB0_6
; GFX9-O0-NEXT: s_branch .LBB0_1
; GFX9-O0-NEXT: .LBB0_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:324 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:328 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:332 ; 4-byte Folded Reload
@@ -914,9 +906,12 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -990,6 +985,7 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:308 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:312 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -1021,9 +1017,6 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:248 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB0_6
; GFX9-O0-NEXT: .LBB0_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
@@ -1032,14 +1025,17 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -1149,6 +1145,7 @@ define i128 @v_srem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -2018,12 +2015,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_branch .LBB1_4
; GFX9-O0-NEXT: .LBB1_6: ; %udiv-do-while
; GFX9-O0-NEXT: ; =>This Inner Loop Header: Depth=1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
-; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
-; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:216 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:220 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:224 ; 4-byte Folded Reload
@@ -2048,8 +2039,13 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:284 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:288 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:292 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: v_readlane_b32 s6, v31, 8
+; GFX9-O0-NEXT: v_readlane_b32 s7, v31, 9
; GFX9-O0-NEXT: s_mov_b32 s4, 63
-; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_lshrrev_b64 v[7:8], s4, v[2:3]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v8
; GFX9-O0-NEXT: s_mov_b32 s5, 1
@@ -2076,7 +2072,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: v_lshrrev_b64 v[0:1], s4, v[0:1]
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v1
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v28
-; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v16, v26
; GFX9-O0-NEXT: v_or3_b32 v6, v6, v11, v16
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v0
@@ -2086,7 +2081,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr0 killed $vgpr0 def $vgpr0_vgpr1 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v1, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v3
-; GFX9-O0-NEXT: s_waitcnt vmcnt(8)
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v24
; GFX9-O0-NEXT: v_or_b32_e64 v6, v6, v11
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v2
@@ -2095,11 +2089,9 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: ; kill: def $vgpr2 killed $vgpr2 def $vgpr2_vgpr3 killed $exec
; GFX9-O0-NEXT: v_mov_b32_e32 v3, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v6, v7
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_mov_b32_e32 v11, v4
; GFX9-O0-NEXT: v_sub_co_u32_e64 v11, s[8:9], v11, v6
; GFX9-O0-NEXT: v_mov_b32_e32 v7, v8
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v5
; GFX9-O0-NEXT: v_subb_co_u32_e64 v4, s[8:9], v4, v7, s[8:9]
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v14
@@ -2234,9 +2226,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_cbranch_execnz .LBB1_6
; GFX9-O0-NEXT: s_branch .LBB1_1
; GFX9-O0-NEXT: .LBB1_7: ; %udiv-preheader
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:296 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:300 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:304 ; 4-byte Folded Reload
@@ -2253,9 +2242,12 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:48 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_waitcnt vmcnt(9)
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(10)
; GFX9-O0-NEXT: v_mov_b32_e32 v4, v10
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_lshrrev_b64 v[6:7], v4, v[18:19]
; GFX9-O0-NEXT: v_mov_b32_e32 v5, v7
; GFX9-O0-NEXT: s_mov_b32 s4, 64
@@ -2329,6 +2321,7 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v16, off, s[0:3], s32 offset:280 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_nop 0
; GFX9-O0-NEXT: buffer_store_dword v17, off, s[0:3], s32 offset:284 ; 4-byte Folded Spill
+; GFX9-O0-NEXT: s_waitcnt vmcnt(4)
; GFX9-O0-NEXT: v_writelane_b32 v31, s4, 8
; GFX9-O0-NEXT: v_writelane_b32 v31, s5, 9
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
@@ -2360,9 +2353,6 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:220 ; 4-byte Folded Spill
; GFX9-O0-NEXT: s_branch .LBB1_6
; GFX9-O0-NEXT: .LBB1_8: ; %udiv-bb1
-; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
-; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:36 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:40 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:44 ; 4-byte Folded Reload
@@ -2371,14 +2361,17 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:24 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:28 ; 4-byte Folded Reload
; GFX9-O0-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
+; GFX9-O0-NEXT: buffer_load_dword v31, off, s[0:3], s32 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: s_mov_b64 exec, s[20:21]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 1
; GFX9-O0-NEXT: s_mov_b32 s4, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(2)
; GFX9-O0-NEXT: v_mov_b32_e32 v2, v3
; GFX9-O0-NEXT: v_add_co_u32_e64 v0, s[4:5], v2, s4
; GFX9-O0-NEXT: s_mov_b32 s6, s7
; GFX9-O0-NEXT: v_mov_b32_e32 v1, s6
-; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
+; GFX9-O0-NEXT: s_waitcnt vmcnt(1)
; GFX9-O0-NEXT: v_addc_co_u32_e64 v3, s[4:5], v4, v1, s[4:5]
; GFX9-O0-NEXT: s_mov_b64 s[6:7], 0
; GFX9-O0-NEXT: s_mov_b32 s8, s6
@@ -2488,6 +2481,7 @@ define i128 @v_urem_i128_vv(i128 %lhs, i128 %rhs) {
; GFX9-O0-NEXT: s_mov_b64 s[6:7], exec
; GFX9-O0-NEXT: s_and_b64 s[4:5], s[6:7], s[4:5]
; GFX9-O0-NEXT: s_xor_b64 s[6:7], s[4:5], s[6:7]
+; GFX9-O0-NEXT: s_waitcnt vmcnt(16)
; GFX9-O0-NEXT: v_writelane_b32 v31, s6, 6
; GFX9-O0-NEXT: v_writelane_b32 v31, s7, 7
; GFX9-O0-NEXT: s_or_saveexec_b64 s[20:21], -1
diff --git a/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir b/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
index 9a2a8b39f3c02..846c24d9254ed 100644
--- a/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
+++ b/llvm/test/CodeGen/AMDGPU/remove-not-short-exec-branch-on-unconditional-jump.mir
@@ -27,7 +27,7 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.3(0x40000000)
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.4, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -41,12 +41,12 @@ body: |
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.5:
; CHECK-NEXT: liveins: $vgpr1, $sgpr0_sgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_CVT_F32_U32_e32 killed $vgpr1, implicit $mode, implicit $exec
; CHECK-NEXT: SI_RETURN_TO_EPILOG killed $vgpr0
bb.0:
@@ -66,7 +66,7 @@ body: |
bb.2:
liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
- $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
bb.3:
@@ -78,12 +78,12 @@ body: |
bb.4:
liveins: $vgpr1, $sgpr0_sgpr1, $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
bb.5:
liveins: $vgpr1, $sgpr0_sgpr1
- $exec = S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr0_sgpr1, implicit-def $scc
renamable $vgpr0 = V_CVT_F32_U32_e32 killed $vgpr1, implicit $mode, implicit $exec
SI_RETURN_TO_EPILOG killed $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir b/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
index f8fa2bda054c3..ad50d3d9d35d8 100644
--- a/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
+++ b/llvm/test/CodeGen/AMDGPU/s_or_saveexec_xor_combine.mir
@@ -18,8 +18,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
...
---
@@ -32,8 +32,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0_sgpr1
- renamable $sgpr0_sgpr1 = S_OR_SAVEEXEC_B64 killed renamable $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr0_sgpr1, implicit-def $scc
+ renamable $sgpr0_sgpr1 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr0_sgpr1, implicit-def $scc
...
---
@@ -46,8 +46,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 renamable $sgpr0, $exec_lo, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 renamable $sgpr0, $exec_lo, implicit-def $scc
...
---
@@ -60,8 +60,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr0, implicit-def $scc
...
---
@@ -75,8 +75,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec_lo = bb-prolog S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
...
---
@@ -92,7 +92,7 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0, $sgpr1
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr1 = S_MOV_B32 renamable $sgpr0
$exec_lo = S_XOR_B32 $exec_lo, renamable $sgpr1, implicit-def $scc
...
@@ -108,7 +108,7 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr0
- renamable $sgpr0 = S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
+ renamable $sgpr0 = bb-prolog S_OR_SAVEEXEC_B32 killed renamable $sgpr0, implicit-def $exec, implicit-def $scc, implicit $exec
...
---
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll b/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
index 2fa366417ee73..f6a9c48aaa326 100644
--- a/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-spill-no-vgprs.ll
@@ -121,6 +121,9 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: s_cbranch_vccnz .LBB0_2
; GCN-NEXT: ; %bb.1: ; %bb0
; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
+; GCN-NEXT: buffer_load_dword v22, off, s[0:3], 0 ; 4-byte Folded Reload
+; GCN-NEXT: s_mov_b64 exec, s[24:25]
+; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
; GCN-NEXT: buffer_load_dword v23, off, s[0:3], 0 offset:4 ; 4-byte Folded Reload
; GCN-NEXT: s_mov_b64 exec, s[24:25]
; GCN-NEXT: s_waitcnt vmcnt(0)
@@ -140,9 +143,6 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: v_readlane_b32 s17, v23, 13
; GCN-NEXT: v_readlane_b32 s18, v23, 14
; GCN-NEXT: v_readlane_b32 s19, v23, 15
-; GCN-NEXT: s_or_saveexec_b64 s[24:25], -1
-; GCN-NEXT: buffer_load_dword v22, off, s[0:3], 0 ; 4-byte Folded Reload
-; GCN-NEXT: s_mov_b64 exec, s[24:25]
; GCN-NEXT: ;;#ASMSTART
; GCN-NEXT: ; use s[4:19]
; GCN-NEXT: ;;#ASMEND
@@ -200,7 +200,6 @@ define amdgpu_kernel void @partial_no_vgprs_last_sgpr_spill(ptr addrspace(1) %ou
; GCN-NEXT: v_readlane_b32 s21, v23, 61
; GCN-NEXT: v_readlane_b32 s22, v23, 62
; GCN-NEXT: v_readlane_b32 s23, v23, 63
-; GCN-NEXT: s_waitcnt vmcnt(0)
; GCN-NEXT: v_readlane_b32 s4, v22, 0
; GCN-NEXT: v_readlane_b32 s5, v22, 1
; GCN-NEXT: ;;#ASMSTART
diff --git a/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir b/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
index 55ec919041e00..bc20ccf85df9a 100644
--- a/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
+++ b/llvm/test/CodeGen/AMDGPU/sgpr-spill-overlap-wwm-reserve.mir
@@ -40,145 +40,145 @@ body: |
; GCN-NEXT: SCRATCH_STORE_DWORD_SADDR killed $vgpr5, $sgpr33, 16, 0, implicit $exec, implicit $flat_scr :: ("amdgpu-thread-private" store (s32) into %stack.72, addrspace 5)
; GCN-NEXT: $exec_lo = S_MOV_B32 killed $sgpr0
; GCN-NEXT: $sgpr32 = frame-setup S_ADD_I32 $sgpr32, 24, implicit-def dead $scc
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr4, 0, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr4, 0, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr4, $vgpr2, 0, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr5, 1, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr5, 1, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr5, $vgpr2, 1, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr6, 2, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr6, 2, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr6, $vgpr2, 2, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr7, 3, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr7, 3, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr7, $vgpr2, 3, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr8, 4, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr8, 4, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr8, $vgpr2, 4, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr9, 5, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr9, 5, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr9, $vgpr2, 5, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr10, 6, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr10, 6, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr10, $vgpr2, 6, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr11, 7, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr11, 7, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr11, $vgpr2, 7, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr12, 8, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr12, 8, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr12, $vgpr2, 8, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr13, 9, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr13, 9, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr13, $vgpr2, 9, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr14, 10, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr14, 10, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr14, $vgpr2, 10, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr15, 11, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr15, 11, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr15, $vgpr2, 11, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr16, 12, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr16, 12, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr16, $vgpr2, 12, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr17, 13, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr17, 13, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr17, $vgpr2, 13, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr18, 14, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr18, 14, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr18, $vgpr2, 14, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr19, 15, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr19, 15, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr19, $vgpr2, 15, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr20, 16, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr20, 16, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr20, $vgpr2, 16, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr21, 17, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr21, 17, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr21, $vgpr2, 17, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr22, 18, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr22, 18, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr22, $vgpr2, 18, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr23, 19, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr23, 19, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr23, $vgpr2, 19, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr24, 20, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr24, 20, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr24, $vgpr2, 20, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr25, 21, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr25, 21, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr25, $vgpr2, 21, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr26, 22, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr26, 22, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr26, $vgpr2, 22, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr27, 23, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr27, 23, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr27, $vgpr2, 23, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr28, 24, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr28, 24, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr28, $vgpr2, 24, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr29, 25, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr29, 25, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr29, $vgpr2, 25, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr64, 26, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr64, 26, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr64, $vgpr2, 26, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr65, 27, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr65, 27, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr65, $vgpr2, 27, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr66, 28, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr66, 28, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr66, $vgpr2, 28, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr67, 29, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr67, 29, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr67, $vgpr2, 29, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr68, 30, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr68, 30, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr68, $vgpr2, 30, 32
- ; GCN-NEXT: $vgpr2 = SI_SPILL_S32_TO_VGPR $sgpr69, 31, $vgpr2
+ ; GCN-NEXT: $vgpr2 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr69, 31, $vgpr2
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr69, $vgpr2, 31, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr70, 0, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr70, 0, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr70, $vgpr3, 0, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr71, 1, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr71, 1, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr71, $vgpr3, 1, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr72, 2, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr72, 2, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr72, $vgpr3, 2, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr73, 3, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr73, 3, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr73, $vgpr3, 3, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr74, 4, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr74, 4, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr74, $vgpr3, 4, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr75, 5, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr75, 5, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr75, $vgpr3, 5, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr76, 6, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr76, 6, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr76, $vgpr3, 6, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr77, 7, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr77, 7, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr77, $vgpr3, 7, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr78, 8, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr78, 8, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr78, $vgpr3, 8, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr79, 9, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr79, 9, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr79, $vgpr3, 9, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr80, 10, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr80, 10, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr80, $vgpr3, 10, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr81, 11, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr81, 11, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr81, $vgpr3, 11, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr82, 12, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr82, 12, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr82, $vgpr3, 12, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr83, 13, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr83, 13, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr83, $vgpr3, 13, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr84, 14, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr84, 14, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr84, $vgpr3, 14, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr85, 15, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr85, 15, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr85, $vgpr3, 15, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr86, 16, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr86, 16, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr86, $vgpr3, 16, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr87, 17, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr87, 17, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr87, $vgpr3, 17, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr88, 18, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr88, 18, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr88, $vgpr3, 18, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr89, 19, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr89, 19, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr89, $vgpr3, 19, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr90, 20, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr90, 20, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr90, $vgpr3, 20, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr91, 21, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr91, 21, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr91, $vgpr3, 21, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr92, 22, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr92, 22, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr92, $vgpr3, 22, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr93, 23, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr93, 23, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr93, $vgpr3, 23, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr94, 24, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr94, 24, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr94, $vgpr3, 24, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr95, 25, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr95, 25, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr95, $vgpr3, 25, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr96, 26, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr96, 26, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr96, $vgpr3, 26, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr97, 27, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr97, 27, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr97, $vgpr3, 27, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr98, 28, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr98, 28, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr98, $vgpr3, 28, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr99, 29, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr99, 29, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr99, $vgpr3, 29, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr100, 30, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr100, 30, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr100, $vgpr3, 30, 32
- ; GCN-NEXT: $vgpr3 = SI_SPILL_S32_TO_VGPR $sgpr101, 31, $vgpr3
+ ; GCN-NEXT: $vgpr3 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr101, 31, $vgpr3
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr101, $vgpr3, 31, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr102, 0, $vgpr4
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr102, 0, $vgpr4
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr102, $vgpr4, 0, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr103, 1, $vgpr4
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr103, 1, $vgpr4
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $sgpr103, $vgpr4, 1, 32
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr30, 2, $vgpr4, implicit-def $sgpr30_sgpr31, implicit $sgpr30_sgpr31
- ; GCN-NEXT: $vgpr4 = SI_SPILL_S32_TO_VGPR $sgpr31, 3, $vgpr4, implicit $sgpr30_sgpr31
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr30, 2, $vgpr4, implicit-def $sgpr30_sgpr31, implicit $sgpr30_sgpr31
+ ; GCN-NEXT: $vgpr4 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr31, 3, $vgpr4, implicit $sgpr30_sgpr31
; GCN-NEXT: frame-setup CFI_INSTRUCTION llvm_vector_registers $pc_reg, $vgpr4, 2, 32, $vgpr4, 3, 32
- ; GCN-NEXT: $sgpr22 = IMPLICIT_DEF
- ; GCN-NEXT: $vgpr5 = IMPLICIT_DEF
- ; GCN-NEXT: $vgpr5 = SI_SPILL_S32_TO_VGPR $sgpr22, 0, killed $vgpr5
- ; GCN-NEXT: $sgpr0 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GCN-NEXT: $sgpr22 = bb-prolog IMPLICIT_DEF
+ ; GCN-NEXT: $vgpr5 = bb-prolog IMPLICIT_DEF
+ ; GCN-NEXT: $vgpr5 = bb-prolog SI_SPILL_S32_TO_VGPR $sgpr22, 0, killed $vgpr5
+ ; GCN-NEXT: $sgpr0 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: dead $vgpr1 = V_SET_INACTIVE_B32 0, $vgpr0, 0, 0, $sgpr_null, implicit $exec, implicit-def $scc
; GCN-NEXT: $exec_lo = EXIT_STRICT_WWM $sgpr0
; GCN-NEXT: {{ $}}
@@ -278,9 +278,9 @@ body: |
bb.0:
liveins: $vgpr0
- $sgpr22 = IMPLICIT_DEF
- SI_SPILL_S32_SAVE $sgpr22, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.0, addrspace 5)
- $sgpr0 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr22 = bb-prolog IMPLICIT_DEF
+ bb-prolog SI_SPILL_S32_SAVE $sgpr22, %stack.0, implicit $exec, implicit $sgpr32 :: (store (s32) into %stack.0, addrspace 5)
+ $sgpr0 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
%0:vgpr_32 = V_SET_INACTIVE_B32 0, $vgpr0, 0, 0, $sgpr_null, implicit $exec, implicit-def $scc
$exec_lo = EXIT_STRICT_WWM $sgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir b/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
index 67b6f10417ea4..0111f21e64c31 100644
--- a/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-fix-sgpr-copies.mir
@@ -31,7 +31,7 @@ body: |
S_BRANCH %bb.1
bb.2:
- SI_END_CF %1, implicit-def $exec, implicit-def $scc, implicit $exec
+ bb-prolog SI_END_CF %1, implicit-def $exec, implicit-def $scc, implicit $exec
%11 = S_MOV_B32 1
%2 = S_ADD_I32 %0, %11, implicit-def $scc
S_BRANCH %bb.1
diff --git a/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir b/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
index 9e01e873686b7..151de488d449d 100644
--- a/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-init-whole-wave.mir
@@ -16,7 +16,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr8
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; CHECK-NEXT: undef [[COPY1:%[0-9]+]].sub0:ccr_sgpr_64 = COPY $sgpr1
; CHECK-NEXT: [[COPY1:%[0-9]+]].sub1:ccr_sgpr_64 = COPY $sgpr2
@@ -33,7 +33,7 @@ body: |
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = V_ADD_U32_e64 5, [[COPY2]], 0, implicit $exec
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY3]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY3]], implicit-def $scc
; CHECK-NEXT: $vgpr8 = COPY [[COPY2]]
; CHECK-NEXT: $sgpr0 = COPY [[COPY]]
; CHECK-NEXT: [[COPY4:%[0-9]+]]:sreg_32 = COPY $sgpr0
@@ -56,7 +56,7 @@ body: |
%37:vgpr_32 = V_ADD_U32_e64 5, %37:vgpr_32, 0, implicit $exec
bb.2:
- $exec_lo = S_OR_B32 $exec_lo, %16:sreg_32_xm0_xexec, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %16:sreg_32_xm0_xexec, implicit-def $scc
$vgpr8 = COPY %37:vgpr_32
$sgpr0 = COPY %9:sreg_32
%2:sreg_32 = COPY $sgpr0
@@ -76,7 +76,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr8, $vgpr9
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def dead $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 -1, implicit-def $exec, implicit-def dead $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $sgpr0
; CHECK-NEXT: undef [[COPY1:%[0-9]+]].sub0:ccr_sgpr_64 = COPY $sgpr1
; CHECK-NEXT: [[COPY1:%[0-9]+]].sub1:ccr_sgpr_64 = COPY $sgpr2
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
index e0312a4d3e329..c82b84f09a18d 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-preserve-dom-tree.mir
@@ -57,9 +57,9 @@ body: |
bb.3:
successors: %bb.4(0x80000000)
- SI_END_CF killed %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %17, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.4:
- SI_END_CF killed %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %13, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
index 318d290440f63..36f3769ce9c70 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-control-flow-remove-redundant-block-liveintervals.mir
@@ -34,7 +34,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, [[COPY3]], implicit-def dead $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, [[COPY3]], implicit-def dead $scc
; CHECK-NEXT: [[REG_SEQUENCE:%[0-9]+]]:vreg_64 = REG_SEQUENCE [[COPY2]], %subreg.sub0, [[COPY1]], %subreg.sub1
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
@@ -70,7 +70,7 @@ body: |
S_BRANCH %bb.3
bb.2:
- SI_END_CF killed %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %4, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%7:vreg_64 = REG_SEQUENCE killed %2, %subreg.sub0, killed %1, %subreg.sub1
S_BRANCH %bb.5
@@ -86,7 +86,7 @@ body: |
S_BRANCH %bb.4
bb.4:
- SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_BRANCH %bb.2
bb.5:
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
index c53592c6114e7..5aeb0e84648c3 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies-order-of-phi-incomings.mir
@@ -129,7 +129,7 @@ body: |
successors: %bb.4(0x04000000), %bb.1(0x7c000000)
%20:vreg_1 = PHI %26, %bb.2, %19, %bb.1 ;%20:vreg_1 = PHI %19, %bb.1, %26, %bb.2 - this is original phi created by SDAG
- SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %22, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%27:sreg_64 = S_MOV_B64 4
%18:vreg_64 = V_ADD_U64_PSEUDO %17, killed %27, implicit-def dead $vcc, implicit $exec
%28:sreg_32 = S_MOV_B32 1
diff --git a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
index fbc617aa46d83..963b10f1aa1df 100644
--- a/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-lower-i1-copies.mir
@@ -57,11 +57,11 @@ body: |
bb.2:
%6:vreg_1 = PHI %5, %bb.1
- SI_END_CF %3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %3, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
bb.3:
%7:vreg_1 = PHI %6, %bb.2, %8, %bb.0
- SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
index d000f5a30b6dd..01a6e2d4f3750 100644
--- a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-bug-deadlanes.mir
@@ -94,7 +94,7 @@ body: |
bb.8:
%13:vreg_128 = PHI %10, %bb.6, %12, %bb.7
- SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %11, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:vreg_128 = REG_SEQUENCE %13.sub2, %subreg.sub0, %13.sub2, %subreg.sub1, killed %13.sub2, %subreg.sub2, undef %3, %subreg.sub3
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
index 23484ddef17ab..1729aa9d212e3 100644
--- a/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-opt-vgpr-liverange-undef-use-livevars.mir
@@ -76,6 +76,6 @@ body: |
bb.5:
%6:vreg_64 = PHI undef %7:vreg_64, %bb.3, %1, %bb.4
- SI_END_CF killed %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF killed %5, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
S_ENDPGM 0, implicit %6
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir b/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
index 9337d244c71a6..898d26252d9d5 100644
--- a/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-optimize-vgpr-live-range-dbg-instr.mir
@@ -106,7 +106,7 @@ body: |
S_BRANCH %bb.1
bb.4:
- SI_END_CF %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %7, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
DBG_VALUE_LIST !4, !DIExpression(DW_OP_LLVM_arg, 0, DW_OP_LLVM_arg, 1, DW_OP_constu, 2712, DW_OP_mul, DW_OP_plus, DW_OP_plus_uconst, 2680, DW_OP_stack_value), %5, 0, debug-location !9
SI_RETURN
...
diff --git a/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir b/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
index 14a94de24de22..64ef9f178fcb7 100644
--- a/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
+++ b/llvm/test/CodeGen/AMDGPU/si-pre-allocate-wwm-regs-preserve-rci.mir
@@ -23,8 +23,8 @@ tracksRegLiveness: true
body: |
bb.0:
liveins: $sgpr1
- %0:vgpr_32 = IMPLICIT_DEF
- renamable $sgpr4_sgpr5 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:vgpr_32 = bb-prolog IMPLICIT_DEF
+ renamable $sgpr4_sgpr5 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
%1:vgpr_32 = V_MOV_B32_e32 0, implicit $exec
%2:vgpr_32 = V_MOV_B32_dpp %1, %0, 323, 12, 15, 0, implicit $exec
$exec = EXIT_STRICT_WWM killed renamable $sgpr4_sgpr5
diff --git a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
index c0f0574411a6a..2ed42e286ba81 100644
--- a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
+++ b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow-postra.mir
@@ -28,7 +28,7 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.2
@@ -55,7 +55,7 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.8, implicit $exec
S_BRANCH %bb.2
@@ -96,7 +96,7 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.2
@@ -123,7 +123,7 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr9, $sgpr0_sgpr1
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.8, implicit $exec
S_BRANCH %bb.2
@@ -164,8 +164,8 @@ body: |
; GFX10-NEXT: successors: %bb.2(0x40000000), %bb.3(0x40000000)
; GFX10-NEXT: liveins: $sgpr6, $sgpr8, $sgpr0_sgpr1, $sgpr2_sgpr3
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr2 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
- ; GFX10-NEXT: $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr2 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_NOP 0, implicit $sgpr2_sgpr3
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.3, implicit $exec
@@ -193,8 +193,8 @@ body: |
successors: %bb.2(0x40000000), %bb.8(0x40000000)
liveins: $sgpr6, $sgpr8, $sgpr0_sgpr1, $sgpr2_sgpr3
- $sgpr2 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
- $sgpr4_sgpr5 = S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr2 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4_sgpr5 = bb-prolog S_AND_SAVEEXEC_B64 $sgpr0_sgpr1, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr14_sgpr15 = S_XOR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
S_NOP 0, implicit $sgpr2_sgpr3
S_CBRANCH_EXECZ %bb.8, implicit $exec
@@ -226,7 +226,7 @@ body: |
; GFX10-NEXT: successors: %bb.7(0x40000000), %bb.2(0x40000000)
; GFX10-NEXT: liveins: $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
; GFX10-NEXT: {{ $}}
@@ -238,7 +238,7 @@ body: |
; GFX10-NEXT: successors: %bb.5(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.4
@@ -262,7 +262,7 @@ body: |
; GFX10-NEXT: successors: %bb.3(0x80000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr5, $sgpr8, $vgpr0, $vgpr1
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
; GFX10-NEXT: INLINEASM &"", sideeffect attdialect, implicit killed renamable $vgpr1
; GFX10-NEXT: S_BRANCH %bb.3
; GFX10-NEXT: {{ $}}
@@ -270,7 +270,7 @@ body: |
; GFX10-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $vgpr0
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
; GFX10-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.8:
@@ -290,7 +290,7 @@ body: |
bb.1:
liveins: $sgpr8, $vgpr0
- $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -301,7 +301,7 @@ body: |
bb.2:
liveins: $sgpr4, $sgpr8, $vgpr0
- $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr8, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
@@ -324,14 +324,14 @@ body: |
bb.5:
liveins: $sgpr4, $sgpr5, $sgpr8, $vgpr0, $vgpr1
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
INLINEASM &"", sideeffect attdialect, implicit killed renamable $vgpr1
S_BRANCH %bb.2
bb.6:
liveins: $sgpr4, $sgpr8, $vgpr0
- $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.10:
@@ -358,7 +358,7 @@ body: |
; GFX10-NEXT: successors: %bb.7(0x40000000), %bb.2(0x40000000)
; GFX10-NEXT: liveins: $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
; GFX10-NEXT: {{ $}}
@@ -370,7 +370,7 @@ body: |
; GFX10-NEXT: successors: %bb.5(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; GFX10-NEXT: $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
; GFX10-NEXT: renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
; GFX10-NEXT: S_CBRANCH_EXECZ %bb.5, implicit $exec
; GFX10-NEXT: S_BRANCH %bb.4
@@ -395,7 +395,7 @@ body: |
; GFX10-NEXT: successors: %bb.3(0x80000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr5, $sgpr6, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
; GFX10-NEXT: INLINEASM &"", sideeffect attdialect, implicit killed renamable $sgpr6
; GFX10-NEXT: S_BRANCH %bb.3
; GFX10-NEXT: {{ $}}
@@ -403,7 +403,7 @@ body: |
; GFX10-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; GFX10-NEXT: liveins: $sgpr4, $sgpr8, $sgpr9
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
; GFX10-NEXT: S_CBRANCH_EXECNZ %bb.1, implicit $exec
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.8:
@@ -424,7 +424,7 @@ body: |
bb.1:
liveins: $sgpr8, $sgpr9
- $sgpr4 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr4 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr4 = S_XOR_B32 $exec_lo, killed renamable $sgpr4, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.6, implicit $exec
@@ -435,7 +435,7 @@ body: |
bb.2:
liveins: $sgpr4, $sgpr8, $sgpr9
- $sgpr5 = S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
+ $sgpr5 = bb-prolog S_AND_SAVEEXEC_B32 $sgpr9, implicit-def $exec, implicit-def $scc, implicit $exec
renamable $sgpr5 = S_XOR_B32 $exec_lo, killed renamable $sgpr5, implicit-def dead $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
@@ -458,14 +458,14 @@ body: |
bb.5:
liveins: $sgpr4, $sgpr5, $sgpr6, $sgpr8, $sgpr9
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr5, implicit-def $scc
INLINEASM &"", sideeffect attdialect, implicit killed renamable $sgpr6
S_BRANCH %bb.2
bb.6:
liveins: $sgpr4, $sgpr8, $sgpr9
- $exec_lo = S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
+ $exec_lo = bb-prolog S_ANDN2_B32 $exec_lo, killed renamable $sgpr4, implicit-def $scc
S_CBRANCH_EXECNZ %bb.1, implicit $exec
bb.10:
diff --git a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
index 653c2c6f5475b..95030c8650acd 100644
--- a/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
+++ b/llvm/test/CodeGen/AMDGPU/sink-after-control-flow.mir
@@ -36,7 +36,7 @@ body: |
; GFX10-NEXT: bb.2:
; GFX10-NEXT: successors: %bb.3(0x40000000), %bb.4(0x40000000)
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_XOR_B32_1]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_XOR_B32_1]], implicit-def $scc
; GFX10-NEXT: [[V_LSHRREV_B32_e64_:%[0-9]+]]:vgpr_32 = V_LSHRREV_B32_e64 [[S_MOV_B32_]], [[DEF]], implicit $exec
; GFX10-NEXT: [[S_MOV_B32_2:%[0-9]+]]:sreg_32 = S_MOV_B32 31
; GFX10-NEXT: [[V_CMP_NE_U32_e64_1:%[0-9]+]]:sreg_32 = V_CMP_NE_U32_e64 [[V_BFE_U32_e64_]], killed [[S_MOV_B32_2]], implicit $exec
@@ -55,7 +55,7 @@ body: |
; GFX10-NEXT: bb.4:
; GFX10-NEXT: successors: %bb.5(0x80000000)
; GFX10-NEXT: {{ $}}
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_XOR_B32_3]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_XOR_B32_3]], implicit-def $scc
; GFX10-NEXT: [[DEF2:%[0-9]+]]:sreg_32 = IMPLICIT_DEF
; GFX10-NEXT: [[S_MOV_B32_3:%[0-9]+]]:sreg_32 = S_MOV_B32 16
; GFX10-NEXT: [[S_MOV_B32_4:%[0-9]+]]:sreg_32 = S_MOV_B32 4
@@ -63,7 +63,7 @@ body: |
; GFX10-NEXT: S_BRANCH %bb.5
; GFX10-NEXT: {{ $}}
; GFX10-NEXT: bb.5:
- ; GFX10-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[DEF2]], implicit-def $scc
+ ; GFX10-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[DEF2]], implicit-def $scc
; GFX10-NEXT: S_ENDPGM 0
bb.0:
successors: %bb.4(0x40000000), %bb.5(0x40000000)
@@ -91,7 +91,7 @@ body: |
bb.5:
successors: %bb.6(0x40000000), %bb.7(0x40000000)
- $exec_lo = S_OR_B32 $exec_lo, %23, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %23, implicit-def $scc
%24:sreg_32 = S_MOV_B32 31
%25:sreg_32 = V_CMP_NE_U32_e64 %18, killed %24, implicit $exec
%26:sreg_32 = S_XOR_B32 %25, -1, implicit-def $scc
@@ -109,14 +109,14 @@ body: |
bb.7:
successors: %bb.8(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %28, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %28, implicit-def $scc
%29:sreg_32 = S_MOV_B32 16
%30:sreg_32 = S_MOV_B32 4
%31:vgpr_32 = nuw nsw V_LSHL_ADD_U32_e64 %17, %30, killed %29, implicit $exec
S_BRANCH %bb.8
bb.8:
- $exec_lo = S_OR_B32 $exec_lo, %14, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %14, implicit-def $scc
S_ENDPGM 0
...
diff --git a/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir b/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
index a4b34780879ef..616b3c8ca8802 100644
--- a/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
+++ b/llvm/test/CodeGen/AMDGPU/skip-branch-taildup-ret.mir
@@ -30,15 +30,15 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
- ; CHECK-NEXT: $exec = S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $sgpr2_sgpr3
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
; CHECK-NEXT: S_ENDPGM 0
; CHECK-NEXT: {{ $}}
@@ -86,15 +86,15 @@ body: |
successors: %bb.3, %bb.4
liveins: $sgpr0_sgpr1, $sgpr2_sgpr3
- renamable $sgpr2_sgpr3 = S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
+ renamable $sgpr2_sgpr3 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr2_sgpr3, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
bb.4:
liveins: $sgpr2_sgpr3
- $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def $scc
renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
S_ENDPGM 0
@@ -141,15 +141,15 @@ body: |
; CHECK-NEXT: successors: %bb.4(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr30_sgpr31, $vgpr1_vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
- ; CHECK-NEXT: $exec = S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $sgpr6_sgpr7, $sgpr30_sgpr31
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
; CHECK-NEXT: S_SETPC_B64_return $sgpr30_sgpr31
; CHECK-NEXT: {{ $}}
@@ -185,15 +185,15 @@ body: |
successors: %bb.3, %bb.4
liveins: $sgpr6_sgpr7, $sgpr30_sgpr31, $vgpr1_vgpr2
- renamable $sgpr6_sgpr7 = S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
+ renamable $sgpr6_sgpr7 = bb-prolog S_OR_SAVEEXEC_B64 killed renamable $sgpr6_sgpr7, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog S_XOR_B64 $exec, renamable $sgpr6_sgpr7, implicit-def $scc
S_CBRANCH_EXECZ %bb.4, implicit $exec
S_BRANCH %bb.3
bb.4:
liveins: $sgpr6_sgpr7, $sgpr30_sgpr31
- $exec = S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr6_sgpr7, implicit-def $scc
renamable $vgpr0 = V_MOV_B32_e32 32, implicit $exec
S_SETPC_B64_return $sgpr30_sgpr31
diff --git a/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir b/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
index 27a0f9ea3a6a4..cf03f0be4a20c 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
+++ b/llvm/test/CodeGen/AMDGPU/spill-before-exec.mir
@@ -47,7 +47,7 @@ body: |
bb.1:
liveins: $sgpr96_sgpr97, $sgpr98_sgpr99, $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr96_sgpr97, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr96_sgpr97, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_CBRANCH_EXECZ %bb.3, implicit $exec
S_BRANCH %bb.2
@@ -55,7 +55,7 @@ body: |
bb.2:
liveins: $sgpr98_sgpr99, $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr98_sgpr99, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr98_sgpr99, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_CBRANCH_EXECZ %bb.3, implicit $exec
S_BRANCH %bb.4
@@ -63,7 +63,7 @@ body: |
bb.3:
liveins: $sgpr102_sgpr103
- %0:sreg_64 = S_OR_SAVEEXEC_B64 $sgpr102_sgpr103, implicit-def $exec, implicit-def $scc, implicit $exec
+ %0:sreg_64 = bb-prolog S_OR_SAVEEXEC_B64 $sgpr102_sgpr103, implicit-def $exec, implicit-def $scc, implicit $exec
$exec = S_XOR_B64_term $exec, %0, implicit-def $scc
S_BRANCH %bb.4
diff --git a/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll b/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
index d45833b1fc262..9ba2cbb2f29dc 100644
--- a/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
+++ b/llvm/test/CodeGen/AMDGPU/spill-scavenge-offset.ll
@@ -9741,36 +9741,23 @@ entry:
define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspace(1) %in) #0 {
; GFX6-LABEL: test_limited_sgpr:
; GFX6: ; %bb.0: ; %entry
-; GFX6-NEXT: s_mov_b32 s40, SCRATCH_RSRC_DWORD0
-; GFX6-NEXT: s_mov_b32 s41, SCRATCH_RSRC_DWORD1
; GFX6-NEXT: s_load_dwordx4 s[0:3], s[4:5], 0x9
-; GFX6-NEXT: s_mov_b32 s42, -1
-; GFX6-NEXT: s_mov_b32 s43, 0xe8f000
-; GFX6-NEXT: s_add_u32 s40, s40, s11
; GFX6-NEXT: v_mbcnt_lo_u32_b32_e64 v0, -1, 0
-; GFX6-NEXT: s_addc_u32 s41, s41, 0
; GFX6-NEXT: v_mbcnt_hi_u32_b32_e32 v0, -1, v0
-; GFX6-NEXT: v_mov_b32_e32 v6, 0
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
; GFX6-NEXT: s_mov_b32 s6, 0
+; GFX6-NEXT: v_mov_b32_e32 v6, 0
; GFX6-NEXT: s_mov_b32 s7, 0xf000
-; GFX6-NEXT: s_mov_b64 exec, 15
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], 0
-; GFX6-NEXT: s_waitcnt expcnt(0) lgkmcnt(0)
-; GFX6-NEXT: v_writelane_b32 v1, s0, 0
-; GFX6-NEXT: v_writelane_b32 v1, s1, 1
-; GFX6-NEXT: v_writelane_b32 v1, s2, 2
-; GFX6-NEXT: v_writelane_b32 v1, s3, 3
-; GFX6-NEXT: s_mov_b32 s8, 0x80400
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s8 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v1, off, s[40:43], 0
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
+; GFX6-NEXT: s_waitcnt lgkmcnt(0)
; GFX6-NEXT: s_mov_b64 s[4:5], s[2:3]
; GFX6-NEXT: v_lshlrev_b32_e32 v5, 8, v0
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:240
-; GFX6-NEXT: s_mov_b32 s2, 0x86800
+; GFX6-NEXT: s_mov_b32 s40, SCRATCH_RSRC_DWORD0
+; GFX6-NEXT: s_mov_b32 s41, SCRATCH_RSRC_DWORD1
+; GFX6-NEXT: s_mov_b32 s42, -1
+; GFX6-NEXT: s_mov_b32 s43, 0xe8f000
+; GFX6-NEXT: s_add_u32 s40, s40, s11
+; GFX6-NEXT: s_addc_u32 s41, s41, 0
+; GFX6-NEXT: s_mov_b32 s2, 0x85c00
; GFX6-NEXT: s_mov_b64 s[8:9], exec
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
@@ -9779,7 +9766,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:224
-; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: s_mov_b32 s2, 0x85800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9787,7 +9774,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:208
-; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: s_mov_b32 s2, 0x85400
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9795,7 +9782,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:192
-; GFX6-NEXT: s_mov_b32 s2, 0x85c00
+; GFX6-NEXT: s_mov_b32 s2, 0x85000
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9803,7 +9790,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:176
-; GFX6-NEXT: s_mov_b32 s2, 0x85800
+; GFX6-NEXT: s_mov_b32 s2, 0x84c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9811,7 +9798,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:160
-; GFX6-NEXT: s_mov_b32 s2, 0x85400
+; GFX6-NEXT: s_mov_b32 s2, 0x84800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9819,7 +9806,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:144
-; GFX6-NEXT: s_mov_b32 s2, 0x85000
+; GFX6-NEXT: s_mov_b32 s2, 0x84400
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9827,7 +9814,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:128
-; GFX6-NEXT: s_mov_b32 s2, 0x84c00
+; GFX6-NEXT: s_mov_b32 s2, 0x84000
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9835,7 +9822,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:112
-; GFX6-NEXT: s_mov_b32 s2, 0x84800
+; GFX6-NEXT: s_mov_b32 s2, 0x83c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9843,7 +9830,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:96
-; GFX6-NEXT: s_mov_b32 s2, 0x84400
+; GFX6-NEXT: s_mov_b32 s2, 0x83800
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
@@ -9851,20 +9838,14 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:80
-; GFX6-NEXT: s_mov_b32 s2, 0x84000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64 offset:64
-; GFX6-NEXT: s_mov_b32 s2, 0x83800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_load_dwordx4 v[12:15], v[5:6], s[4:7], 0 addr64 offset:32
+; GFX6-NEXT: s_mov_b32 s2, 0x83400
+; GFX6-NEXT: s_waitcnt vmcnt(1)
; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_load_dwordx4 v[16:19], v[5:6], s[4:7], 0 addr64 offset:64
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dwordx4 v[0:3], v[5:6], s[4:7], 0 addr64
; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:16
@@ -9874,14 +9855,6 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:32
-; GFX6-NEXT: s_mov_b32 s2, 0x83400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v7, off, s[40:43], s2 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
; GFX6-NEXT: s_mov_b64 s[2:3], s[6:7]
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
@@ -9890,24 +9863,17 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s1, 1
; GFX6-NEXT: v_writelane_b32 v4, s2, 2
; GFX6-NEXT: v_writelane_b32 v4, s3, 3
-; GFX6-NEXT: s_mov_b32 s10, 0x80800
+; GFX6-NEXT: s_mov_b32 s10, 0x80400
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s10 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: s_mov_b64 exec, s[8:9]
-; GFX6-NEXT: buffer_load_dwordx4 v[7:10], v[5:6], s[4:7], 0 addr64 offset:48
-; GFX6-NEXT: s_mov_b32 s0, 0x83c00
+; GFX6-NEXT: buffer_load_dwordx4 v[20:23], v[5:6], s[4:7], 0 addr64 offset:48
; GFX6-NEXT: v_lshlrev_b32_e32 v4, 13, v0
; GFX6-NEXT: v_add_i32_e32 v4, vcc, 16, v4
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dword v7, off, s[40:43], s0 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Spill
-; GFX6-NEXT: buffer_store_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Spill
-; GFX6-NEXT: s_waitcnt expcnt(3)
; GFX6-NEXT: v_mov_b32_e32 v7, 1
-; GFX6-NEXT: s_mov_b64 s[0:1], exec
+; GFX6-NEXT: s_mov_b64 s[2:3], exec
; GFX6-NEXT: buffer_store_dword v7, v4, s[40:43], 0 offen
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s[4:11]
@@ -9923,12 +9889,12 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s9, 5
; GFX6-NEXT: v_writelane_b32 v4, s10, 6
; GFX6-NEXT: v_writelane_b32 v4, s11, 7
-; GFX6-NEXT: s_mov_b32 s2, 0x80c00
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s12, 0x80800
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s12 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[0:1]
+; GFX6-NEXT: s_mov_b64 exec, s[2:3]
; GFX6-NEXT: v_cmp_eq_u32_e32 vcc, 0, v0
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s[8:15]
@@ -9940,10 +9906,10 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ; def s[24:31]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
-; GFX6-NEXT: ; def s[0:3]
+; GFX6-NEXT: ; def s[4:7]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
-; GFX6-NEXT: ; def s[4:5]
+; GFX6-NEXT: ; def s[2:3]
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; def s33
@@ -9951,7 +9917,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: s_and_saveexec_b64 s[34:35], vcc
; GFX6-NEXT: s_cbranch_execz .LBB1_2
; GFX6-NEXT: ; %bb.1: ; %bb0
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -9963,18 +9929,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s13, 5
; GFX6-NEXT: v_writelane_b32 v4, s14, 6
; GFX6-NEXT: v_writelane_b32 v4, s15, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x81400
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x81000
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x80c00
+; GFX6-NEXT: s_mov_b32 s38, 0x80800
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s8, v4, 0
; GFX6-NEXT: v_readlane_b32 s9, v4, 1
@@ -9986,8 +9952,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s15, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -9999,18 +9965,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s21, 5
; GFX6-NEXT: v_writelane_b32 v4, s22, 6
; GFX6-NEXT: v_writelane_b32 v4, s23, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x81c00
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x81800
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x81400
+; GFX6-NEXT: s_mov_b32 s38, 0x81000
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s16, v4, 0
; GFX6-NEXT: v_readlane_b32 s17, v4, 1
@@ -10022,8 +9988,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s23, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -10035,18 +10001,18 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s29, 5
; GFX6-NEXT: v_writelane_b32 v4, s30, 6
; GFX6-NEXT: v_writelane_b32 v4, s31, 7
-; GFX6-NEXT: s_mov_b32 s36, 0x82400
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s38, 0x82000
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s36, 0x81c00
+; GFX6-NEXT: s_mov_b32 s38, 0x81800
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s36 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
; GFX6-NEXT: v_readlane_b32 s24, v4, 0
; GFX6-NEXT: v_readlane_b32 s25, v4, 1
@@ -10058,8 +10024,8 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_readlane_b32 s31, v4, 7
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 s[6:7], exec
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt expcnt(0)
@@ -10067,17 +10033,31 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: v_writelane_b32 v4, s1, 1
; GFX6-NEXT: v_writelane_b32 v4, s2, 2
; GFX6-NEXT: v_writelane_b32 v4, s3, 3
-; GFX6-NEXT: s_mov_b32 s36, 0x82c00
+; GFX6-NEXT: s_mov_b32 s38, 0x82c00
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s38 ; 4-byte Folded Spill
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: s_mov_b64 exec, s[36:37]
+; GFX6-NEXT: s_mov_b64 s[0:1], exec
+; GFX6-NEXT: s_mov_b64 exec, 15
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: v_writelane_b32 v4, s4, 0
+; GFX6-NEXT: v_writelane_b32 v4, s5, 1
+; GFX6-NEXT: v_writelane_b32 v4, s6, 2
+; GFX6-NEXT: v_writelane_b32 v4, s7, 3
+; GFX6-NEXT: s_mov_b32 s36, 0x82800
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], s36 ; 4-byte Folded Spill
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[6:7]
-; GFX6-NEXT: s_mov_b64 vcc, s[4:5]
+; GFX6-NEXT: s_mov_b64 exec, s[0:1]
+; GFX6-NEXT: s_mov_b64 vcc, s[2:3]
; GFX6-NEXT: s_mov_b64 s[36:37], exec
; GFX6-NEXT: s_mov_b64 exec, 0xff
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s38, 0x82400
+; GFX6-NEXT: s_mov_b32 s38, 0x82000
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s38 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
@@ -10095,7 +10075,7 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: s_mov_b64 s[44:45], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: v_mov_b32_e32 v7, 0x20b0
+; GFX6-NEXT: v_mov_b32_e32 v7, 0x20a0
; GFX6-NEXT: s_waitcnt expcnt(0)
; GFX6-NEXT: buffer_load_dword v4, v7, s[40:43], 0 offen ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
@@ -10109,8 +10089,63 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ; use s[8:15],s[16:23],s[24:31],s[0:7],s[36:39],vcc
; GFX6-NEXT: ;;#ASMEND
+; GFX6-NEXT: s_mov_b64 s[4:5], exec
+; GFX6-NEXT: s_mov_b64 exec, 15
+; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_mov_b32 s6, 0x82c00
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: v_readlane_b32 s0, v4, 0
+; GFX6-NEXT: v_readlane_b32 s1, v4, 1
+; GFX6-NEXT: v_readlane_b32 s2, v4, 2
+; GFX6-NEXT: v_readlane_b32 s3, v4, 3
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: s_mov_b64 exec, s[4:5]
+; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: buffer_store_dword v0, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: buffer_store_dword v12, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v13, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v14, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v15, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: s_mov_b32 s2, 0x86800
+; GFX6-NEXT: buffer_store_dword v16, off, s[40:43], s2 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v17, off, s[40:43], s2 offset:4 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v18, off, s[40:43], s2 offset:8 ; 4-byte Folded Spill
+; GFX6-NEXT: buffer_store_dword v19, off, s[40:43], s2 offset:12 ; 4-byte Folded Spill
+; GFX6-NEXT: v_mov_b32_e32 v0, v20
+; GFX6-NEXT: v_mov_b32_e32 v1, v21
+; GFX6-NEXT: v_mov_b32_e32 v2, v22
+; GFX6-NEXT: v_mov_b32_e32 v3, v23
+; GFX6-NEXT: s_waitcnt expcnt(3)
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ;;#ASMEND
+; GFX6-NEXT: buffer_load_dword v16, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(2)
+; GFX6-NEXT: buffer_load_dword v17, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(1)
+; GFX6-NEXT: buffer_load_dword v18, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v19, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s2, 0x86400
+; GFX6-NEXT: buffer_load_dword v12, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v13, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v14, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v15, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s2, 0x86000
+; GFX6-NEXT: v_mov_b32_e32 v23, v3
+; GFX6-NEXT: v_mov_b32_e32 v22, v2
+; GFX6-NEXT: v_mov_b32_e32 v21, v1
+; GFX6-NEXT: v_mov_b32_e32 v20, v0
+; GFX6-NEXT: buffer_load_dword v0, off, s[40:43], s2 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v1, off, s[40:43], s2 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v2, off, s[40:43], s2 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v3, off, s[40:43], s2 offset:12 ; 4-byte Folded Reload
; GFX6-NEXT: ;;#ASMSTART
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: ;;#ASMSTART
@@ -10125,156 +10160,120 @@ define amdgpu_kernel void @test_limited_sgpr(ptr addrspace(1) %out, ptr addrspac
; GFX6-NEXT: ;;#ASMEND
; GFX6-NEXT: .LBB1_2: ; %ret
; GFX6-NEXT: s_or_b64 exec, exec, s[34:35]
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
+; GFX6-NEXT: s_mov_b64 s[2:3], exec
; GFX6-NEXT: s_mov_b64 exec, 15
; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s6, 0x80400
+; GFX6-NEXT: s_mov_b32 s8, 0x80400
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s8 ; 4-byte Folded Reload
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_readlane_b32 s0, v4, 0
-; GFX6-NEXT: v_readlane_b32 s1, v4, 1
-; GFX6-NEXT: v_readlane_b32 s2, v4, 2
-; GFX6-NEXT: v_readlane_b32 s3, v4, 3
+; GFX6-NEXT: v_readlane_b32 s4, v4, 0
+; GFX6-NEXT: v_readlane_b32 s5, v4, 1
+; GFX6-NEXT: v_readlane_b32 s6, v4, 2
+; GFX6-NEXT: v_readlane_b32 s7, v4, 3
; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
-; GFX6-NEXT: s_mov_b64 s[36:37], s[0:1]
-; GFX6-NEXT: s_mov_b64 s[4:5], exec
-; GFX6-NEXT: s_mov_b64 exec, 15
-; GFX6-NEXT: buffer_store_dword v4, off, s[40:43], 0
-; GFX6-NEXT: s_mov_b32 s6, 0x80800
+; GFX6-NEXT: s_mov_b64 exec, s[2:3]
+; GFX6-NEXT: s_mov_b32 s4, 0x85c00
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b64 s[2:3], s[6:7]
+; GFX6-NEXT: s_mov_b32 s4, 0x85800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:240
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], s6 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x85400
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: v_readlane_b32 s0, v4, 0
-; GFX6-NEXT: v_readlane_b32 s1, v4, 1
-; GFX6-NEXT: v_readlane_b32 s2, v4, 2
-; GFX6-NEXT: v_readlane_b32 s3, v4, 3
-; GFX6-NEXT: buffer_load_dword v4, off, s[40:43], 0
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:224
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x85000
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: s_mov_b64 exec, s[4:5]
-; GFX6-NEXT: s_mov_b32 s0, 0x86800
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b64 s[38:39], s[2:3]
-; GFX6-NEXT: s_mov_b32 s0, 0x86400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:240
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x86000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:224
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:208
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:192
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:176
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x85000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:160
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:144
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:128
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84400
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:112
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x84000
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:96
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83800
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:80
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83c00
-; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:64
-; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83400
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:208
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84c00
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:192
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:176
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84400
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:48
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:160
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
-; GFX6-NEXT: s_mov_b32 s0, 0x83000
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x84000
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:32
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:144
; GFX6-NEXT: s_waitcnt expcnt(0)
-; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s0 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s0 offset:4 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s0 offset:8 ; 4-byte Folded Reload
-; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s0 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83c00
; GFX6-NEXT: s_waitcnt vmcnt(0)
-; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[36:39], 0 addr64 offset:16
-; GFX6-NEXT: buffer_store_dwordx4 v[0:3], v[5:6], s[36:39], 0 addr64
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:128
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83800
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:112
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83400
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:96
+; GFX6-NEXT: s_waitcnt expcnt(0)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_mov_b32 s4, 0x83000
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:80
+; GFX6-NEXT: buffer_store_dwordx4 v[16:19], v[5:6], s[0:3], 0 addr64 offset:64
+; GFX6-NEXT: buffer_store_dwordx4 v[20:23], v[5:6], s[0:3], 0 addr64 offset:48
+; GFX6-NEXT: buffer_store_dwordx4 v[12:15], v[5:6], s[0:3], 0 addr64 offset:32
+; GFX6-NEXT: s_waitcnt expcnt(3)
+; GFX6-NEXT: buffer_load_dword v7, off, s[40:43], s4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v8, off, s[40:43], s4 offset:4 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v9, off, s[40:43], s4 offset:8 ; 4-byte Folded Reload
+; GFX6-NEXT: buffer_load_dword v10, off, s[40:43], s4 offset:12 ; 4-byte Folded Reload
+; GFX6-NEXT: s_waitcnt vmcnt(0)
+; GFX6-NEXT: buffer_store_dwordx4 v[7:10], v[5:6], s[0:3], 0 addr64 offset:16
+; GFX6-NEXT: buffer_store_dwordx4 v[0:3], v[5:6], s[0:3], 0 addr64
; GFX6-NEXT: s_endpgm
;
; GFX9-FLATSCR-LABEL: test_limited_sgpr:
diff --git a/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir b/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
index 3810fc46792fc..2d21bd8775787 100644
--- a/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
+++ b/llvm/test/CodeGen/AMDGPU/stale-livevar-in-twoaddr-pass.mir
@@ -21,7 +21,7 @@ body: |
bb.2:
%4:vgpr_32 = PHI %5, %bb.3, %3, %bb.1
- SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %6, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
EXP_DONE 0, %4, %4, %4, %4, -1, 0, 15, implicit $exec
S_ENDPGM 0
diff --git a/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir b/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
index a3ebe94e04684..81d036b61aa8b 100644
--- a/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
+++ b/llvm/test/CodeGen/AMDGPU/stop-tail-duplicate-cfg-intrinsic.mir
@@ -57,7 +57,7 @@ body: |
S_BRANCH %bb.4
bb.3:
- SI_END_CF %8:sreg_64_xexec, implicit-def $exec, implicit-def $scc, implicit $exec
+ bb-prolog SI_END_CF %8:sreg_64_xexec, implicit-def $exec, implicit-def $scc, implicit $exec
%13:sreg_32 = S_MOV_B32 1
%15:vgpr_32 = COPY %13:sreg_32
%10:vgpr_32, dead %20:sreg_64_xexec = V_ADD_CO_U32_e64 %6:vgpr_32, %15:vgpr_32, 0, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir b/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
index 43b35d523e8c4..44db5cef5ddd3 100644
--- a/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
+++ b/llvm/test/CodeGen/AMDGPU/subreg-split-live-in-error.mir
@@ -218,7 +218,7 @@ body: |
bb.18:
successors: %bb.20(0x40000000), %bb.19(0x40000000)
- $exec = S_OR_B64 $exec, %47, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, %47, implicit-def $scc
%52:vgpr_32 = V_MAD_F32_e64 0, %3.sub1, 0, target-flags(amdgpu-gotprel32-lo) 0, 1, %3.sub0, 0, 0, implicit $mode, implicit $exec
%53:vgpr_32 = V_MUL_F32_e32 -2147483648, %3.sub1, implicit $mode, implicit $exec
%53:vgpr_32 = V_MAC_F32_e32 target-flags(amdgpu-gotprel32-hi) 1065353216, %3.sub2, %53, implicit $mode, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll b/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
index d9002facc2917..ae61f265ec765 100644
--- a/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
+++ b/llvm/test/CodeGen/AMDGPU/transform-block-with-return-to-epilog.ll
@@ -121,7 +121,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: successors: %bb.9(0x80000000)
; GCN-NEXT: liveins: $sgpr2_sgpr3
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def dead $scc
; GCN-NEXT: S_BRANCH %bb.9
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.3.flow.preheader:
@@ -144,7 +144,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: successors: %bb.6(0x40000000), %bb.2(0x40000000)
; GCN-NEXT: liveins: $sgpr0_sgpr1, $sgpr2_sgpr3, $sgpr4_sgpr5
; GCN-NEXT: {{ $}}
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr4_sgpr5, implicit-def dead $scc
; GCN-NEXT: $sgpr2_sgpr3 = S_ANDN2_SAVEEXEC_B64 killed $sgpr2_sgpr3, implicit-def $exec, implicit-def $scc, implicit $exec
; GCN-NEXT: S_CBRANCH_EXECZ %bb.2, implicit $exec
; GCN-NEXT: {{ $}}
@@ -160,7 +160,7 @@ define amdgpu_ps { <4 x float> } @test_return_to_epilog_with_optimized_kill(floa
; GCN-NEXT: liveins: $sgpr2_sgpr3, $scc
; GCN-NEXT: {{ $}}
; GCN-NEXT: $exec = S_MOV_B64 0
- ; GCN-NEXT: $exec = S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def dead $scc
+ ; GCN-NEXT: $exec = bb-prolog S_OR_B64 $exec, killed renamable $sgpr2_sgpr3, implicit-def dead $scc
; GCN-NEXT: S_BRANCH %bb.9
; GCN-NEXT: {{ $}}
; GCN-NEXT: bb.8:
diff --git a/llvm/test/CodeGen/AMDGPU/trap-abis.ll b/llvm/test/CodeGen/AMDGPU/trap-abis.ll
index 5a359a5b2f3cc..558f9c3e1a09c 100644
--- a/llvm/test/CodeGen/AMDGPU/trap-abis.ll
+++ b/llvm/test/CodeGen/AMDGPU/trap-abis.ll
@@ -379,15 +379,14 @@ define amdgpu_kernel void @trap_with_use_after(ptr addrspace(1) %arg0, ptr addrs
; HSA-TRAP-GFX1100-O0-NEXT: scratch_store_b32 off, v0, off offset:4 ; 4-byte Folded Spill
; HSA-TRAP-GFX1100-O0-NEXT: s_cbranch_execnz .LBB2_2
; HSA-TRAP-GFX1100-O0-NEXT: ; %bb.1:
+; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v0, off, off offset:8 ; 4-byte Folded Reload
+; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v1, off, off offset:4 ; 4-byte Folded Reload
; HSA-TRAP-GFX1100-O0-NEXT: s_or_saveexec_b32 s6, -1
; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v2, off, off ; 4-byte Folded Reload
; HSA-TRAP-GFX1100-O0-NEXT: s_mov_b32 exec_lo, s6
; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt vmcnt(0)
; HSA-TRAP-GFX1100-O0-NEXT: v_readlane_b32 s0, v2, 0
; HSA-TRAP-GFX1100-O0-NEXT: v_readlane_b32 s1, v2, 1
-; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v0, off, off offset:8 ; 4-byte Folded Reload
-; HSA-TRAP-GFX1100-O0-NEXT: scratch_load_b32 v1, off, off offset:4 ; 4-byte Folded Reload
-; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt vmcnt(0)
; HSA-TRAP-GFX1100-O0-NEXT: global_store_b32 v0, v1, s[0:1] dlc
; HSA-TRAP-GFX1100-O0-NEXT: s_waitcnt_vscnt null, 0x0
; HSA-TRAP-GFX1100-O0-NEXT: s_endpgm
diff --git a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
index c70327aaea993..ff77abc71301b 100644
--- a/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
+++ b/llvm/test/CodeGen/AMDGPU/undefined-physreg-sgpr-spill.mir
@@ -68,8 +68,8 @@ body: |
successors:
liveins: $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr13
- $sgpr0_sgpr1 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $sgpr0_sgpr1 = bb-prolog SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
bb.3:
liveins: $vgpr0, $vgpr1, $vgpr2, $sgpr2_sgpr3, $sgpr4_sgpr5
@@ -133,8 +133,8 @@ body: |
successors:
liveins: $sgpr8_sgpr9_sgpr10_sgpr11, $sgpr13
- $sgpr0_sgpr1 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
- $exec = S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
+ $sgpr0_sgpr1 = bb-prolog SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr8_sgpr9_sgpr10_sgpr11, implicit $sgpr13, implicit-def dead $m0 :: (load (s64) from %stack.0, align 4, addrspace 5)
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr0_sgpr1, implicit-def $scc
bb.3:
liveins: $vgpr0, $vgpr1, $vgpr2, $sgpr2_sgpr3, $sgpr4_sgpr5
diff --git a/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir b/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
index 0192da03a3345..5c54c9c805b17 100644
--- a/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
+++ b/llvm/test/CodeGen/AMDGPU/v-cmp-cse-across-loop.mir
@@ -57,7 +57,7 @@ body: |
bb.2:
successors: %bb.3(0x80000000)
- SI_END_CF %4:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
+ bb-prolog SI_END_CF %4:sreg_64, implicit-def dead $exec, implicit-def dead $scc, implicit $exec
%5:sreg_64 = V_CMP_GT_U32_e64 %0:vgpr_32, %1:sreg_32, implicit $exec
%6:sreg_64 = S_AND_B64 %5:sreg_64, $exec, implicit-def $scc
S_BRANCH %bb.3
diff --git a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
index 323d898b5b2d9..49da0b742534c 100644
--- a/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
+++ b/llvm/test/CodeGen/AMDGPU/vgpr-mark-last-scratch-load.mir
@@ -155,7 +155,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.5(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %mask, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask, implicit-def $scc
; CHECK-NEXT: S_BRANCH %bb.5
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.3:
@@ -202,7 +202,7 @@ body: |
; CHECK-NEXT: bb.4:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, %mask2, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask2, implicit-def $scc
; CHECK-NEXT: SI_SPILL_V32_SAVE [[V_MOV_B32_e32_4]], %stack.16, $sgpr32, 0, implicit $exec :: (store (s32) into %stack.16, addrspace 5)
; CHECK-NEXT: S_BRANCH %bb.2
; CHECK-NEXT: {{ $}}
@@ -270,7 +270,7 @@ body: |
bb.2: ; flow
successors: %bb.5(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %mask, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask, implicit-def $scc
S_BRANCH %bb.5
bb.3: ; loop
@@ -298,7 +298,7 @@ body: |
bb.4: ; flow
successors: %bb.2(0x80000000)
- $exec_lo = S_OR_B32 $exec_lo, %mask2, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %mask2, implicit-def $scc
S_BRANCH %bb.2
bb.5: ; exit
diff --git a/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir b/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
index 4ee427ad51390..f76ed27a6dbb8 100644
--- a/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
+++ b/llvm/test/CodeGen/AMDGPU/waitcnt-kmcnt-scc-different-block.mir
@@ -150,7 +150,7 @@ body: |
S_CBRANCH_EXECZ %bb.4, implicit $exec
bb.1:
- V_CMPX_EQ_U32_nosdst_e32 0, killed $vgpr1, implicit-def $exec, implicit $exec
+ bb-prolog V_CMPX_EQ_U32_nosdst_e32 0, killed $vgpr1, implicit-def $exec, implicit $exec
S_CBRANCH_EXECZ %bb.3, implicit $exec
bb.2:
diff --git a/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll b/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
index 51303f11a8f47..427577f21ca94 100644
--- a/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
+++ b/llvm/test/CodeGen/AMDGPU/waterfall-call-target-av-register-failure.ll
@@ -58,20 +58,21 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
; CHECK-NEXT: .LBB0_2: ; =>This Inner Loop Header: Depth=1
+; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
+; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b64 s[34:35], -1
; CHECK-NEXT: buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
-; CHECK-NEXT: buffer_load_dword v0, off, s[0:3], s33 offset:4 ; 4-byte Folded Reload
-; CHECK-NEXT: buffer_load_dword v1, off, s[0:3], s33 offset:8 ; 4-byte Folded Reload
-; CHECK-NEXT: s_waitcnt vmcnt(1)
+; CHECK-NEXT: s_waitcnt vmcnt(2)
; CHECK-NEXT: v_readfirstlane_b32 s6, v0
-; CHECK-NEXT: s_waitcnt vmcnt(0)
+; CHECK-NEXT: s_waitcnt vmcnt(1)
; CHECK-NEXT: v_readfirstlane_b32 s8, v1
; CHECK-NEXT: s_mov_b32 s4, s6
; CHECK-NEXT: s_mov_b32 s5, s8
; CHECK-NEXT: v_cmp_eq_u64_e64 s[4:5], s[4:5], v[0:1]
; CHECK-NEXT: ; kill: def $sgpr6 killed $sgpr6 def $sgpr6_sgpr7
; CHECK-NEXT: s_mov_b32 s7, s8
+; CHECK-NEXT: s_waitcnt vmcnt(0)
; CHECK-NEXT: v_writelane_b32 v41, s6, 14
; CHECK-NEXT: v_writelane_b32 v41, s7, 15
; CHECK-NEXT: s_and_saveexec_b64 s[4:5], s[4:5]
@@ -81,6 +82,7 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: buffer_store_dword v41, off, s[0:3], s33 ; 4-byte Folded Spill
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
; CHECK-NEXT: ; %bb.3: ; in Loop: Header=BB0_2 Depth=1
+; CHECK-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: s_or_saveexec_b64 s[34:35], -1
; CHECK-NEXT: buffer_load_dword v41, off, s[0:3], s33 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 exec, s[34:35]
@@ -99,7 +101,6 @@ define i32 @fix_sgpr_copies_indirect_call(ptr addrspace(5) %ptr) #0 {
; CHECK-NEXT: v_readlane_b32 s7, v41, 9
; CHECK-NEXT: v_readlane_b32 s4, v41, 10
; CHECK-NEXT: v_readlane_b32 s5, v41, 11
-; CHECK-NEXT: buffer_load_dword v31, off, s[0:3], s33 offset:12 ; 4-byte Folded Reload
; CHECK-NEXT: s_mov_b64 s[22:23], s[2:3]
; CHECK-NEXT: s_mov_b64 s[20:21], s[0:1]
; CHECK-NEXT: s_mov_b64 s[0:1], s[20:21]
diff --git a/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir b/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
index 5f3b543acf6a6..2cdebeb9075c1 100644
--- a/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
+++ b/llvm/test/CodeGen/AMDGPU/whole-wave-functions-pei.mir
@@ -479,7 +479,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: liveins: $vcc_lo, $sgpr1, $vgpr0, $vgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
; CHECK-NEXT: renamable $vgpr0 = V_CNDMASK_B32_e64 0, $vgpr1, 0, $vgpr0, $vcc_lo, implicit $exec
; CHECK-NEXT: $exec_lo = S_XOR_B32 $vcc_lo, -1, implicit-def $scc
; CHECK-NEXT: $vgpr0 = SCRATCH_LOAD_DWORD_SADDR $sgpr32, 0, 0, implicit $exec, implicit $flat_scr, implicit $vgpr0(tied-def 0) :: ("amdgpu-thread-private" load (s32) from %stack.0, addrspace 5)
@@ -490,7 +490,7 @@ body: |
successors: %bb.1, %bb.2
liveins: $vgpr0, $vgpr1
- renamable $vcc_lo = SI_WHOLE_WAVE_FUNC_SETUP implicit-def dead $exec, implicit $exec
+ renamable $vcc_lo = bb-prolog SI_WHOLE_WAVE_FUNC_SETUP implicit-def dead $exec, implicit $exec
$sgpr1 = S_MOV_B32 $exec_lo
V_CMPX_EQ_U32_nosdst_e64 $vgpr0, $vgpr1, implicit-def $exec, implicit $exec
S_CBRANCH_EXECZ %bb.2, implicit $exec
@@ -504,7 +504,7 @@ body: |
bb.2:
liveins: $vcc_lo, $sgpr1, $vgpr0, $vgpr1
- $exec_lo = S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, killed renamable $sgpr1, implicit-def $scc
renamable $vgpr0 = V_CNDMASK_B32_e64 0, $vgpr1, 0, $vgpr0, $vcc_lo, implicit $exec
SI_WHOLE_WAVE_FUNC_RETURN killed renamable $vcc_lo, implicit $vgpr0
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
index 73e910e75fc19..1b80c82ae6713 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr-terminator.mir
@@ -45,8 +45,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $m0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
- ; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr1
; CHECK-NEXT: [[DEF:%[0-9]+]]:sgpr_256 = IMPLICIT_DEF
@@ -58,7 +58,7 @@ body: |
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.2(0x40000000)
; CHECK-NEXT: liveins: $m0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WQM:%[0-9]+]]:sreg_32 = ENTER_STRICT_WQM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WQM:%[0-9]+]]:sreg_32 = bb-prolog ENTER_STRICT_WQM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[DS_PARAM_LOAD:%[0-9]+]]:vgpr_32 = DS_PARAM_LOAD 0, 0, 0, 1, implicit $m0, implicit $exec
; CHECK-NEXT: $exec_lo = EXIT_STRICT_WQM [[ENTER_STRICT_WQM]]
; CHECK-NEXT: DBG_VALUE $noreg, $noreg
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
index 604ab24bf02bc..f66400b270418 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-debug-instr.mir
@@ -67,7 +67,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir b/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
index d77704857bba5..9b512c6220c36 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-propagate-for-execz-side-effect.mir
@@ -24,8 +24,8 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x40000000), %bb.8(0x40000000)
; CHECK-NEXT: liveins: $vgpr0, $vgpr1, $vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
- ; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr2
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vgpr_32 = COPY $vgpr1
; CHECK-NEXT: [[COPY3:%[0-9]+]]:vgpr_32 = COPY $vgpr0
@@ -79,7 +79,7 @@ body: |
; CHECK-NEXT: bb.2:
; CHECK-NEXT: successors: %bb.3(0x40000000), %bb.7(0x40000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_OR_SAVEEXEC_B32_:%[0-9]+]]:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 [[S_XOR_B32_]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[S_AND_B32_2:%[0-9]+]]:sreg_32 = S_AND_B32 $exec_lo, [[S_OR_SAVEEXEC_B32_]], implicit-def $scc
; CHECK-NEXT: $exec_lo = S_XOR_B32_term $exec_lo, [[S_AND_B32_2]], implicit-def $scc
; CHECK-NEXT: S_CBRANCH_EXECZ %bb.7, implicit $exec
@@ -110,7 +110,7 @@ body: |
; CHECK-NEXT: bb.5:
; CHECK-NEXT: successors: %bb.2(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[S_AND_SAVEEXEC_B32_1:%[0-9]+]]:sreg_32 = S_AND_SAVEEXEC_B32 [[COPY]], implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[S_AND_SAVEEXEC_B32_1:%[0-9]+]]:sreg_32 = bb-prolog S_AND_SAVEEXEC_B32 [[COPY]], implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub1:vreg_128 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub2:vreg_128 = COPY [[V_OR_B16_t16_e64_]].sub1
; CHECK-NEXT: [[V_OR_B16_t16_e64_:%[0-9]+]].sub3:vreg_128 = COPY [[V_OR_B16_t16_e64_]].sub1
@@ -123,16 +123,16 @@ body: |
; CHECK-NEXT: bb.6:
; CHECK-NEXT: successors: %bb.7(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY7]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY7]], implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.7:
; CHECK-NEXT: successors: %bb.8(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[S_AND_B32_2]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[S_AND_B32_2]], implicit-def $scc
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.8:
- ; CHECK-NEXT: $exec_lo = S_OR_B32 $exec_lo, [[COPY4]], implicit-def $scc
- ; CHECK-NEXT: $exec_lo = S_AND_B32 $exec_lo, [[COPY]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_OR_B32 $exec_lo, [[COPY4]], implicit-def $scc
+ ; CHECK-NEXT: $exec_lo = bb-prolog S_AND_B32 $exec_lo, [[COPY]], implicit-def $scc
; CHECK-NEXT: $vgpr0 = COPY [[V_MOV_B32_e32_]]
; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
bb.0:
@@ -190,7 +190,7 @@ body: |
bb.2:
successors: %bb.3, %bb.7
- %24:sreg_32 = S_OR_SAVEEXEC_B32 %23, implicit-def $exec, implicit-def $scc, implicit $exec
+ %24:sreg_32 = bb-prolog S_OR_SAVEEXEC_B32 %23, implicit-def $exec, implicit-def $scc, implicit $exec
%25:sreg_32 = S_AND_B32 $exec_lo, %24, implicit-def $scc
$exec_lo = S_XOR_B32_term $exec_lo, %25, implicit-def $scc
S_CBRANCH_EXECZ %bb.7, implicit $exec
@@ -226,13 +226,13 @@ body: |
S_BRANCH %bb.2
bb.6:
- $exec_lo = S_OR_B32 $exec_lo, %29, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %29, implicit-def $scc
bb.7:
- $exec_lo = S_OR_B32 $exec_lo, %25, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %25, implicit-def $scc
bb.8:
- $exec_lo = S_OR_B32 $exec_lo, %6, implicit-def $scc
+ $exec_lo = bb-prolog S_OR_B32 $exec_lo, %6, implicit-def $scc
$vgpr0 = COPY %5
SI_RETURN_TO_EPILOG $vgpr0
...
diff --git a/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir b/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
index c8b0c4ef68623..b15842e59f31a 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm-terminators.mir
@@ -28,7 +28,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr0, $vgpr0_vgpr1
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = COPY $exec_lo
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_32 = bb-prolog COPY $exec_lo
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr0
; CHECK-NEXT: $exec_lo = S_WQM_B32 $exec_lo, implicit-def $scc
; CHECK-NEXT: [[COPY2:%[0-9]+]]:vreg_64 = COPY $vgpr0_vgpr1
diff --git a/llvm/test/CodeGen/AMDGPU/wqm.mir b/llvm/test/CodeGen/AMDGPU/wqm.mir
index 61acddb6b170b..d4d1e2f65c6ca 100644
--- a/llvm/test/CodeGen/AMDGPU/wqm.mir
+++ b/llvm/test/CodeGen/AMDGPU/wqm.mir
@@ -6,6 +6,9 @@
define amdgpu_ps void @test_strict_wwm_scc() {
ret void
}
+ define amdgpu_ps void @test_strict_wwm_pseudo_probe() {
+ ret void
+ }
define amdgpu_ps void @test_strict_wwm_scc2() {
ret void
}
@@ -83,7 +86,7 @@ body: |
; CHECK-LABEL: name: test_strict_wwm_scc
; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
; CHECK-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY $sgpr1
@@ -108,6 +111,68 @@ body: |
...
+---
+# A pseudo probe at the block entry does not separate the exec mode switch from
+# the block start: it is still a block prolog instruction.
+name: test_strict_wwm_pseudo_probe
+alignment: 1
+exposesReturnsTwice: false
+legalized: false
+regBankSelected: false
+selected: false
+tracksRegLiveness: true
+registers:
+ - { id: 0, class: sgpr_32, preferred-register: '' }
+ - { id: 1, class: sgpr_32, preferred-register: '' }
+ - { id: 2, class: sgpr_32, preferred-register: '' }
+ - { id: 3, class: vgpr_32, preferred-register: '' }
+ - { id: 4, class: vgpr_32, preferred-register: '' }
+ - { id: 5, class: sgpr_32, preferred-register: '' }
+ - { id: 6, class: vgpr_32, preferred-register: '' }
+ - { id: 7, class: vgpr_32, preferred-register: '' }
+ - { id: 8, class: sreg_32_xm0, preferred-register: '' }
+ - { id: 9, class: sreg_32, preferred-register: '' }
+ - { id: 10, class: sreg_32, preferred-register: '' }
+ - { id: 11, class: vgpr_32, preferred-register: '' }
+ - { id: 12, class: vgpr_32, preferred-register: '' }
+liveins:
+ - { reg: '$sgpr0', virtual-reg: '%0' }
+ - { reg: '$sgpr1', virtual-reg: '%1' }
+ - { reg: '$sgpr2', virtual-reg: '%2' }
+ - { reg: '$vgpr0', virtual-reg: '%3' }
+body: |
+ bb.0:
+ liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
+
+ ; CHECK-LABEL: name: test_strict_wwm_pseudo_probe
+ ; CHECK: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: PSEUDO_PROBE 1, 1, 0, 0
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
+ ; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
+ ; CHECK-NEXT: [[COPY2:%[0-9]+]]:sgpr_32 = COPY $sgpr1
+ ; CHECK-NEXT: [[COPY3:%[0-9]+]]:sgpr_32 = COPY $sgpr0
+ ; CHECK-NEXT: S_CMP_LT_I32 0, [[COPY3]], implicit-def $scc
+ ; CHECK-NEXT: [[V_ADD_CO_U32_e32_:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[COPY]], [[COPY]], implicit-def $vcc, implicit $exec
+ ; CHECK-NEXT: [[S_CSELECT_B32_:%[0-9]+]]:sgpr_32 = S_CSELECT_B32 [[COPY1]], [[COPY2]], implicit $scc
+ ; CHECK-NEXT: [[V_ADD_CO_U32_e32_1:%[0-9]+]]:vgpr_32 = V_ADD_CO_U32_e32 [[S_CSELECT_B32_]], [[V_ADD_CO_U32_e32_]], implicit-def $vcc, implicit $exec
+ ; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
+ ; CHECK-NEXT: early-clobber $vgpr0 = V_MOV_B32_e32 [[V_ADD_CO_U32_e32_1]], implicit $exec
+ ; CHECK-NEXT: SI_RETURN_TO_EPILOG $vgpr0
+ PSEUDO_PROBE 1, 1, 0, 0
+ %3 = COPY $vgpr0
+ %2 = COPY $sgpr2
+ %1 = COPY $sgpr1
+ %0 = COPY $sgpr0
+ S_CMP_LT_I32 0, %0, implicit-def $scc
+ %12 = V_ADD_CO_U32_e32 %3, %3, implicit-def $vcc, implicit $exec
+ %5 = S_CSELECT_B32 %2, %1, implicit $scc
+ %11 = V_ADD_CO_U32_e32 %5, %12, implicit-def $vcc, implicit $exec
+ $vgpr0 = STRICT_WWM %11, implicit $exec
+ SI_RETURN_TO_EPILOG $vgpr0
+
+...
---
# Second test for awareness that s_or_saveexec_b64 clobbers SCC
# Because entry block is treated differently.
@@ -119,7 +184,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $sgpr0, $sgpr1, $sgpr2, $vgpr0
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ ; CHECK-NEXT: [[ENTER_STRICT_WWM:%[0-9]+]]:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
; CHECK-NEXT: [[COPY:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: $exec = EXIT_STRICT_WWM [[ENTER_STRICT_WWM]]
; CHECK-NEXT: [[COPY1:%[0-9]+]]:sgpr_32 = COPY $sgpr2
@@ -301,7 +366,7 @@ body: |
; CHECK-NEXT: successors: %bb.2(0x40000000), %bb.1(0x40000000)
; CHECK-NEXT: liveins: $sgpr1, $sgpr2, $vgpr1, $vgpr2
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = COPY $exec
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = bb-prolog COPY $exec
; CHECK-NEXT: $m0 = COPY $sgpr1
; CHECK-NEXT: $exec = S_WQM_B64 $exec, implicit-def $scc
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr1
@@ -420,8 +485,8 @@ body: |
; CHECK: bb.0:
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = COPY $exec
- ; CHECK-NEXT: $exec = S_WQM_B64 $exec, implicit-def $scc
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = bb-prolog COPY $exec
+ ; CHECK-NEXT: $exec = bb-prolog S_WQM_B64 $exec, implicit-def $scc
; CHECK-NEXT: undef [[V_MOV_B32_e32_:%[0-9]+]].sub0:vreg_64 = V_MOV_B32_e32 -10, implicit $exec
; CHECK-NEXT: [[V_MOV_B32_e32_:%[0-9]+]].sub1:vreg_64 = V_MOV_B32_e32 0, implicit $exec
; CHECK-NEXT: [[S_GETPC_B64_:%[0-9]+]]:sreg_64 = S_GETPC_B64
@@ -447,7 +512,7 @@ body: |
; CHECK-NEXT: S_BRANCH %bb.4
; CHECK-NEXT: {{ $}}
; CHECK-NEXT: bb.4:
- ; CHECK-NEXT: $exec = S_AND_B64 $exec, [[COPY]], implicit-def $scc
+ ; CHECK-NEXT: $exec = bb-prolog S_AND_B64 $exec, [[COPY]], implicit-def $scc
; CHECK-NEXT: [[DEF:%[0-9]+]]:sgpr_128 = IMPLICIT_DEF
; CHECK-NEXT: [[IMAGE_SAMPLE_V4_V2_:%[0-9]+]]:vreg_128 = IMAGE_SAMPLE_V4_V2 [[V_MOV_B32_e32_]], [[S_LOAD_DWORDX8_IMM]], [[DEF]], 15, 0, 0, 0, 0, 0, 0, 0, implicit $exec :: (dereferenceable load (s128), addrspace 7)
; CHECK-NEXT: $vgpr0 = COPY [[IMAGE_SAMPLE_V4_V2_]].sub0
@@ -629,7 +694,7 @@ body: |
; CHECK-NEXT: successors: %bb.1(0x80000000)
; CHECK-NEXT: liveins: $vgpr0, $vcc
; CHECK-NEXT: {{ $}}
- ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = COPY $exec
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:sreg_64 = bb-prolog COPY $exec
; CHECK-NEXT: [[COPY1:%[0-9]+]]:vgpr_32 = COPY $vgpr0
; CHECK-NEXT: [[COPY2:%[0-9]+]]:sreg_64 = COPY $vcc
; CHECK-NEXT: V_CMP_NGT_F32_e32 0, [[COPY1]], implicit-def $vcc, implicit $mode, implicit $exec
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir b/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
index 8647c6314c88e..87874efd16a68 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
+++ b/llvm/test/CodeGen/AMDGPU/wwm-regalloc-preallocation-guard.mir
@@ -30,8 +30,8 @@ machineFunctionInfo:
body: |
bb.0:
liveins: $sgpr34, $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99
- %0:sreg_64 = ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
- $exec = EXIT_STRICT_WWM killed %0
+ %0:sreg_64 = bb-prolog ENTER_STRICT_WWM -1, implicit-def $exec, implicit-def $scc, implicit $exec
+ $exec = bb-prolog EXIT_STRICT_WWM killed %0
SI_SPILL_S1024_SAVE killed $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67, %stack.0, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
SI_SPILL_S1024_SAVE killed $sgpr68_sgpr69_sgpr70_sgpr71_sgpr72_sgpr73_sgpr74_sgpr75_sgpr76_sgpr77_sgpr78_sgpr79_sgpr80_sgpr81_sgpr82_sgpr83_sgpr84_sgpr85_sgpr86_sgpr87_sgpr88_sgpr89_sgpr90_sgpr91_sgpr92_sgpr93_sgpr94_sgpr95_sgpr96_sgpr97_sgpr98_sgpr99, %stack.1, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
SI_SPILL_S32_SAVE killed $sgpr34, %stack.2, implicit $exec, implicit $sgpr0_sgpr1_sgpr2_sgpr3, implicit $sgpr32
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll b/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
index 9989907f4b9b9..dbefc926c1510 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
+++ b/llvm/test/CodeGen/AMDGPU/wwm-reserved-spill.ll
@@ -212,12 +212,12 @@ define amdgpu_gfx void @strict_wwm_cfg(ptr addrspace(8) inreg %tmp14, i32 %arg)
; GFX9-O0-NEXT: v_readlane_b32 s36, v5, 4
; GFX9-O0-NEXT: v_readlane_b32 s37, v5, 5
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[36:37]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s38, v5, 0
; GFX9-O0-NEXT: v_readlane_b32 s39, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s34, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s35, v5, 3
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[36:37], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[36:37]
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll b/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
index 0ef5725607920..c14cd487c7d56 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
+++ b/llvm/test/CodeGen/AMDGPU/wwm-reserved.ll
@@ -189,12 +189,12 @@ define amdgpu_cs void @cfg(ptr addrspace(8) inreg %tmp14, i32 %arg) {
; GFX9-O0-NEXT: v_readlane_b32 s4, v5, 5
; GFX9-O0-NEXT: v_readlane_b32 s5, v5, 6
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[4:5]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s2, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s3, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s0, v5, 3
; GFX9-O0-NEXT: v_readlane_b32 s1, v5, 4
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[4:5]
@@ -1052,12 +1052,12 @@ define amdgpu_cs void @strict_wwm_cfg(ptr addrspace(8) inreg %tmp14, i32 %arg) {
; GFX9-O0-NEXT: v_readlane_b32 s4, v5, 5
; GFX9-O0-NEXT: v_readlane_b32 s5, v5, 6
; GFX9-O0-NEXT: s_or_b64 exec, exec, s[4:5]
+; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
+; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: v_readlane_b32 s2, v5, 1
; GFX9-O0-NEXT: v_readlane_b32 s3, v5, 2
; GFX9-O0-NEXT: v_readlane_b32 s0, v5, 3
; GFX9-O0-NEXT: v_readlane_b32 s1, v5, 4
-; GFX9-O0-NEXT: buffer_load_dword v0, off, s[16:19], 0 offset:8 ; 4-byte Folded Reload
-; GFX9-O0-NEXT: buffer_load_dword v3, off, s[16:19], 0 offset:4 ; 4-byte Folded Reload
; GFX9-O0-NEXT: s_waitcnt vmcnt(0)
; GFX9-O0-NEXT: v_cmp_eq_u32_e64 s[4:5], v0, v3
; GFX9-O0-NEXT: v_cndmask_b32_e64 v0, 0, -1, s[4:5]
diff --git a/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir b/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
index ce8aebe7458ed..3d14dcf053af7 100644
--- a/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
+++ b/llvm/test/CodeGen/AMDGPU/wwm-spill-superclass-pseudo.mir
@@ -73,7 +73,7 @@ body: |
bb.7:
liveins: $sgpr60_sgpr61
- $exec = S_OR_B64 $exec, killed $sgpr60_sgpr61, implicit-def $scc
+ $exec = bb-prolog S_OR_B64 $exec, killed $sgpr60_sgpr61, implicit-def $scc
$sgpr2_sgpr3 = SI_SPILL_S64_RESTORE %stack.0, implicit $exec, implicit $sgpr32
S_BRANCH %bb.5
>From b85f5f8dfe97ed5e30d7c1213bbee311065cd29c Mon Sep 17 00:00:00 2001
From: Gabriel Baraldi <baraldigabriel at gmail.com>
Date: Tue, 22 Sep 2026 15:24:18 +0000
Subject: [PATCH 3/3] [AMDGPU] Regenerate amdgcn.bitcast tests for the BBProlog
flag
Register assignment shifts because sgpr-regalloc split copies now follow
the exec restore in Flow blocks. Pure regeneration with
update_llc_test_checks.py; s_or_saveexec/s_andn2_saveexec counts are
unchanged.
Co-Authored-By: Claude Fable 5.1 <noreply at anthropic.com>
---
.../CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll | 13540 ++++++++--------
.../CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll | 2023 +--
.../CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll | 684 +-
.../CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll | 1892 ++-
.../CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll | 2811 ++--
.../CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll | 4860 +++---
.../CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll | 6734 ++++----
.../CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll | 7506 +++++----
8 files changed, 20982 insertions(+), 19068 deletions(-)
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
index 88ef88da73bce..fef27c52cbbcb 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
@@ -22760,139 +22760,139 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: v_writelane_b32 v33, s55, 13
; SI-NEXT: v_writelane_b32 v33, s64, 14
; SI-NEXT: v_writelane_b32 v33, s65, 15
-; SI-NEXT: v_writelane_b32 v33, s66, 16
-; SI-NEXT: v_writelane_b32 v33, s67, 17
-; SI-NEXT: v_writelane_b32 v33, s68, 18
-; SI-NEXT: v_writelane_b32 v33, s69, 19
-; SI-NEXT: v_writelane_b32 v33, s70, 20
-; SI-NEXT: v_writelane_b32 v33, s71, 21
-; SI-NEXT: v_writelane_b32 v33, s80, 22
-; SI-NEXT: v_writelane_b32 v33, s81, 23
-; SI-NEXT: v_writelane_b32 v33, s82, 24
-; SI-NEXT: v_writelane_b32 v33, s83, 25
-; SI-NEXT: v_writelane_b32 v33, s84, 26
-; SI-NEXT: v_writelane_b32 v33, s85, 27
-; SI-NEXT: v_writelane_b32 v33, s86, 28
-; SI-NEXT: v_writelane_b32 v33, s87, 29
-; SI-NEXT: v_writelane_b32 v33, s96, 30
-; SI-NEXT: v_writelane_b32 v33, s97, 31
-; SI-NEXT: v_writelane_b32 v33, s98, 32
-; SI-NEXT: v_writelane_b32 v33, s30, 33
-; SI-NEXT: v_writelane_b32 v33, s31, 34
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_readfirstlane_b32 s4, v17
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: v_readfirstlane_b32 s31, v15
-; SI-NEXT: v_readfirstlane_b32 s34, v14
-; SI-NEXT: v_readfirstlane_b32 s35, v13
-; SI-NEXT: v_readfirstlane_b32 s36, v12
-; SI-NEXT: v_readfirstlane_b32 s37, v11
-; SI-NEXT: v_readfirstlane_b32 s38, v10
-; SI-NEXT: v_readfirstlane_b32 s39, v9
-; SI-NEXT: v_readfirstlane_b32 s48, v8
-; SI-NEXT: v_readfirstlane_b32 s49, v7
-; SI-NEXT: v_readfirstlane_b32 s50, v6
-; SI-NEXT: v_readfirstlane_b32 s51, v5
-; SI-NEXT: v_readfirstlane_b32 s52, v4
-; SI-NEXT: v_readfirstlane_b32 s53, v3
-; SI-NEXT: v_readfirstlane_b32 s54, v2
-; SI-NEXT: v_readfirstlane_b32 s55, v1
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_readfirstlane_b32 s64, v0
+; SI-NEXT: v_writelane_b32 v33, s30, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 17
+; SI-NEXT: v_readfirstlane_b32 s7, v17
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s9, v16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
+; SI-NEXT: v_writelane_b32 v34, s7, 0
+; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: v_writelane_b32 v34, s9, 1
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v34, s11, 2
+; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v34, s13, 3
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_writelane_b32 v34, s15, 4
+; SI-NEXT: v_readfirstlane_b32 s43, v11
+; SI-NEXT: v_writelane_b32 v34, s41, 5
+; SI-NEXT: v_readfirstlane_b32 s45, v10
+; SI-NEXT: v_writelane_b32 v34, s43, 6
+; SI-NEXT: v_readfirstlane_b32 s47, v9
+; SI-NEXT: v_writelane_b32 v34, s45, 7
+; SI-NEXT: v_readfirstlane_b32 s57, v8
+; SI-NEXT: v_writelane_b32 v34, s47, 8
+; SI-NEXT: v_readfirstlane_b32 s59, v7
+; SI-NEXT: v_writelane_b32 v34, s57, 9
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_writelane_b32 v34, s59, 10
+; SI-NEXT: v_readfirstlane_b32 s63, v5
+; SI-NEXT: v_writelane_b32 v34, s61, 11
+; SI-NEXT: v_readfirstlane_b32 s73, v4
+; SI-NEXT: v_writelane_b32 v34, s63, 12
+; SI-NEXT: v_readfirstlane_b32 s6, v3
+; SI-NEXT: v_writelane_b32 v34, s73, 13
+; SI-NEXT: v_readfirstlane_b32 s5, v2
+; SI-NEXT: v_writelane_b32 v34, s6, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s5, 15
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: v_writelane_b32 v34, s4, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 17
; SI-NEXT: s_cbranch_scc0 .LBB17_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s4, 16
-; SI-NEXT: s_and_b32 s4, s50, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 2
-; SI-NEXT: s_lshl_b32 s4, s50, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 1
-; SI-NEXT: s_and_b32 s4, s51, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 4
-; SI-NEXT: s_lshl_b32 s4, s51, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 3
-; SI-NEXT: s_and_b32 s4, s52, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 6
-; SI-NEXT: s_lshl_b32 s4, s52, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 5
-; SI-NEXT: s_and_b32 s4, s53, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 8
-; SI-NEXT: s_lshl_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 7
-; SI-NEXT: s_and_b32 s4, s55, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 10
-; SI-NEXT: s_lshl_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 9
-; SI-NEXT: s_and_b32 s4, s64, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 12
-; SI-NEXT: s_lshl_b32 s4, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 11
-; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 14
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 13
-; SI-NEXT: s_and_b32 s4, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 15
+; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s4, 16
; SI-NEXT: s_and_b32 s4, s27, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 18
+; SI-NEXT: v_writelane_b32 v34, s4, 19
; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 17
+; SI-NEXT: v_writelane_b32 v34, s4, 18
; SI-NEXT: s_and_b32 s4, s26, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: v_writelane_b32 v34, s4, 21
; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: s_and_b32 s10, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s30, 16
-; SI-NEXT: s_and_b32 s12, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_and_b32 s14, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s34, 16
-; SI-NEXT: s_and_b32 s40, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s35, 16
-; SI-NEXT: s_and_b32 s42, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s36, 16
-; SI-NEXT: s_and_b32 s44, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s37, 16
-; SI-NEXT: s_and_b32 s46, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s38, 16
-; SI-NEXT: s_and_b32 s56, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s39, 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s48, 16
-; SI-NEXT: s_and_b32 s60, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s49, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 19
-; SI-NEXT: s_and_b32 s62, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s25, 16
-; SI-NEXT: s_and_b32 s72, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s24, 16
-; SI-NEXT: s_and_b32 s74, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s23, 16
-; SI-NEXT: s_and_b32 s76, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s75, s22, 16
-; SI-NEXT: s_and_b32 s88, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s21, 16
-; SI-NEXT: s_and_b32 s90, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s20, 16
-; SI-NEXT: s_and_b32 s92, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s19, 16
-; SI-NEXT: s_and_b32 s94, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s18, 16
-; SI-NEXT: s_and_b32 vcc_lo, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s95, s17, 16
-; SI-NEXT: s_and_b32 s6, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s16, 16
+; SI-NEXT: s_and_b32 s55, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_and_b32 s76, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s75, s6, 16
+; SI-NEXT: s_and_b32 s78, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s5, 16
+; SI-NEXT: s_and_b32 s90, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s8, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s6, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s8, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s16, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB17_3
; SI-NEXT: .LBB17_2:
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr95
@@ -22905,6 +22905,8 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr73
@@ -22913,8 +22915,6 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $sgpr57
@@ -22936,39 +22936,7 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
; SI-NEXT: ; implicit-def: $sgpr4
@@ -22980,70 +22948,79 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s65, s63
-; SI-NEXT: s_mov_b32 s66, s72
-; SI-NEXT: s_mov_b32 s67, s73
-; SI-NEXT: s_mov_b32 s68, s74
-; SI-NEXT: s_mov_b32 s69, s75
-; SI-NEXT: s_mov_b32 s70, s76
-; SI-NEXT: s_mov_b32 s71, s79
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s81, s89
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s83, s91
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s85, s93
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s87, s95
-; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s61
-; SI-NEXT: s_mov_b32 s97, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_hi, s62
-; SI-NEXT: s_mov_b32 s98, s6
+; SI-NEXT: s_mov_b32 s64, s30
+; SI-NEXT: s_mov_b32 s30, s6
+; SI-NEXT: s_mov_b32 s65, s8
; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s8
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s9, s11
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s11, s13
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s13, s15
-; SI-NEXT: s_mov_b32 s14, s40
-; SI-NEXT: s_mov_b32 s15, s41
-; SI-NEXT: s_mov_b32 s40, s42
-; SI-NEXT: s_mov_b32 s41, s43
-; SI-NEXT: s_mov_b32 s42, s44
-; SI-NEXT: s_mov_b32 s43, s45
-; SI-NEXT: s_mov_b32 s44, s46
-; SI-NEXT: s_mov_b32 s45, s47
-; SI-NEXT: s_mov_b32 s46, s56
-; SI-NEXT: s_mov_b32 s47, s57
-; SI-NEXT: s_mov_b32 s56, s58
-; SI-NEXT: s_mov_b32 s57, s59
-; SI-NEXT: s_mov_b32 s58, s60
-; SI-NEXT: v_readlane_b32 s59, v34, 1
-; SI-NEXT: v_readlane_b32 s60, v34, 2
-; SI-NEXT: v_readlane_b32 s61, v34, 3
-; SI-NEXT: v_readlane_b32 s62, v34, 4
-; SI-NEXT: v_readlane_b32 s63, v34, 5
-; SI-NEXT: v_readlane_b32 s72, v34, 6
-; SI-NEXT: v_readlane_b32 s73, v34, 7
-; SI-NEXT: v_readlane_b32 s74, v34, 8
-; SI-NEXT: v_readlane_b32 s6, v34, 9
-; SI-NEXT: v_readlane_b32 s75, v34, 10
-; SI-NEXT: v_readlane_b32 s76, v34, 11
-; SI-NEXT: v_readlane_b32 s79, v34, 12
-; SI-NEXT: v_readlane_b32 s88, v34, 13
-; SI-NEXT: v_readlane_b32 s89, v34, 14
-; SI-NEXT: v_readlane_b32 s90, v34, 15
-; SI-NEXT: v_readlane_b32 s91, v34, 16
-; SI-NEXT: v_readlane_b32 s92, v34, 17
-; SI-NEXT: v_readlane_b32 s93, v34, 18
-; SI-NEXT: v_readlane_b32 s94, v34, 19
-; SI-NEXT: v_readlane_b32 s95, v34, 20
+; SI-NEXT: s_mov_b32 s5, s55
+; SI-NEXT: s_mov_b32 s55, s9
+; SI-NEXT: s_mov_b32 s9, s10
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s12
+; SI-NEXT: s_mov_b32 s12, s13
+; SI-NEXT: s_mov_b32 s13, s14
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s40, s41
+; SI-NEXT: s_mov_b32 s41, s42
+; SI-NEXT: s_mov_b32 s42, s43
+; SI-NEXT: s_mov_b32 s43, s44
+; SI-NEXT: s_mov_b32 s44, s45
+; SI-NEXT: s_mov_b32 s45, s46
+; SI-NEXT: s_mov_b32 s46, s47
+; SI-NEXT: s_mov_b32 s47, s56
+; SI-NEXT: s_mov_b32 s56, s57
+; SI-NEXT: s_mov_b32 s57, s58
+; SI-NEXT: s_mov_b32 s58, s59
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s61
+; SI-NEXT: s_mov_b32 s61, s62
+; SI-NEXT: s_mov_b32 s62, s63
+; SI-NEXT: s_mov_b32 s63, s72
+; SI-NEXT: s_mov_b32 s72, s73
+; SI-NEXT: s_mov_b32 s73, s74
+; SI-NEXT: s_mov_b32 s74, s75
+; SI-NEXT: s_mov_b32 s75, s76
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: v_readlane_b32 s7, v34, 19
+; SI-NEXT: v_readlane_b32 s8, v34, 20
+; SI-NEXT: v_readlane_b32 s76, v34, 21
; SI-NEXT: s_cbranch_scc1 .LBB17_5
; SI-NEXT: ; %bb.4: ; %cmp.true
+; SI-NEXT: v_readlane_b32 s4, v34, 17
+; SI-NEXT: s_add_i32 s6, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 16
+; SI-NEXT: s_add_i32 s7, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 15
+; SI-NEXT: s_add_i32 s8, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 14
+; SI-NEXT: s_add_i32 s74, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 13
+; SI-NEXT: s_add_i32 s72, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 12
+; SI-NEXT: s_add_i32 s62, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 11
+; SI-NEXT: s_add_i32 s60, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 10
+; SI-NEXT: s_add_i32 s58, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 9
+; SI-NEXT: s_add_i32 s56, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 8
+; SI-NEXT: s_add_i32 s46, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 7
+; SI-NEXT: s_add_i32 s44, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 6
+; SI-NEXT: s_add_i32 s42, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 5
+; SI-NEXT: s_add_i32 s40, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 4
+; SI-NEXT: s_add_i32 s14, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 3
+; SI-NEXT: s_add_i32 s12, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 2
+; SI-NEXT: s_add_i32 s10, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 1
+; SI-NEXT: s_add_i32 s11, s4, 3
; SI-NEXT: v_readlane_b32 s4, v34, 0
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_add_i32 s17, s17, 3
@@ -23059,236 +23036,202 @@ define inreg <64 x bfloat> @bitcast_v32i32_to_v64bf16_scalar(<32 x i32> inreg %a
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s64, s64, 3
-; SI-NEXT: s_add_i32 s55, s55, 3
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: s_add_i32 s52, s52, 3
-; SI-NEXT: s_add_i32 s51, s51, 3
-; SI-NEXT: s_add_i32 s50, s50, 3
-; SI-NEXT: s_add_i32 s49, s49, 3
-; SI-NEXT: s_add_i32 s48, s48, 3
-; SI-NEXT: s_add_i32 s39, s39, 3
-; SI-NEXT: s_add_i32 s38, s38, 3
-; SI-NEXT: s_add_i32 s37, s37, 3
-; SI-NEXT: s_add_i32 s36, s36, 3
-; SI-NEXT: s_add_i32 s35, s35, 3
-; SI-NEXT: s_add_i32 s34, s34, 3
-; SI-NEXT: s_add_i32 s31, s31, 3
-; SI-NEXT: s_add_i32 s30, s30, 3
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_and_b32 s8, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_and_b32 s10, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s31, 16
-; SI-NEXT: s_and_b32 s12, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s34, 16
-; SI-NEXT: s_and_b32 s14, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s35, 16
-; SI-NEXT: s_and_b32 s40, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s36, 16
-; SI-NEXT: s_and_b32 s42, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s37, 16
-; SI-NEXT: s_and_b32 s44, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s38, 16
-; SI-NEXT: s_and_b32 s46, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s39, 16
-; SI-NEXT: s_and_b32 s56, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s48, 16
-; SI-NEXT: s_and_b32 s58, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s49, 16
-; SI-NEXT: s_and_b32 s60, s50, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s50, 16
-; SI-NEXT: s_and_b32 s62, s51, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s51, 16
-; SI-NEXT: s_and_b32 s72, s52, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s52, 16
-; SI-NEXT: s_and_b32 s74, s53, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s53, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: s_and_b32 s75, s55, 0xffff0000
-; SI-NEXT: s_lshl_b32 s6, s55, 16
-; SI-NEXT: s_and_b32 s79, s64, 0xffff0000
-; SI-NEXT: s_lshl_b32 s76, s64, 16
-; SI-NEXT: s_and_b32 s89, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s29, 16
-; SI-NEXT: s_and_b32 s91, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s28, 16
-; SI-NEXT: s_and_b32 s93, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s27, 16
-; SI-NEXT: s_and_b32 s95, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s26, 16
-; SI-NEXT: s_and_b32 vcc_hi, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s25, 16
-; SI-NEXT: s_and_b32 s66, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s24, 16
-; SI-NEXT: s_and_b32 s68, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s23, 16
-; SI-NEXT: s_and_b32 s70, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s22, 16
-; SI-NEXT: s_and_b32 s80, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s21, 16
-; SI-NEXT: s_and_b32 s82, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s20, 16
-; SI-NEXT: s_and_b32 s84, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s19, 16
-; SI-NEXT: s_and_b32 s86, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s18, 16
-; SI-NEXT: s_and_b32 s96, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s17, 16
-; SI-NEXT: s_and_b32 s98, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s97, s16, 16
+; SI-NEXT: s_and_b32 s9, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s55, s11, 16
+; SI-NEXT: s_and_b32 s11, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s13, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s15, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s41, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_and_b32 s63, s62, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_and_b32 s73, s72, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_and_b32 s75, s74, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s74, 16
+; SI-NEXT: s_and_b32 s78, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s8, 16
+; SI-NEXT: s_and_b32 s88, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s7, 16
+; SI-NEXT: s_and_b32 s90, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s6, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: s_and_b32 s7, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s6, s27, 16
+; SI-NEXT: s_and_b32 s76, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s26, 16
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s30, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s65, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s16, 16
; SI-NEXT: .LBB17_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s64
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s34
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s36
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s38
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s48
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s50
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s52
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s54
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_hi
-; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_lo
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s95
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s8
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s93
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s91
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s89
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s79
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s77
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s63
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s59
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s47
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s15
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s13
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s55
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v33, 33
+; SI-NEXT: v_readlane_b32 s30, v33, 16
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
-; SI-NEXT: v_readlane_b32 s31, v33, 34
-; SI-NEXT: v_readlane_b32 s98, v33, 32
-; SI-NEXT: v_readlane_b32 s97, v33, 31
-; SI-NEXT: v_readlane_b32 s96, v33, 30
-; SI-NEXT: v_readlane_b32 s87, v33, 29
-; SI-NEXT: v_readlane_b32 s86, v33, 28
-; SI-NEXT: v_readlane_b32 s85, v33, 27
-; SI-NEXT: v_readlane_b32 s84, v33, 26
-; SI-NEXT: v_readlane_b32 s83, v33, 25
-; SI-NEXT: v_readlane_b32 s82, v33, 24
-; SI-NEXT: v_readlane_b32 s81, v33, 23
-; SI-NEXT: v_readlane_b32 s80, v33, 22
-; SI-NEXT: v_readlane_b32 s71, v33, 21
-; SI-NEXT: v_readlane_b32 s70, v33, 20
-; SI-NEXT: v_readlane_b32 s69, v33, 19
-; SI-NEXT: v_readlane_b32 s68, v33, 18
-; SI-NEXT: v_readlane_b32 s67, v33, 17
-; SI-NEXT: v_readlane_b32 s66, v33, 16
+; SI-NEXT: v_readlane_b32 s31, v33, 17
; SI-NEXT: v_readlane_b32 s65, v33, 15
; SI-NEXT: v_readlane_b32 s64, v33, 14
; SI-NEXT: v_readlane_b32 s55, v33, 13
@@ -32494,183 +32437,213 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB23_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB23_3
; SI-NEXT: .LBB23_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB23_3: ; %Flow
@@ -32679,9 +32652,9 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB23_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -32693,7 +32666,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -32701,11 +32674,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -32716,7 +32689,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -32725,11 +32698,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -32740,7 +32713,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -32749,11 +32722,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -32764,7 +32737,7 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -32773,11 +32746,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -32785,47 +32758,56 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -32833,10 +32815,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -32844,12 +32827,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -32857,10 +32842,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -32868,12 +32854,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -32881,10 +32869,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -32892,12 +32881,14 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -32905,10 +32896,11 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -32924,7 +32916,8 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -32972,24 +32965,8 @@ define inreg <32 x i32> @bitcast_v64f16_to_v32i32_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB23_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -35537,195 +35514,215 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB27_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB27_3
; SI-NEXT: .LBB27_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB27_3: ; %Flow
@@ -35736,168 +35733,188 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB27_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -35930,23 +35947,7 @@ define inreg <32 x i32> @bitcast_v64i16_to_v32i32_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -43433,224 +43434,283 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_writelane_b32 v63, s30, 34
; SI-NEXT: v_writelane_b32 v63, s31, 35
; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: v_readfirstlane_b32 s9, v18
-; SI-NEXT: v_readfirstlane_b32 s8, v17
-; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_readfirstlane_b32 s30, v15
-; SI-NEXT: v_readfirstlane_b32 s35, v14
-; SI-NEXT: v_readfirstlane_b32 s34, v13
-; SI-NEXT: v_readfirstlane_b32 s37, v12
-; SI-NEXT: v_readfirstlane_b32 s36, v11
-; SI-NEXT: v_readfirstlane_b32 s39, v10
-; SI-NEXT: v_readfirstlane_b32 s38, v9
-; SI-NEXT: v_readfirstlane_b32 s49, v8
-; SI-NEXT: v_readfirstlane_b32 s48, v7
-; SI-NEXT: v_readfirstlane_b32 s51, v6
-; SI-NEXT: v_readfirstlane_b32 s50, v5
-; SI-NEXT: v_readfirstlane_b32 s53, v4
-; SI-NEXT: v_readfirstlane_b32 s52, v3
-; SI-NEXT: v_readfirstlane_b32 s55, v2
+; SI-NEXT: v_readfirstlane_b32 s61, v18
+; SI-NEXT: v_readfirstlane_b32 s60, v17
+; SI-NEXT: v_readfirstlane_b32 s15, v16
+; SI-NEXT: v_readfirstlane_b32 s14, v15
+; SI-NEXT: v_readfirstlane_b32 s59, v14
+; SI-NEXT: v_readfirstlane_b32 s58, v13
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_readfirstlane_b32 s40, v11
+; SI-NEXT: v_readfirstlane_b32 s47, v10
+; SI-NEXT: v_readfirstlane_b32 s46, v9
+; SI-NEXT: v_readfirstlane_b32 s45, v8
+; SI-NEXT: v_readfirstlane_b32 s44, v7
+; SI-NEXT: v_readfirstlane_b32 s43, v6
+; SI-NEXT: v_readfirstlane_b32 s42, v5
+; SI-NEXT: v_readfirstlane_b32 s63, v4
+; SI-NEXT: v_readfirstlane_b32 s62, v3
+; SI-NEXT: v_readfirstlane_b32 s95, v2
; SI-NEXT: s_cmp_lg_u32 s4, 0
-; SI-NEXT: v_readfirstlane_b32 s54, v1
+; SI-NEXT: v_readfirstlane_b32 s94, v1
; SI-NEXT: ; implicit-def: $vgpr61 : SGPR spill to VGPR lane
; SI-NEXT: ; implicit-def: $vgpr62 : SGPR spill to VGPR lane
; SI-NEXT: s_cbranch_scc0 .LBB37_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshr_b32 s4, s53, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 18
-; SI-NEXT: s_lshr_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 17
-; SI-NEXT: s_lshr_b32 s4, s55, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 16
-; SI-NEXT: s_lshr_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 15
-; SI-NEXT: s_lshr_b32 s4, s55, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 14
-; SI-NEXT: s_lshr_b32 s4, s29, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 13
-; SI-NEXT: s_lshr_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 12
-; SI-NEXT: s_lshr_b32 s4, s29, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 11
-; SI-NEXT: s_lshr_b32 s4, s27, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 10
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 9
-; SI-NEXT: s_lshr_b32 s4, s27, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 8
-; SI-NEXT: s_lshr_b32 s4, s25, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 7
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 6
-; SI-NEXT: s_lshr_b32 s4, s25, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 5
-; SI-NEXT: s_lshr_b32 s4, s23, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 4
-; SI-NEXT: s_lshr_b32 s4, s23, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 3
-; SI-NEXT: s_lshr_b32 s4, s23, 8
-; SI-NEXT: v_writelane_b32 v62, s4, 2
-; SI-NEXT: s_lshr_b32 s4, s21, 24
-; SI-NEXT: v_writelane_b32 v62, s4, 1
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s4, 0
-; SI-NEXT: s_lshr_b32 s4, s21, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 63
-; SI-NEXT: s_lshr_b32 s4, s19, 24
-; SI-NEXT: v_writelane_b32 v61, s4, 62
-; SI-NEXT: s_lshr_b32 s4, s19, 16
-; SI-NEXT: v_writelane_b32 v61, s4, 61
-; SI-NEXT: s_lshr_b32 s4, s19, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 60
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: v_writelane_b32 v61, s4, 59
-; SI-NEXT: s_lshr_b32 s4, s17, 8
-; SI-NEXT: v_writelane_b32 v61, s4, 58
-; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 24
-; SI-NEXT: v_writelane_b32 v61, s4, 56
-; SI-NEXT: v_writelane_b32 v61, s5, 57
-; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
-; SI-NEXT: v_writelane_b32 v61, s4, 54
-; SI-NEXT: v_writelane_b32 v61, s5, 55
-; SI-NEXT: s_lshr_b64 s[12:13], s[30:31], 24
-; SI-NEXT: v_writelane_b32 v61, s12, 52
-; SI-NEXT: v_writelane_b32 v61, s13, 53
-; SI-NEXT: s_lshr_b64 s[40:41], s[30:31], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 50
-; SI-NEXT: v_writelane_b32 v61, s41, 51
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 48
-; SI-NEXT: v_writelane_b32 v61, s41, 49
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 16
-; SI-NEXT: v_writelane_b32 v61, s40, 46
-; SI-NEXT: v_writelane_b32 v61, s41, 47
-; SI-NEXT: s_lshr_b64 s[40:41], s[34:35], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 44
-; SI-NEXT: v_writelane_b32 v61, s41, 45
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 42
-; SI-NEXT: v_writelane_b32 v61, s41, 43
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 16
-; SI-NEXT: v_writelane_b32 v61, s40, 40
-; SI-NEXT: v_writelane_b32 v61, s41, 41
-; SI-NEXT: s_lshr_b64 s[40:41], s[36:37], 8
-; SI-NEXT: v_writelane_b32 v61, s40, 38
-; SI-NEXT: v_writelane_b32 v61, s41, 39
-; SI-NEXT: s_lshr_b64 s[40:41], s[38:39], 24
-; SI-NEXT: v_writelane_b32 v61, s40, 36
-; SI-NEXT: v_writelane_b32 v61, s41, 37
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 24
+; SI-NEXT: s_lshr_b64 s[6:7], s[60:61], 8
+; SI-NEXT: v_writelane_b32 v61, s6, 62
+; SI-NEXT: v_writelane_b32 v61, s7, 63
+; SI-NEXT: s_lshr_b64 s[6:7], s[14:15], 24
+; SI-NEXT: v_writelane_b32 v61, s6, 60
+; SI-NEXT: v_writelane_b32 v61, s7, 61
+; SI-NEXT: s_lshr_b64 s[6:7], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v61, s6, 58
+; SI-NEXT: v_writelane_b32 v61, s7, 59
+; SI-NEXT: s_lshr_b64 s[8:9], s[58:59], 24
+; SI-NEXT: v_writelane_b32 v61, s8, 56
+; SI-NEXT: v_writelane_b32 v61, s9, 57
+; SI-NEXT: s_lshr_b64 s[8:9], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v61, s8, 54
+; SI-NEXT: v_writelane_b32 v61, s9, 55
+; SI-NEXT: s_lshr_b64 s[10:11], s[58:59], 8
+; SI-NEXT: v_writelane_b32 v61, s10, 52
+; SI-NEXT: v_writelane_b32 v61, s11, 53
+; SI-NEXT: s_lshr_b64 s[10:11], s[40:41], 24
+; SI-NEXT: v_writelane_b32 v61, s10, 50
+; SI-NEXT: v_writelane_b32 v61, s11, 51
+; SI-NEXT: s_lshr_b64 s[10:11], s[40:41], 16
+; SI-NEXT: v_writelane_b32 v61, s10, 48
+; SI-NEXT: v_writelane_b32 v61, s11, 49
+; SI-NEXT: s_lshr_b64 s[12:13], s[40:41], 8
+; SI-NEXT: v_writelane_b32 v61, s12, 46
+; SI-NEXT: v_writelane_b32 v61, s13, 47
+; SI-NEXT: s_lshr_b64 s[12:13], s[46:47], 24
+; SI-NEXT: v_writelane_b32 v61, s12, 44
+; SI-NEXT: s_lshr_b32 s48, s59, 24
+; SI-NEXT: s_lshr_b32 s50, s59, 16
+; SI-NEXT: s_lshr_b32 s52, s59, 8
+; SI-NEXT: s_lshr_b32 s37, s41, 24
+; SI-NEXT: s_lshr_b32 s39, s41, 16
+; SI-NEXT: s_lshr_b32 s49, s41, 8
+; SI-NEXT: s_lshr_b32 s99, s47, 24
+; SI-NEXT: s_lshr_b32 s36, s47, 16
+; SI-NEXT: s_lshr_b32 s38, s47, 8
+; SI-NEXT: s_mov_b64 s[8:9], s[58:59]
+; SI-NEXT: s_mov_b64 s[10:11], s[40:41]
+; SI-NEXT: v_writelane_b32 v61, s13, 45
+; SI-NEXT: s_lshr_b64 s[40:41], s[46:47], 16
+; SI-NEXT: s_mov_b64 s[12:13], s[46:47]
+; SI-NEXT: s_lshr_b64 s[58:59], s[46:47], 8
+; SI-NEXT: s_lshr_b64 s[46:47], s[42:43], 24
+; SI-NEXT: v_writelane_b32 v61, s46, 42
+; SI-NEXT: v_writelane_b32 v61, s47, 43
+; SI-NEXT: s_lshr_b64 s[46:47], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v61, s46, 40
+; SI-NEXT: s_lshr_b32 s85, s43, 24
+; SI-NEXT: s_lshr_b32 s87, s43, 16
+; SI-NEXT: s_lshr_b32 s97, s43, 8
+; SI-NEXT: v_writelane_b32 v61, s47, 41
+; SI-NEXT: s_mov_b64 s[74:75], s[42:43]
+; SI-NEXT: s_lshr_b64 s[42:43], s[42:43], 8
+; SI-NEXT: v_writelane_b32 v61, s42, 38
+; SI-NEXT: v_writelane_b32 v61, s43, 39
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 24
+; SI-NEXT: v_writelane_b32 v61, s42, 36
+; SI-NEXT: v_writelane_b32 v61, s43, 37
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 16
; SI-NEXT: v_writelane_b32 v61, s42, 34
; SI-NEXT: v_writelane_b32 v61, s43, 35
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 8
; SI-NEXT: v_writelane_b32 v61, s42, 32
; SI-NEXT: v_writelane_b32 v61, s43, 33
-; SI-NEXT: s_lshr_b64 s[42:43], s[50:51], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 24
; SI-NEXT: v_writelane_b32 v61, s42, 30
; SI-NEXT: v_writelane_b32 v61, s43, 31
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 16
; SI-NEXT: v_writelane_b32 v61, s42, 28
; SI-NEXT: v_writelane_b32 v61, s43, 29
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[94:95], 8
; SI-NEXT: v_writelane_b32 v61, s42, 26
; SI-NEXT: v_writelane_b32 v61, s43, 27
-; SI-NEXT: s_lshr_b64 s[42:43], s[52:53], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 24
; SI-NEXT: v_writelane_b32 v61, s42, 24
; SI-NEXT: v_writelane_b32 v61, s43, 25
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 16
; SI-NEXT: v_writelane_b32 v61, s42, 22
; SI-NEXT: v_writelane_b32 v61, s43, 23
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 8
; SI-NEXT: v_writelane_b32 v61, s42, 20
; SI-NEXT: v_writelane_b32 v61, s43, 21
-; SI-NEXT: s_lshr_b64 s[42:43], s[54:55], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 24
; SI-NEXT: v_writelane_b32 v61, s42, 18
+; SI-NEXT: s_lshr_b32 s4, s27, 24
; SI-NEXT: v_writelane_b32 v61, s43, 19
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 17
+; SI-NEXT: s_lshr_b32 s4, s25, 24
; SI-NEXT: v_writelane_b32 v61, s42, 16
+; SI-NEXT: v_writelane_b32 v62, s4, 16
+; SI-NEXT: s_lshr_b32 s4, s25, 16
; SI-NEXT: v_writelane_b32 v61, s43, 17
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 15
+; SI-NEXT: s_lshr_b32 s4, s23, 24
; SI-NEXT: v_writelane_b32 v61, s42, 14
+; SI-NEXT: v_writelane_b32 v62, s4, 14
+; SI-NEXT: s_lshr_b32 s4, s23, 16
; SI-NEXT: v_writelane_b32 v61, s43, 15
-; SI-NEXT: s_lshr_b64 s[42:43], s[28:29], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 13
+; SI-NEXT: s_lshr_b32 s4, s23, 8
; SI-NEXT: v_writelane_b32 v61, s42, 12
+; SI-NEXT: v_writelane_b32 v62, s4, 12
+; SI-NEXT: s_lshr_b32 s4, s21, 24
; SI-NEXT: v_writelane_b32 v61, s43, 13
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 11
+; SI-NEXT: s_lshr_b32 s4, s21, 16
; SI-NEXT: v_writelane_b32 v61, s42, 10
+; SI-NEXT: v_writelane_b32 v62, s4, 10
+; SI-NEXT: s_lshr_b32 s4, s21, 8
; SI-NEXT: v_writelane_b32 v61, s43, 11
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 9
+; SI-NEXT: s_lshr_b32 s4, s19, 24
; SI-NEXT: v_writelane_b32 v61, s42, 8
+; SI-NEXT: v_writelane_b32 v62, s4, 8
+; SI-NEXT: s_lshr_b32 s4, s19, 16
; SI-NEXT: v_writelane_b32 v61, s43, 9
-; SI-NEXT: s_lshr_b64 s[42:43], s[26:27], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 7
+; SI-NEXT: s_lshr_b32 s4, s19, 8
; SI-NEXT: v_writelane_b32 v61, s42, 6
+; SI-NEXT: v_writelane_b32 v62, s4, 6
+; SI-NEXT: s_lshr_b32 s4, s17, 16
; SI-NEXT: v_writelane_b32 v61, s43, 7
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 24
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 16
+; SI-NEXT: v_writelane_b32 v62, s4, 5
+; SI-NEXT: s_lshr_b32 s4, s17, 8
; SI-NEXT: v_writelane_b32 v61, s42, 4
+; SI-NEXT: v_writelane_b32 v62, s4, 4
+; SI-NEXT: s_lshr_b64 s[4:5], s[60:61], 24
; SI-NEXT: v_writelane_b32 v61, s43, 5
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[22:23], 8
+; SI-NEXT: v_writelane_b32 v62, s4, 2
; SI-NEXT: v_writelane_b32 v61, s42, 2
+; SI-NEXT: v_writelane_b32 v62, s5, 3
+; SI-NEXT: s_lshr_b64 s[4:5], s[60:61], 16
; SI-NEXT: v_writelane_b32 v61, s43, 3
-; SI-NEXT: s_lshr_b64 s[42:43], s[24:25], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[20:21], 24
+; SI-NEXT: v_writelane_b32 v62, s4, 0
; SI-NEXT: v_writelane_b32 v61, s42, 0
-; SI-NEXT: s_lshr_b32 s15, s9, 24
-; SI-NEXT: s_lshr_b32 s69, s9, 16
-; SI-NEXT: s_lshr_b32 s70, s9, 8
-; SI-NEXT: s_lshr_b32 s10, s31, 24
-; SI-NEXT: s_lshr_b32 s14, s31, 16
-; SI-NEXT: s_lshr_b32 s68, s31, 8
-; SI-NEXT: s_lshr_b32 s98, s35, 24
-; SI-NEXT: s_lshr_b32 s7, s35, 16
-; SI-NEXT: s_lshr_b32 s11, s35, 8
-; SI-NEXT: s_lshr_b32 s96, s37, 24
-; SI-NEXT: s_lshr_b32 s67, s37, 16
-; SI-NEXT: s_lshr_b32 s99, s37, 8
-; SI-NEXT: s_lshr_b32 s85, s39, 24
-; SI-NEXT: s_lshr_b32 s87, s39, 16
-; SI-NEXT: s_lshr_b32 s97, s39, 8
-; SI-NEXT: s_lshr_b32 s82, s49, 24
-; SI-NEXT: s_lshr_b32 s84, s49, 16
-; SI-NEXT: s_lshr_b32 s86, s49, 8
-; SI-NEXT: s_lshr_b32 s71, s51, 24
-; SI-NEXT: s_lshr_b32 s81, s51, 16
-; SI-NEXT: s_lshr_b32 s83, s51, 8
-; SI-NEXT: s_lshr_b32 s80, s53, 8
-; SI-NEXT: s_lshr_b32 s66, s17, 24
-; SI-NEXT: s_mov_b64 s[4:5], s[8:9]
-; SI-NEXT: s_lshr_b64 s[8:9], s[8:9], 8
-; SI-NEXT: s_lshr_b64 s[12:13], s[30:31], 16
-; SI-NEXT: s_lshr_b64 s[40:41], s[38:39], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[38:39], 8
-; SI-NEXT: s_lshr_b64 s[44:45], s[48:49], 24
-; SI-NEXT: s_lshr_b64 s[56:57], s[48:49], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[48:49], 8
+; SI-NEXT: s_lshr_b32 s54, s61, 24
+; SI-NEXT: s_lshr_b32 s64, s61, 16
+; SI-NEXT: s_lshr_b32 s65, s61, 8
+; SI-NEXT: s_lshr_b32 s51, s15, 24
+; SI-NEXT: s_lshr_b32 s53, s15, 16
+; SI-NEXT: s_lshr_b32 s55, s15, 8
+; SI-NEXT: s_lshr_b32 s96, s45, 24
+; SI-NEXT: s_lshr_b32 s98, s45, 16
+; SI-NEXT: s_lshr_b32 s35, s45, 8
+; SI-NEXT: s_lshr_b32 s82, s63, 24
+; SI-NEXT: s_lshr_b32 s84, s63, 16
+; SI-NEXT: s_lshr_b32 s86, s63, 8
+; SI-NEXT: s_lshr_b32 s71, s95, 24
+; SI-NEXT: s_lshr_b32 s81, s95, 16
+; SI-NEXT: s_lshr_b32 s83, s95, 8
+; SI-NEXT: s_lshr_b32 s68, s29, 24
+; SI-NEXT: s_lshr_b32 s70, s29, 16
+; SI-NEXT: s_lshr_b32 s80, s29, 8
+; SI-NEXT: s_lshr_b32 s67, s27, 16
+; SI-NEXT: s_lshr_b32 s69, s27, 8
+; SI-NEXT: s_lshr_b32 s66, s25, 8
+; SI-NEXT: s_lshr_b32 s34, s17, 24
+; SI-NEXT: v_writelane_b32 v62, s5, 1
+; SI-NEXT: s_mov_b64 s[4:5], s[60:61]
+; SI-NEXT: s_mov_b64 s[6:7], s[14:15]
+; SI-NEXT: s_lshr_b64 s[14:15], s[14:15], 8
+; SI-NEXT: s_lshr_b64 s[60:61], s[44:45], 24
+; SI-NEXT: s_lshr_b64 s[72:73], s[44:45], 16
+; SI-NEXT: s_mov_b64 s[56:57], s[44:45]
+; SI-NEXT: s_lshr_b64 s[44:45], s[44:45], 8
; SI-NEXT: v_writelane_b32 v61, s43, 1
-; SI-NEXT: s_lshr_b64 s[74:75], s[22:23], 24
-; SI-NEXT: s_lshr_b64 s[46:47], s[22:23], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[22:23], 8
-; SI-NEXT: s_lshr_b64 s[58:59], s[20:21], 24
-; SI-NEXT: s_lshr_b64 s[60:61], s[20:21], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[20:21], 8
-; SI-NEXT: s_lshr_b64 s[88:89], s[18:19], 24
-; SI-NEXT: s_lshr_b64 s[90:91], s[18:19], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[18:19], 8
-; SI-NEXT: s_lshr_b64 s[94:95], s[16:17], 24
+; SI-NEXT: s_lshr_b64 s[46:47], s[20:21], 16
+; SI-NEXT: s_lshr_b64 s[42:43], s[20:21], 8
+; SI-NEXT: s_lshr_b64 s[78:79], s[18:19], 24
+; SI-NEXT: s_lshr_b64 s[88:89], s[18:19], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[18:19], 8
+; SI-NEXT: s_lshr_b64 s[92:93], s[16:17], 24
; SI-NEXT: s_lshr_b64 vcc, s[16:17], 16
-; SI-NEXT: s_lshr_b64 s[42:43], s[16:17], 8
-; SI-NEXT: s_mov_b64 s[64:65], 0
+; SI-NEXT: s_lshr_b64 s[76:77], s[16:17], 8
+; SI-NEXT: s_mov_b64 s[30:31], 0
; SI-NEXT: s_branch .LBB37_3
; SI-NEXT: .LBB37_2:
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr67
+; SI-NEXT: s_mov_b64 s[6:7], s[14:15]
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; kill: killed $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: s_mov_b64 s[74:75], s[42:43]
+; SI-NEXT: v_writelane_b32 v61, s14, 0
+; SI-NEXT: v_writelane_b32 v61, s15, 1
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[56:57], s[44:45]
+; SI-NEXT: v_writelane_b32 v61, s14, 2
+; SI-NEXT: v_writelane_b32 v61, s15, 3
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[12:13], s[46:47]
+; SI-NEXT: v_writelane_b32 v61, s14, 4
+; SI-NEXT: v_writelane_b32 v61, s15, 5
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[10:11], s[40:41]
+; SI-NEXT: v_writelane_b32 v61, s14, 6
+; SI-NEXT: v_writelane_b32 v61, s15, 7
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[8:9], s[58:59]
+; SI-NEXT: v_writelane_b32 v61, s14, 8
+; SI-NEXT: v_writelane_b32 v61, s15, 9
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[4:5], s[60:61]
+; SI-NEXT: v_writelane_b32 v61, s14, 10
+; SI-NEXT: v_writelane_b32 v61, s15, 11
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: s_mov_b64 s[30:31], -1
+; SI-NEXT: v_writelane_b32 v61, s14, 12
+; SI-NEXT: v_writelane_b32 v61, s15, 13
+; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: s_mov_b64 s[4:5], s[8:9]
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b64 s[64:65], -1
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr83
; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr71
@@ -43660,289 +43720,240 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: ; implicit-def: $sgpr97
; SI-NEXT: ; implicit-def: $sgpr87
; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr99
-; SI-NEXT: ; implicit-def: $sgpr96
-; SI-NEXT: ; implicit-def: $sgpr11
; SI-NEXT: ; implicit-def: $sgpr98
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr69
-; SI-NEXT: ; implicit-def: $sgpr15
-; SI-NEXT: ; implicit-def: $sgpr42
+; SI-NEXT: ; implicit-def: $sgpr96
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr99
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr53
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr56
; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr60
+; SI-NEXT: ; implicit-def: $sgpr58
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 0
-; SI-NEXT: v_writelane_b32 v61, s7, 1
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 2
-; SI-NEXT: v_writelane_b32 v61, s7, 3
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 4
-; SI-NEXT: v_writelane_b32 v61, s7, 5
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 6
-; SI-NEXT: v_writelane_b32 v61, s7, 7
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 8
-; SI-NEXT: v_writelane_b32 v61, s7, 9
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 10
-; SI-NEXT: v_writelane_b32 v61, s7, 11
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 12
-; SI-NEXT: v_writelane_b32 v61, s7, 13
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 14
-; SI-NEXT: v_writelane_b32 v61, s7, 15
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 16
-; SI-NEXT: v_writelane_b32 v61, s7, 17
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 18
-; SI-NEXT: v_writelane_b32 v61, s7, 19
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 20
-; SI-NEXT: v_writelane_b32 v61, s7, 21
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 22
-; SI-NEXT: v_writelane_b32 v61, s7, 23
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 24
-; SI-NEXT: v_writelane_b32 v61, s7, 25
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 26
-; SI-NEXT: v_writelane_b32 v61, s7, 27
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 28
-; SI-NEXT: v_writelane_b32 v61, s7, 29
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 30
-; SI-NEXT: v_writelane_b32 v61, s7, 31
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 32
-; SI-NEXT: v_writelane_b32 v61, s7, 33
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 34
-; SI-NEXT: v_writelane_b32 v61, s7, 35
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 36
-; SI-NEXT: v_writelane_b32 v61, s7, 37
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 38
-; SI-NEXT: v_writelane_b32 v61, s7, 39
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 40
-; SI-NEXT: v_writelane_b32 v61, s7, 41
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 42
-; SI-NEXT: v_writelane_b32 v61, s7, 43
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 44
-; SI-NEXT: v_writelane_b32 v61, s7, 45
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 46
-; SI-NEXT: v_writelane_b32 v61, s7, 47
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 48
-; SI-NEXT: v_writelane_b32 v61, s7, 49
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 50
-; SI-NEXT: v_writelane_b32 v61, s7, 51
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 52
-; SI-NEXT: v_writelane_b32 v61, s7, 53
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: v_writelane_b32 v61, s6, 54
-; SI-NEXT: v_writelane_b32 v61, s7, 55
-; SI-NEXT: v_writelane_b32 v61, s66, 56
-; SI-NEXT: v_writelane_b32 v61, s67, 57
-; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: v_writelane_b32 v61, s14, 14
+; SI-NEXT: v_writelane_b32 v61, s15, 15
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 16
+; SI-NEXT: v_writelane_b32 v61, s15, 17
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 18
+; SI-NEXT: v_writelane_b32 v61, s15, 19
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 20
+; SI-NEXT: v_writelane_b32 v61, s15, 21
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 22
+; SI-NEXT: v_writelane_b32 v61, s15, 23
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 24
+; SI-NEXT: v_writelane_b32 v61, s15, 25
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 26
+; SI-NEXT: v_writelane_b32 v61, s15, 27
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 28
+; SI-NEXT: v_writelane_b32 v61, s15, 29
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 30
+; SI-NEXT: v_writelane_b32 v61, s15, 31
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 32
+; SI-NEXT: v_writelane_b32 v61, s15, 33
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 34
+; SI-NEXT: v_writelane_b32 v61, s15, 35
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 36
+; SI-NEXT: v_writelane_b32 v61, s15, 37
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 38
+; SI-NEXT: v_writelane_b32 v61, s15, 39
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 40
+; SI-NEXT: v_writelane_b32 v61, s15, 41
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 42
+; SI-NEXT: v_writelane_b32 v61, s15, 43
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 44
+; SI-NEXT: v_writelane_b32 v61, s15, 45
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 46
+; SI-NEXT: v_writelane_b32 v61, s15, 47
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 48
+; SI-NEXT: v_writelane_b32 v61, s15, 49
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 50
+; SI-NEXT: v_writelane_b32 v61, s15, 51
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 52
+; SI-NEXT: v_writelane_b32 v61, s15, 53
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 54
+; SI-NEXT: v_writelane_b32 v61, s15, 55
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s14, 56
+; SI-NEXT: v_writelane_b32 v61, s15, 57
+; SI-NEXT: v_writelane_b32 v61, s34, 58
+; SI-NEXT: v_writelane_b32 v61, s35, 59
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: v_writelane_b32 v61, s34, 60
+; SI-NEXT: v_writelane_b32 v61, s35, 61
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v61, s34, 62
+; SI-NEXT: v_writelane_b32 v61, s35, 63
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v62, s34, 0
+; SI-NEXT: v_writelane_b32 v62, s35, 1
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: v_writelane_b32 v62, s34, 2
+; SI-NEXT: v_writelane_b32 v62, s35, 3
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: .LBB37_3: ; %Flow
-; SI-NEXT: s_and_b64 s[64:65], s[64:65], exec
-; SI-NEXT: s_cselect_b32 s6, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s6, 1
+; SI-NEXT: s_and_b64 s[30:31], s[30:31], exec
+; SI-NEXT: s_cselect_b32 s15, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s15, 1
; SI-NEXT: s_cbranch_scc1 .LBB37_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_add_f32_e64 v1, s38, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s12, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s44, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s36, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s10, 1.0
; SI-NEXT: v_add_f32_e64 v22, s26, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s26, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s34, 1.0
-; SI-NEXT: v_add_f32_e64 v29, s20, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s8, 1.0
+; SI-NEXT: v_add_f32_e64 v28, s20, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s20, v1
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_add_f32_e64 v1, s30, 1.0
+; SI-NEXT: v_add_f32_e64 v1, s6, 1.0
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s14, v1
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_add_f32_e64 v1, s5, 1.0
; SI-NEXT: v_add_f32_e64 v3, s4, 1.0
-; SI-NEXT: v_add_f32_e64 v15, s53, 1.0
+; SI-NEXT: v_add_f32_e64 v16, s95, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
; SI-NEXT: v_readfirstlane_b32 s8, v3
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v1
-; SI-NEXT: v_add_f32_e64 v17, s55, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v15
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v16
+; SI-NEXT: v_add_f32_e64 v18, s29, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v17
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v16
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v17
-; SI-NEXT: v_add_f32_e64 v19, s29, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v18
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v17
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v18
+; SI-NEXT: v_add_f32_e64 v21, s27, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v19
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v18
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v19
-; SI-NEXT: v_add_f32_e64 v21, s27, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v21
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v19
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v21
+; SI-NEXT: v_add_f32_e64 v23, s25, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v21
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v21
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v21
-; SI-NEXT: v_add_f32_e64 v23, s25, 1.0
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v23
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v21
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v23
+; SI-NEXT: v_add_f32_e64 v25, s23, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v23
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v23
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v23
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v25
; SI-NEXT: v_add_f32_e64 v34, s17, 1.0
-; SI-NEXT: v_add_f32_e64 v38, s16, 1.0
-; SI-NEXT: v_add_f32_e64 v31, s19, 1.0
+; SI-NEXT: v_add_f32_e64 v36, s16, 1.0
+; SI-NEXT: v_add_f32_e64 v29, s19, 1.0
; SI-NEXT: v_add_f32_e64 v33, s18, 1.0
; SI-NEXT: v_add_f32_e64 v27, s21, 1.0
-; SI-NEXT: v_add_f32_e64 v25, s23, 1.0
; SI-NEXT: v_add_f32_e64 v26, s22, 1.0
; SI-NEXT: v_add_f32_e64 v24, s24, 1.0
; SI-NEXT: v_add_f32_e64 v20, s28, 1.0
-; SI-NEXT: v_add_f32_e64 v18, s54, 1.0
-; SI-NEXT: v_add_f32_e64 v16, s52, 1.0
-; SI-NEXT: v_add_f32_e64 v13, s51, 1.0
-; SI-NEXT: v_add_f32_e64 v14, s50, 1.0
-; SI-NEXT: v_add_f32_e64 v11, s49, 1.0
-; SI-NEXT: v_add_f32_e64 v12, s48, 1.0
-; SI-NEXT: v_add_f32_e64 v9, s39, 1.0
-; SI-NEXT: v_add_f32_e64 v7, s37, 1.0
-; SI-NEXT: v_add_f32_e64 v5, s35, 1.0
-; SI-NEXT: v_add_f32_e64 v2, s31, 1.0
+; SI-NEXT: v_add_f32_e64 v19, s94, 1.0
+; SI-NEXT: v_add_f32_e64 v14, s63, 1.0
+; SI-NEXT: v_add_f32_e64 v17, s62, 1.0
+; SI-NEXT: v_add_f32_e64 v12, s75, 1.0
+; SI-NEXT: v_add_f32_e64 v15, s74, 1.0
+; SI-NEXT: v_add_f32_e64 v10, s57, 1.0
+; SI-NEXT: v_add_f32_e64 v13, s56, 1.0
+; SI-NEXT: v_add_f32_e64 v8, s13, 1.0
+; SI-NEXT: v_add_f32_e64 v6, s11, 1.0
+; SI-NEXT: v_add_f32_e64 v4, s9, 1.0
+; SI-NEXT: v_add_f32_e64 v2, s7, 1.0
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v23
-; SI-NEXT: v_readfirstlane_b32 s98, v38
+; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v25
+; SI-NEXT: v_readfirstlane_b32 s98, v36
; SI-NEXT: v_readfirstlane_b32 s99, v34
; SI-NEXT: v_readfirstlane_b32 s84, v33
-; SI-NEXT: v_readfirstlane_b32 s85, v31
-; SI-NEXT: v_readfirstlane_b32 s70, v29
+; SI-NEXT: v_readfirstlane_b32 s85, v29
+; SI-NEXT: v_readfirstlane_b32 s70, v28
; SI-NEXT: v_readfirstlane_b32 s71, v27
; SI-NEXT: v_readfirstlane_b32 s64, v26
; SI-NEXT: v_readfirstlane_b32 s65, v25
-; SI-NEXT: v_readfirstlane_b32 s92, v24
-; SI-NEXT: v_readfirstlane_b32 s93, v23
-; SI-NEXT: v_readfirstlane_b32 s78, v22
-; SI-NEXT: v_readfirstlane_b32 s79, v21
-; SI-NEXT: v_readfirstlane_b32 s76, v20
-; SI-NEXT: v_readfirstlane_b32 s77, v19
-; SI-NEXT: v_readfirstlane_b32 s88, v18
-; SI-NEXT: v_readfirstlane_b32 s89, v17
-; SI-NEXT: v_readfirstlane_b32 s90, v16
-; SI-NEXT: v_readfirstlane_b32 s91, v15
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v13
-; SI-NEXT: v_readfirstlane_b32 s42, v12
-; SI-NEXT: v_readfirstlane_b32 s43, v11
-; SI-NEXT: v_readfirstlane_b32 s45, v9
-; SI-NEXT: v_readfirstlane_b32 s27, v7
-; SI-NEXT: v_readfirstlane_b32 s21, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v24
+; SI-NEXT: v_readfirstlane_b32 s51, v23
+; SI-NEXT: v_readfirstlane_b32 s36, v22
+; SI-NEXT: v_readfirstlane_b32 s37, v21
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v20
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v18
+; SI-NEXT: v_readfirstlane_b32 s88, v19
+; SI-NEXT: v_readfirstlane_b32 s89, v16
+; SI-NEXT: v_readfirstlane_b32 s76, v17
+; SI-NEXT: v_readfirstlane_b32 s77, v14
+; SI-NEXT: v_readfirstlane_b32 s62, v15
+; SI-NEXT: v_readfirstlane_b32 s63, v12
+; SI-NEXT: v_readfirstlane_b32 s42, v13
+; SI-NEXT: v_readfirstlane_b32 s43, v10
+; SI-NEXT: v_readfirstlane_b32 s45, v8
+; SI-NEXT: v_readfirstlane_b32 s27, v6
+; SI-NEXT: v_readfirstlane_b32 s21, v4
; SI-NEXT: v_readfirstlane_b32 s15, v2
; SI-NEXT: v_readfirstlane_b32 s9, v1
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v25
+; SI-NEXT: v_lshrrev_b32_e32 v3, 8, v25
; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 24
; SI-NEXT: s_lshr_b64 s[6:7], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[8:9], s[8:9], 8
@@ -43957,30 +43968,30 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: s_lshr_b64 s[26:27], s[26:27], 8
; SI-NEXT: s_lshr_b64 s[28:29], s[44:45], 24
; SI-NEXT: s_lshr_b64 s[40:41], s[44:45], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[44:45], 8
-; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 24
-; SI-NEXT: s_lshr_b64 s[56:57], s[42:43], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[42:43], 8
-; SI-NEXT: s_lshr_b64 s[42:43], s[58:59], 24
-; SI-NEXT: s_lshr_b64 s[46:47], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[58:59], s[58:59], 8
-; SI-NEXT: s_lshr_b64 s[60:61], s[90:91], 24
-; SI-NEXT: s_lshr_b64 s[74:75], s[90:91], 16
-; SI-NEXT: s_lshr_b64 s[30:31], s[90:91], 8
-; SI-NEXT: s_lshr_b64 s[34:35], s[88:89], 24
-; SI-NEXT: s_lshr_b64 s[36:37], s[88:89], 16
-; SI-NEXT: s_lshr_b64 s[38:39], s[88:89], 8
-; SI-NEXT: s_lshr_b64 s[48:49], s[76:77], 24
-; SI-NEXT: s_lshr_b64 s[50:51], s[76:77], 16
-; SI-NEXT: s_lshr_b64 s[52:53], s[76:77], 8
-; SI-NEXT: s_lshr_b64 s[54:55], s[78:79], 24
-; SI-NEXT: s_lshr_b64 s[76:77], s[78:79], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 8
-; SI-NEXT: s_lshr_b64 s[88:89], s[92:93], 24
-; SI-NEXT: s_lshr_b64 s[90:91], s[92:93], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 8
-; SI-NEXT: s_lshr_b64 s[94:95], s[64:65], 24
-; SI-NEXT: s_lshr_b64 vcc, s[64:65], 16
+; SI-NEXT: s_lshr_b64 s[58:59], s[44:45], 8
+; SI-NEXT: s_lshr_b64 s[60:61], s[42:43], 24
+; SI-NEXT: s_lshr_b64 s[72:73], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 8
+; SI-NEXT: s_lshr_b64 s[42:43], s[62:63], 24
+; SI-NEXT: s_lshr_b64 s[46:47], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[56:57], s[62:63], 8
+; SI-NEXT: s_lshr_b64 s[62:63], s[76:77], 24
+; SI-NEXT: s_lshr_b64 s[74:75], s[76:77], 16
+; SI-NEXT: s_lshr_b64 s[94:95], s[76:77], 8
+; SI-NEXT: s_lshr_b64 s[76:77], s[88:89], 24
+; SI-NEXT: s_lshr_b64 s[78:79], s[88:89], 16
+; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 8
+; SI-NEXT: s_lshr_b64 s[90:91], vcc, 24
+; SI-NEXT: s_lshr_b64 s[92:93], vcc, 16
+; SI-NEXT: s_lshr_b64 vcc, vcc, 8
+; SI-NEXT: s_lshr_b64 s[30:31], s[36:37], 24
+; SI-NEXT: s_lshr_b64 s[34:35], s[36:37], 16
+; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 8
+; SI-NEXT: s_lshr_b64 s[38:39], s[50:51], 24
+; SI-NEXT: s_lshr_b64 s[48:49], s[50:51], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 8
+; SI-NEXT: s_lshr_b64 s[52:53], s[64:65], 24
+; SI-NEXT: s_lshr_b64 s[54:55], s[64:65], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 8
; SI-NEXT: s_lshr_b64 s[66:67], s[70:71], 24
; SI-NEXT: s_lshr_b64 s[68:69], s[70:71], 16
@@ -43996,353 +44007,343 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_lshrrev_b32_e32 v42, 24, v2
; SI-NEXT: v_lshrrev_b32_e32 v43, 16, v2
; SI-NEXT: v_lshrrev_b32_e32 v44, 8, v2
-; SI-NEXT: v_lshrrev_b32_e32 v45, 24, v5
-; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v5
-; SI-NEXT: v_lshrrev_b32_e32 v47, 8, v5
-; SI-NEXT: v_lshrrev_b32_e32 v56, 24, v7
-; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v7
-; SI-NEXT: v_lshrrev_b32_e32 v58, 8, v7
-; SI-NEXT: v_lshrrev_b32_e32 v59, 24, v9
-; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v9
-; SI-NEXT: v_lshrrev_b32_e32 v28, 8, v9
-; SI-NEXT: v_lshrrev_b32_e32 v30, 24, v11
-; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v11
-; SI-NEXT: v_lshrrev_b32_e32 v35, 8, v11
-; SI-NEXT: v_lshrrev_b32_e32 v36, 24, v13
-; SI-NEXT: v_lshrrev_b32_e32 v37, 16, v13
-; SI-NEXT: v_lshrrev_b32_e32 v39, 8, v13
-; SI-NEXT: v_lshrrev_b32_e32 v48, 24, v15
-; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v15
+; SI-NEXT: v_lshrrev_b32_e32 v45, 24, v4
+; SI-NEXT: v_lshrrev_b32_e32 v46, 16, v4
+; SI-NEXT: v_lshrrev_b32_e32 v47, 8, v4
+; SI-NEXT: v_lshrrev_b32_e32 v56, 24, v6
+; SI-NEXT: v_lshrrev_b32_e32 v57, 16, v6
+; SI-NEXT: v_lshrrev_b32_e32 v58, 8, v6
+; SI-NEXT: v_lshrrev_b32_e32 v59, 24, v8
+; SI-NEXT: v_lshrrev_b32_e32 v60, 16, v8
+; SI-NEXT: v_lshrrev_b32_e32 v30, 8, v8
+; SI-NEXT: v_lshrrev_b32_e32 v31, 24, v10
+; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v10
+; SI-NEXT: v_lshrrev_b32_e32 v35, 8, v10
+; SI-NEXT: v_lshrrev_b32_e32 v37, 24, v12
+; SI-NEXT: v_lshrrev_b32_e32 v38, 16, v12
+; SI-NEXT: v_lshrrev_b32_e32 v39, 8, v12
+; SI-NEXT: v_lshrrev_b32_e32 v48, 24, v14
+; SI-NEXT: v_lshrrev_b32_e32 v49, 16, v14
+; SI-NEXT: v_lshrrev_b32_e32 v50, 8, v14
+; SI-NEXT: v_lshrrev_b32_e32 v51, 24, v16
; SI-NEXT: buffer_store_dword v3, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v25
-; SI-NEXT: v_lshrrev_b32_e32 v50, 8, v25
-; SI-NEXT: v_lshrrev_b32_e32 v4, 24, v27
-; SI-NEXT: v_lshrrev_b32_e32 v51, 16, v27
-; SI-NEXT: v_lshrrev_b32_e32 v52, 8, v27
-; SI-NEXT: v_lshrrev_b32_e32 v6, 24, v31
-; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v31
-; SI-NEXT: v_lshrrev_b32_e32 v53, 8, v31
-; SI-NEXT: v_lshrrev_b32_e32 v54, 24, v34
-; SI-NEXT: v_lshrrev_b32_e32 v55, 16, v34
-; SI-NEXT: v_lshrrev_b32_e32 v10, 8, v34
+; SI-NEXT: v_lshrrev_b32_e32 v3, 24, v27
+; SI-NEXT: v_lshrrev_b32_e32 v52, 16, v27
+; SI-NEXT: v_lshrrev_b32_e32 v5, 8, v27
+; SI-NEXT: v_lshrrev_b32_e32 v7, 24, v29
+; SI-NEXT: v_lshrrev_b32_e32 v53, 16, v29
+; SI-NEXT: v_lshrrev_b32_e32 v54, 8, v29
+; SI-NEXT: v_lshrrev_b32_e32 v55, 24, v34
+; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v34
+; SI-NEXT: v_lshrrev_b32_e32 v11, 8, v34
; SI-NEXT: s_branch .LBB37_6
; SI-NEXT: .LBB37_5:
; SI-NEXT: v_mov_b32_e32 v1, s16
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:132 ; 4-byte Folded Spill
-; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:132 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s38
+; SI-NEXT: v_mov_b32_e32 v1, s12
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s36
+; SI-NEXT: v_mov_b32_e32 v1, s10
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s34
+; SI-NEXT: v_mov_b32_e32 v1, s8
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v1, s30
+; SI-NEXT: v_mov_b32_e32 v1, s6
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 58
-; SI-NEXT: v_mov_b32_e32 v10, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 59
-; SI-NEXT: v_mov_b32_e32 v55, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 60
-; SI-NEXT: v_mov_b32_e32 v53, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 61
-; SI-NEXT: v_mov_b32_e32 v8, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 62
-; SI-NEXT: v_mov_b32_e32 v6, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 63
-; SI-NEXT: v_mov_b32_e32 v52, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 0
-; SI-NEXT: v_mov_b32_e32 v51, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 1
-; SI-NEXT: v_mov_b32_e32 v4, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 2
-; SI-NEXT: v_mov_b32_e32 v50, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 3
-; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 4
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v11, s4
; SI-NEXT: v_readlane_b32 s4, v62, 5
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v9, s4
; SI-NEXT: v_readlane_b32 s4, v62, 6
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v54, s4
; SI-NEXT: v_readlane_b32 s4, v62, 7
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v53, s4
; SI-NEXT: v_readlane_b32 s4, v62, 8
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v7, s4
; SI-NEXT: v_readlane_b32 s4, v62, 9
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v5, s4
; SI-NEXT: v_readlane_b32 s4, v62, 10
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v52, s4
; SI-NEXT: v_readlane_b32 s4, v62, 11
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 12
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 13
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:128 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 14
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:124 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:120 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s66
; SI-NEXT: v_readlane_b32 s4, v62, 15
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:116 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
; SI-NEXT: v_readlane_b32 s4, v62, 16
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:112 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s4
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:108 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s69
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:104 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s67
; SI-NEXT: v_readlane_b32 s4, v62, 17
-; SI-NEXT: v_mov_b32_e32 v49, s4
-; SI-NEXT: v_readlane_b32 s4, v62, 18
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:52 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v1, s5
-; SI-NEXT: v_mov_b32_e32 v48, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 8
-; SI-NEXT: v_readlane_b32 s5, v61, 9
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:100 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 30
+; SI-NEXT: v_readlane_b32 s5, v61, 31
; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 6
-; SI-NEXT: v_readlane_b32 s5, v61, 7
-; SI-NEXT: v_mov_b32_e32 v42, s10
+; SI-NEXT: v_readlane_b32 s4, v61, 28
+; SI-NEXT: v_readlane_b32 s5, v61, 29
+; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 26
+; SI-NEXT: v_readlane_b32 s5, v61, 27
; SI-NEXT: s_mov_b32 s10, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 4
-; SI-NEXT: v_readlane_b32 s5, v61, 5
-; SI-NEXT: v_mov_b32_e32 v43, s14
-; SI-NEXT: s_mov_b32 s14, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 2
-; SI-NEXT: v_readlane_b32 s5, v61, 3
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s80
+; SI-NEXT: v_readlane_b32 s4, v61, 24
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:96 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s80
+; SI-NEXT: v_readlane_b32 s5, v61, 25
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:92 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s70
+; SI-NEXT: s_mov_b32 s12, s4
+; SI-NEXT: v_readlane_b32 s4, v61, 22
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:88 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s68
+; SI-NEXT: v_readlane_b32 s5, v61, 23
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:84 ; 4-byte Folded Spill
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_mov_b32_e32 v13, s83
; SI-NEXT: s_mov_b32 s16, s4
-; SI-NEXT: v_readlane_b32 s4, v61, 0
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: v_readlane_b32 s4, v61, 20
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s15
-; SI-NEXT: v_readlane_b32 s5, v61, 1
+; SI-NEXT: v_mov_b32_e32 v13, s81
+; SI-NEXT: v_mov_b32_e32 v40, s54
+; SI-NEXT: v_readlane_b32 s5, v61, 21
; SI-NEXT: v_mov_b32_e32 v34, s17
-; SI-NEXT: v_mov_b32_e32 v31, s19
+; SI-NEXT: v_mov_b32_e32 v29, s19
; SI-NEXT: v_mov_b32_e32 v27, s21
; SI-NEXT: v_mov_b32_e32 v25, s23
; SI-NEXT: v_mov_b32_e32 v23, s25
; SI-NEXT: v_mov_b32_e32 v21, s27
-; SI-NEXT: v_mov_b32_e32 v19, s29
-; SI-NEXT: v_mov_b32_e32 v17, s55
-; SI-NEXT: v_mov_b32_e32 v15, s53
-; SI-NEXT: v_mov_b32_e32 v13, s51
-; SI-NEXT: v_mov_b32_e32 v11, s49
-; SI-NEXT: v_mov_b32_e32 v9, s39
-; SI-NEXT: v_mov_b32_e32 v7, s37
-; SI-NEXT: v_mov_b32_e32 v5, s35
-; SI-NEXT: v_mov_b32_e32 v2, s31
-; SI-NEXT: v_mov_b32_e32 v54, s66
-; SI-NEXT: v_mov_b32_e32 v35, s86
-; SI-NEXT: v_mov_b32_e32 v32, s84
-; SI-NEXT: v_mov_b32_e32 v30, s82
-; SI-NEXT: v_mov_b32_e32 v56, s96
-; SI-NEXT: v_mov_b32_e32 v47, s11
-; SI-NEXT: v_mov_b32_e32 v46, s7
-; SI-NEXT: v_mov_b32_e32 v45, s98
-; SI-NEXT: v_mov_b32_e32 v44, s68
-; SI-NEXT: v_mov_b32_e32 v41, s70
-; SI-NEXT: buffer_store_dword v12, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v18, s29
+; SI-NEXT: v_mov_b32_e32 v16, s95
+; SI-NEXT: v_mov_b32_e32 v14, s63
+; SI-NEXT: v_mov_b32_e32 v12, s75
+; SI-NEXT: v_mov_b32_e32 v10, s57
+; SI-NEXT: v_mov_b32_e32 v8, s13
+; SI-NEXT: v_mov_b32_e32 v6, s11
+; SI-NEXT: v_mov_b32_e32 v4, s9
+; SI-NEXT: v_mov_b32_e32 v2, s7
+; SI-NEXT: v_mov_b32_e32 v55, s34
+; SI-NEXT: buffer_store_dword v13, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
+; SI-NEXT: v_mov_b32_e32 v50, s86
+; SI-NEXT: v_mov_b32_e32 v49, s84
+; SI-NEXT: v_mov_b32_e32 v48, s82
+; SI-NEXT: v_mov_b32_e32 v35, s35
+; SI-NEXT: v_mov_b32_e32 v32, s98
+; SI-NEXT: v_mov_b32_e32 v31, s96
+; SI-NEXT: v_mov_b32_e32 v30, s38
+; SI-NEXT: v_mov_b32_e32 v60, s36
+; SI-NEXT: v_mov_b32_e32 v58, s49
+; SI-NEXT: v_mov_b32_e32 v57, s39
+; SI-NEXT: v_mov_b32_e32 v56, s37
+; SI-NEXT: v_mov_b32_e32 v47, s52
+; SI-NEXT: v_mov_b32_e32 v46, s50
+; SI-NEXT: v_mov_b32_e32 v45, s48
+; SI-NEXT: v_mov_b32_e32 v44, s55
+; SI-NEXT: v_mov_b32_e32 v43, s53
+; SI-NEXT: v_mov_b32_e32 v42, s51
+; SI-NEXT: buffer_store_dword v40, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v12, s48
-; SI-NEXT: v_mov_b32_e32 v14, s50
-; SI-NEXT: v_mov_b32_e32 v16, s52
-; SI-NEXT: v_mov_b32_e32 v18, s54
+; SI-NEXT: v_mov_b32_e32 v40, s64
+; SI-NEXT: v_mov_b32_e32 v41, s65
+; SI-NEXT: v_mov_b32_e32 v13, s56
+; SI-NEXT: v_mov_b32_e32 v15, s74
+; SI-NEXT: v_mov_b32_e32 v17, s62
+; SI-NEXT: v_mov_b32_e32 v19, s94
; SI-NEXT: v_mov_b32_e32 v20, s28
; SI-NEXT: v_mov_b32_e32 v22, s26
; SI-NEXT: v_mov_b32_e32 v24, s24
; SI-NEXT: v_mov_b32_e32 v26, s22
-; SI-NEXT: v_mov_b32_e32 v29, s20
+; SI-NEXT: v_mov_b32_e32 v28, s20
; SI-NEXT: v_mov_b32_e32 v33, s18
-; SI-NEXT: v_readlane_b32 s30, v61, 24
-; SI-NEXT: v_readlane_b32 s34, v61, 22
-; SI-NEXT: v_readlane_b32 s36, v61, 20
-; SI-NEXT: v_readlane_b32 s38, v61, 18
-; SI-NEXT: v_readlane_b32 s48, v61, 16
-; SI-NEXT: v_readlane_b32 s50, v61, 14
-; SI-NEXT: v_readlane_b32 s52, v61, 12
-; SI-NEXT: v_readlane_b32 s54, v61, 10
-; SI-NEXT: s_mov_b32 s64, s76
-; SI-NEXT: s_mov_b32 s76, s6
-; SI-NEXT: s_mov_b32 s66, s58
-; SI-NEXT: s_mov_b32 s68, s60
-; SI-NEXT: s_mov_b32 s70, s78
-; SI-NEXT: s_mov_b32 s78, s10
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s88, s14
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s90, s16
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s92, s4
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s94, s74
+; SI-NEXT: v_readlane_b32 s94, v61, 32
+; SI-NEXT: v_readlane_b32 s30, v61, 18
+; SI-NEXT: v_readlane_b32 s34, v61, 16
+; SI-NEXT: v_readlane_b32 s36, v61, 14
+; SI-NEXT: v_readlane_b32 s38, v61, 12
+; SI-NEXT: v_readlane_b32 s48, v61, 10
+; SI-NEXT: v_readlane_b32 s50, v61, 8
+; SI-NEXT: v_readlane_b32 s52, v61, 6
+; SI-NEXT: v_readlane_b32 s54, v61, 4
+; SI-NEXT: v_readlane_b32 s64, v61, 2
+; SI-NEXT: v_readlane_b32 s66, v61, 0
+; SI-NEXT: s_mov_b32 s68, s46
+; SI-NEXT: s_mov_b32 s70, s42
+; SI-NEXT: s_mov_b32 s80, s78
+; SI-NEXT: s_mov_b32 s78, s8
+; SI-NEXT: s_mov_b32 s82, s88
+; SI-NEXT: s_mov_b32 s88, s10
+; SI-NEXT: s_mov_b32 s84, s90
+; SI-NEXT: s_mov_b32 s90, s12
+; SI-NEXT: s_mov_b32 s86, s92
+; SI-NEXT: s_mov_b32 s92, s16
; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s46
-; SI-NEXT: s_mov_b32 s98, s42
-; SI-NEXT: v_readlane_b32 s4, v61, 56
-; SI-NEXT: v_readlane_b32 s6, v61, 54
-; SI-NEXT: v_readlane_b32 s10, v61, 52
-; SI-NEXT: v_readlane_b32 s14, v61, 50
-; SI-NEXT: v_readlane_b32 s16, v61, 48
-; SI-NEXT: v_readlane_b32 s18, v61, 46
-; SI-NEXT: v_readlane_b32 s20, v61, 44
-; SI-NEXT: v_readlane_b32 s22, v61, 42
-; SI-NEXT: v_readlane_b32 s24, v61, 40
-; SI-NEXT: v_readlane_b32 s26, v61, 38
-; SI-NEXT: v_readlane_b32 s28, v61, 36
-; SI-NEXT: v_readlane_b32 s42, v61, 34
-; SI-NEXT: v_readlane_b32 s46, v61, 32
-; SI-NEXT: v_readlane_b32 s58, v61, 30
-; SI-NEXT: v_readlane_b32 s60, v61, 28
-; SI-NEXT: v_readlane_b32 s74, v61, 26
-; SI-NEXT: v_mov_b32_e32 v39, s83
-; SI-NEXT: v_mov_b32_e32 v37, s81
-; SI-NEXT: v_mov_b32_e32 v36, s71
-; SI-NEXT: v_mov_b32_e32 v28, s97
-; SI-NEXT: v_mov_b32_e32 v60, s87
-; SI-NEXT: v_mov_b32_e32 v59, s85
-; SI-NEXT: v_mov_b32_e32 v58, s99
-; SI-NEXT: v_mov_b32_e32 v57, s67
-; SI-NEXT: v_mov_b32_e32 v40, s69
-; SI-NEXT: v_readlane_b32 s31, v61, 25
-; SI-NEXT: v_readlane_b32 s35, v61, 23
-; SI-NEXT: v_readlane_b32 s37, v61, 21
-; SI-NEXT: v_readlane_b32 s39, v61, 19
-; SI-NEXT: v_readlane_b32 s49, v61, 17
-; SI-NEXT: v_readlane_b32 s51, v61, 15
-; SI-NEXT: v_readlane_b32 s53, v61, 13
-; SI-NEXT: v_readlane_b32 s55, v61, 11
-; SI-NEXT: v_readlane_b32 s5, v61, 57
-; SI-NEXT: v_readlane_b32 s7, v61, 55
-; SI-NEXT: v_readlane_b32 s11, v61, 53
-; SI-NEXT: v_readlane_b32 s15, v61, 51
-; SI-NEXT: v_readlane_b32 s17, v61, 49
-; SI-NEXT: v_readlane_b32 s19, v61, 47
-; SI-NEXT: v_readlane_b32 s21, v61, 45
-; SI-NEXT: v_readlane_b32 s23, v61, 43
-; SI-NEXT: v_readlane_b32 s25, v61, 41
-; SI-NEXT: v_readlane_b32 s27, v61, 39
-; SI-NEXT: v_readlane_b32 s29, v61, 37
-; SI-NEXT: v_readlane_b32 s43, v61, 35
-; SI-NEXT: v_readlane_b32 s47, v61, 33
-; SI-NEXT: v_readlane_b32 s59, v61, 31
-; SI-NEXT: v_readlane_b32 s61, v61, 29
-; SI-NEXT: v_readlane_b32 s75, v61, 27
+; SI-NEXT: s_mov_b32 vcc_lo, s4
+; SI-NEXT: s_mov_b32 s98, s76
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: v_readlane_b32 s4, v62, 2
+; SI-NEXT: v_readlane_b32 s6, v62, 0
+; SI-NEXT: v_readlane_b32 s8, v61, 62
+; SI-NEXT: v_readlane_b32 s10, v61, 60
+; SI-NEXT: v_readlane_b32 s12, v61, 58
+; SI-NEXT: v_readlane_b32 s16, v61, 56
+; SI-NEXT: v_readlane_b32 s18, v61, 54
+; SI-NEXT: v_readlane_b32 s20, v61, 52
+; SI-NEXT: v_readlane_b32 s22, v61, 50
+; SI-NEXT: v_readlane_b32 s24, v61, 48
+; SI-NEXT: v_readlane_b32 s26, v61, 46
+; SI-NEXT: v_readlane_b32 s28, v61, 44
+; SI-NEXT: v_readlane_b32 s42, v61, 42
+; SI-NEXT: v_readlane_b32 s46, v61, 40
+; SI-NEXT: v_readlane_b32 s56, v61, 38
+; SI-NEXT: v_readlane_b32 s62, v61, 36
+; SI-NEXT: v_readlane_b32 s74, v61, 34
+; SI-NEXT: v_mov_b32_e32 v51, s71
+; SI-NEXT: v_mov_b32_e32 v39, s97
+; SI-NEXT: v_mov_b32_e32 v38, s87
+; SI-NEXT: v_mov_b32_e32 v37, s85
+; SI-NEXT: v_mov_b32_e32 v59, s99
+; SI-NEXT: v_readlane_b32 s95, v61, 33
+; SI-NEXT: v_readlane_b32 s31, v61, 19
+; SI-NEXT: v_readlane_b32 s35, v61, 17
+; SI-NEXT: v_readlane_b32 s37, v61, 15
+; SI-NEXT: v_readlane_b32 s39, v61, 13
+; SI-NEXT: v_readlane_b32 s49, v61, 11
+; SI-NEXT: v_readlane_b32 s51, v61, 9
+; SI-NEXT: v_readlane_b32 s53, v61, 7
+; SI-NEXT: v_readlane_b32 s55, v61, 5
+; SI-NEXT: v_readlane_b32 s65, v61, 3
+; SI-NEXT: v_readlane_b32 s67, v61, 1
+; SI-NEXT: v_readlane_b32 s5, v62, 3
+; SI-NEXT: v_readlane_b32 s7, v62, 1
+; SI-NEXT: v_readlane_b32 s9, v61, 63
+; SI-NEXT: v_readlane_b32 s11, v61, 61
+; SI-NEXT: v_readlane_b32 s13, v61, 59
+; SI-NEXT: v_readlane_b32 s17, v61, 57
+; SI-NEXT: v_readlane_b32 s19, v61, 55
+; SI-NEXT: v_readlane_b32 s21, v61, 53
+; SI-NEXT: v_readlane_b32 s23, v61, 51
+; SI-NEXT: v_readlane_b32 s25, v61, 49
+; SI-NEXT: v_readlane_b32 s27, v61, 47
+; SI-NEXT: v_readlane_b32 s29, v61, 45
+; SI-NEXT: v_readlane_b32 s43, v61, 43
+; SI-NEXT: v_readlane_b32 s47, v61, 41
+; SI-NEXT: v_readlane_b32 s57, v61, 39
+; SI-NEXT: v_readlane_b32 s63, v61, 37
+; SI-NEXT: v_readlane_b32 s75, v61, 35
; SI-NEXT: .LBB37_6: ; %end
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_and_b32_e32 v38, 0xff, v38
+; SI-NEXT: v_and_b32_e32 v36, 0xff, v36
; SI-NEXT: s_lshl_b32 s5, s98, 8
-; SI-NEXT: v_or_b32_e32 v38, s5, v38
+; SI-NEXT: v_and_b32_e32 v34, 0xff, v34
+; SI-NEXT: v_lshlrev_b32_e32 v11, 8, v11
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v9
+; SI-NEXT: v_or_b32_e32 v36, s5, v36
; SI-NEXT: s_and_b32 s5, s96, 0xff
+; SI-NEXT: v_or_b32_e32 v11, v34, v11
+; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
+; SI-NEXT: v_lshlrev_b32_e32 v34, 24, v55
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s86, 24
-; SI-NEXT: v_and_b32_e32 v38, 0xffff, v38
+; SI-NEXT: v_or_b32_e32 v9, v34, v9
+; SI-NEXT: v_and_b32_e32 v11, 0xffff, v11
+; SI-NEXT: v_and_b32_e32 v36, 0xffff, v36
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_and_b32_e32 v34, 0xff, v34
-; SI-NEXT: v_lshlrev_b32_e32 v10, 8, v10
-; SI-NEXT: v_or_b32_e32 v38, s5, v38
-; SI-NEXT: v_or_b32_e32 v10, v34, v10
-; SI-NEXT: v_and_b32_e32 v34, 0xff, v55
-; SI-NEXT: buffer_store_dword v38, v0, s[0:3], 0 offen
-; SI-NEXT: v_lshlrev_b32_e32 v34, 16, v34
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v38, 24, v54
-; SI-NEXT: v_or_b32_e32 v34, v38, v34
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
-; SI-NEXT: v_or_b32_e32 v10, v10, v34
-; SI-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:4
+; SI-NEXT: v_or_b32_e32 v9, v11, v9
+; SI-NEXT: v_or_b32_e32 v36, s5, v36
+; SI-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:4
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v10, 0xff, v33
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v33
; SI-NEXT: s_lshl_b32 s5, s84, 8
-; SI-NEXT: v_or_b32_e32 v10, s5, v10
+; SI-NEXT: v_or_b32_e32 v9, s5, v9
; SI-NEXT: s_and_b32 s5, s82, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s80, 24
-; SI-NEXT: v_and_b32_e32 v10, 0xffff, v10
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v10, s5, v10
-; SI-NEXT: buffer_store_dword v10, v0, s[0:3], 0 offen offset:8
+; SI-NEXT: v_or_b32_e32 v9, s5, v9
+; SI-NEXT: buffer_store_dword v9, v0, s[0:3], 0 offen offset:8
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v10, 0xff, v31
-; SI-NEXT: v_lshlrev_b32_e32 v31, 8, v53
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v8
-; SI-NEXT: v_or_b32_e32 v10, v10, v31
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
-; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xffff, v10
-; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:12
+; SI-NEXT: v_and_b32_e32 v9, 0xff, v29
+; SI-NEXT: v_lshlrev_b32_e32 v11, 8, v54
+; SI-NEXT: v_or_b32_e32 v9, v9, v11
+; SI-NEXT: v_and_b32_e32 v11, 0xff, v53
+; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
+; SI-NEXT: v_or_b32_e32 v7, v7, v11
+; SI-NEXT: v_and_b32_e32 v9, 0xffff, v9
+; SI-NEXT: v_or_b32_e32 v7, v9, v7
+; SI-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:12
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v29
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v28
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: v_or_b32_e32 v6, s5, v6
+; SI-NEXT: v_or_b32_e32 v7, s5, v7
; SI-NEXT: s_and_b32 s5, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: s_lshl_b32 s7, s66, 24
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
+; SI-NEXT: v_and_b32_e32 v7, 0xffff, v7
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v6, s5, v6
-; SI-NEXT: buffer_store_dword v6, v0, s[0:3], 0 offen offset:16
+; SI-NEXT: v_or_b32_e32 v7, s5, v7
+; SI-NEXT: buffer_store_dword v7, v0, s[0:3], 0 offen offset:16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v6, 0xff, v27
-; SI-NEXT: v_lshlrev_b32_e32 v8, 8, v52
-; SI-NEXT: v_or_b32_e32 v6, v6, v8
-; SI-NEXT: v_and_b32_e32 v8, 0xff, v51
-; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
-; SI-NEXT: v_lshlrev_b32_e32 v4, 24, v4
-; SI-NEXT: v_or_b32_e32 v4, v4, v8
-; SI-NEXT: v_and_b32_e32 v6, 0xffff, v6
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:20
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v27
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_and_b32_e32 v7, 0xff, v52
+; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
+; SI-NEXT: v_lshlrev_b32_e32 v3, 24, v3
+; SI-NEXT: v_or_b32_e32 v3, v3, v7
+; SI-NEXT: v_and_b32_e32 v5, 0xffff, v5
+; SI-NEXT: v_or_b32_e32 v3, v5, v3
+; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:20
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v26
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v26
; SI-NEXT: s_lshl_b32 s5, s64, 8
-; SI-NEXT: v_or_b32_e32 v4, s5, v4
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xff
+; SI-NEXT: v_or_b32_e32 v3, s5, v3
+; SI-NEXT: s_and_b32 s5, s54, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s7, s94, 24
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
+; SI-NEXT: s_lshl_b32 s7, s52, 24
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: v_or_b32_e32 v4, s5, v4
-; SI-NEXT: buffer_store_dword v4, v0, s[0:3], 0 offen offset:24
+; SI-NEXT: v_or_b32_e32 v3, s5, v3
+; SI-NEXT: buffer_store_dword v36, v0, s[0:3], 0 offen
+; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:24
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v25
-; SI-NEXT: v_lshlrev_b32_e32 v6, 8, v50
-; SI-NEXT: v_or_b32_e32 v4, v4, v6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:128 ; 4-byte Folded Reload
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
-; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v3
-; SI-NEXT: v_and_b32_e32 v4, 0xffff, v4
-; SI-NEXT: s_lshl_b32 s5, s92, 8
-; SI-NEXT: s_lshl_b32 s7, s88, 24
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v25
+; SI-NEXT: s_lshl_b32 s5, s50, 8
+; SI-NEXT: s_lshl_b32 s7, s38, 24
; SI-NEXT: v_and_b32_e32 v2, 0xff, v2
; SI-NEXT: s_lshl_b32 s4, s4, 24
; SI-NEXT: v_and_b32_e32 v1, 0xff, v1
@@ -44367,140 +44368,147 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_readlane_b32 s65, v63, 15
; SI-NEXT: v_readlane_b32 s64, v63, 14
; SI-NEXT: v_readlane_b32 s55, v63, 13
+; SI-NEXT: v_readlane_b32 s54, v63, 12
; SI-NEXT: v_readlane_b32 s53, v63, 11
+; SI-NEXT: v_readlane_b32 s52, v63, 10
; SI-NEXT: v_readlane_b32 s51, v63, 9
+; SI-NEXT: v_readlane_b32 s50, v63, 8
; SI-NEXT: v_readlane_b32 s49, v63, 7
; SI-NEXT: v_readlane_b32 s39, v63, 5
+; SI-NEXT: v_readlane_b32 s38, v63, 4
; SI-NEXT: v_readlane_b32 s37, v63, 3
; SI-NEXT: v_readlane_b32 s35, v63, 1
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
-; SI-NEXT: v_or_b32_e32 v3, v6, v3
-; SI-NEXT: v_or_b32_e32 v3, v4, v3
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:28
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v24
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s90, 0xff
+; SI-NEXT: s_and_b32 s5, s48, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:32
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v23
-; SI-NEXT: s_lshl_b32 s5, s78, 8
-; SI-NEXT: s_lshl_b32 s7, s54, 24
-; SI-NEXT: v_readlane_b32 s54, v63, 12
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:116 ; 4-byte Folded Reload
+; SI-NEXT: s_lshl_b32 s5, s36, 8
+; SI-NEXT: s_lshl_b32 s7, s30, 24
+; SI-NEXT: v_readlane_b32 s30, v63, 34
+; SI-NEXT: v_readlane_b32 s31, v63, 35
+; SI-NEXT: v_readlane_b32 s48, v63, 6
+; SI-NEXT: v_readlane_b32 s36, v63, 2
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:120 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:36
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v22
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s76, 0xff
+; SI-NEXT: s_and_b32 s5, s34, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:40
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:112 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v21
-; SI-NEXT: s_lshl_b32 s5, s52, 8
-; SI-NEXT: s_lshl_b32 s7, s48, 24
-; SI-NEXT: v_readlane_b32 s52, v63, 10
-; SI-NEXT: v_readlane_b32 s48, v63, 6
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:104 ; 4-byte Folded Reload
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 8
+; SI-NEXT: s_lshl_b32 s7, s90, 24
+; SI-NEXT: v_readlane_b32 s34, v63, 0
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:108 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:44
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v20
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s50, 0xff
+; SI-NEXT: s_and_b32 s5, s92, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:48
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:100 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v19
-; SI-NEXT: s_lshl_b32 s5, s38, 8
-; SI-NEXT: s_lshl_b32 s7, s34, 24
-; SI-NEXT: v_readlane_b32 s50, v63, 8
-; SI-NEXT: v_readlane_b32 s38, v63, 4
-; SI-NEXT: v_readlane_b32 s34, v63, 0
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:92 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v18
+; SI-NEXT: s_lshl_b32 s5, s88, 8
+; SI-NEXT: s_lshl_b32 s7, s76, 24
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:96 ; 4-byte Folded Reload
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v7
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:52
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v18
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v19
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s36, 0xff
+; SI-NEXT: s_and_b32 s5, s78, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:56
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v17
-; SI-NEXT: s_lshl_b32 s5, s30, 8
-; SI-NEXT: s_lshl_b32 s7, s60, 24
-; SI-NEXT: v_readlane_b32 s30, v63, 34
-; SI-NEXT: v_readlane_b32 s31, v63, 35
-; SI-NEXT: v_readlane_b32 s36, v63, 2
-; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v16
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v51
+; SI-NEXT: s_lshl_b32 s5, s94, 8
+; SI-NEXT: s_lshl_b32 s7, s62, 24
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v6
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v4
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v5
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:60
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v16
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v17
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: s_and_b32 s5, s74, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
@@ -44508,69 +44516,67 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:64
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:76 ; 4-byte Folded Reload
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v15
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v48
-; SI-NEXT: s_lshl_b32 s5, s58, 8
-; SI-NEXT: s_lshl_b32 s7, s42, 24
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v49
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v14
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v50
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v49
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v48
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:68
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v14
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v15
+; SI-NEXT: s_lshl_b32 s5, s56, 8
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: s_and_b32 s5, s46, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s7, s42, 24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:72
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v13
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v39
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v37
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v36
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v12
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v39
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v38
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v37
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:76
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v12
-; SI-NEXT: s_lshl_b32 s5, s62, 8
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v13
+; SI-NEXT: s_lshl_b32 s5, s44, 8
; SI-NEXT: v_or_b32_e32 v3, s5, v3
-; SI-NEXT: s_and_b32 s5, s56, 0xff
+; SI-NEXT: s_and_b32 s5, s72, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s7, s44, 24
+; SI-NEXT: s_lshl_b32 s7, s60, 24
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: s_or_b32 s5, s7, s5
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:80
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v11
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v35
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v32
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v30
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v10
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v35
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v32
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v31
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:84
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:68 ; 4-byte Folded Reload
-; SI-NEXT: s_lshl_b32 s5, s72, 8
+; SI-NEXT: s_lshl_b32 s5, s58, 8
; SI-NEXT: s_lshl_b32 s7, s28, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v28
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v59
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v30
+; SI-NEXT: v_lshlrev_b32_e32 v7, 24, v59
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44581,20 +44587,19 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:88
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v9
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v60
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v8
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v60
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v7, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:92
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:64 ; 4-byte Folded Reload
; SI-NEXT: s_lshl_b32 s5, s26, 8
; SI-NEXT: s_lshl_b32 s7, s22, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v58
-; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v56
+; SI-NEXT: v_lshlrev_b32_e32 v5, 8, v58
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44605,19 +44610,20 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:96
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v7
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
-; SI-NEXT: v_and_b32_e32 v4, 0xff, v57
-; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v6
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
+; SI-NEXT: v_and_b32_e32 v5, 0xff, v57
+; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
+; SI-NEXT: v_lshlrev_b32_e32 v6, 24, v56
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
-; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_or_b32_e32 v3, v3, v4
+; SI-NEXT: v_or_b32_e32 v5, v6, v5
+; SI-NEXT: v_or_b32_e32 v3, v3, v5
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:100
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:60 ; 4-byte Folded Reload
; SI-NEXT: s_lshl_b32 s5, s20, 8
; SI-NEXT: s_lshl_b32 s7, s16, 24
-; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v47
+; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v45
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_and_b32_e32 v3, 0xff, v3
; SI-NEXT: v_or_b32_e32 v3, s5, v3
@@ -44628,11 +44634,11 @@ define inreg <128 x i8> @bitcast_v32f32_to_v128i8_scalar(<32 x float> inreg %a,
; SI-NEXT: v_or_b32_e32 v3, s5, v3
; SI-NEXT: buffer_store_dword v3, v0, s[0:3], 0 offen offset:104
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_and_b32_e32 v3, 0xff, v5
+; SI-NEXT: v_and_b32_e32 v3, 0xff, v4
+; SI-NEXT: v_lshlrev_b32_e32 v4, 8, v47
; SI-NEXT: v_or_b32_e32 v3, v3, v4
; SI-NEXT: v_and_b32_e32 v4, 0xff, v46
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
-; SI-NEXT: v_lshlrev_b32_e32 v5, 24, v45
; SI-NEXT: v_and_b32_e32 v3, 0xffff, v3
; SI-NEXT: v_or_b32_e32 v4, v5, v4
; SI-NEXT: v_or_b32_e32 v3, v3, v4
@@ -68900,183 +68906,213 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -69085,9 +69121,9 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -69099,7 +69135,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -69107,11 +69143,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -69122,7 +69158,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -69131,11 +69167,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -69146,7 +69182,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -69155,11 +69191,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -69170,7 +69206,7 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -69179,11 +69215,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -69191,47 +69227,56 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -69239,10 +69284,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -69250,12 +69296,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -69263,10 +69311,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -69274,12 +69323,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -69287,10 +69338,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -69298,12 +69350,14 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -69311,10 +69365,11 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -69330,7 +69385,8 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -69378,24 +69434,8 @@ define inreg <32 x float> @bitcast_v64f16_to_v32f32_scalar(<64 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -72096,195 +72136,215 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB51_3: ; %Flow
@@ -72295,168 +72355,188 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -72489,23 +72569,7 @@ define inreg <32 x float> @bitcast_v64i16_to_v32f32_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -93215,139 +93279,139 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: v_writelane_b32 v33, s55, 13
; SI-NEXT: v_writelane_b32 v33, s64, 14
; SI-NEXT: v_writelane_b32 v33, s65, 15
-; SI-NEXT: v_writelane_b32 v33, s66, 16
-; SI-NEXT: v_writelane_b32 v33, s67, 17
-; SI-NEXT: v_writelane_b32 v33, s68, 18
-; SI-NEXT: v_writelane_b32 v33, s69, 19
-; SI-NEXT: v_writelane_b32 v33, s70, 20
-; SI-NEXT: v_writelane_b32 v33, s71, 21
-; SI-NEXT: v_writelane_b32 v33, s80, 22
-; SI-NEXT: v_writelane_b32 v33, s81, 23
-; SI-NEXT: v_writelane_b32 v33, s82, 24
-; SI-NEXT: v_writelane_b32 v33, s83, 25
-; SI-NEXT: v_writelane_b32 v33, s84, 26
-; SI-NEXT: v_writelane_b32 v33, s85, 27
-; SI-NEXT: v_writelane_b32 v33, s86, 28
-; SI-NEXT: v_writelane_b32 v33, s87, 29
-; SI-NEXT: v_writelane_b32 v33, s96, 30
-; SI-NEXT: v_writelane_b32 v33, s97, 31
-; SI-NEXT: v_writelane_b32 v33, s98, 32
-; SI-NEXT: v_writelane_b32 v33, s30, 33
-; SI-NEXT: v_writelane_b32 v33, s31, 34
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_readfirstlane_b32 s4, v17
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: v_readfirstlane_b32 s31, v15
-; SI-NEXT: v_readfirstlane_b32 s34, v14
-; SI-NEXT: v_readfirstlane_b32 s35, v13
-; SI-NEXT: v_readfirstlane_b32 s36, v12
-; SI-NEXT: v_readfirstlane_b32 s37, v11
-; SI-NEXT: v_readfirstlane_b32 s38, v10
-; SI-NEXT: v_readfirstlane_b32 s39, v9
-; SI-NEXT: v_readfirstlane_b32 s48, v8
-; SI-NEXT: v_readfirstlane_b32 s49, v7
-; SI-NEXT: v_readfirstlane_b32 s50, v6
-; SI-NEXT: v_readfirstlane_b32 s51, v5
-; SI-NEXT: v_readfirstlane_b32 s52, v4
-; SI-NEXT: v_readfirstlane_b32 s53, v3
-; SI-NEXT: v_readfirstlane_b32 s54, v2
-; SI-NEXT: v_readfirstlane_b32 s55, v1
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_readfirstlane_b32 s64, v0
+; SI-NEXT: v_writelane_b32 v33, s30, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 17
+; SI-NEXT: v_readfirstlane_b32 s7, v17
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s9, v16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
+; SI-NEXT: v_writelane_b32 v34, s7, 0
+; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: v_writelane_b32 v34, s9, 1
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v34, s11, 2
+; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v34, s13, 3
+; SI-NEXT: v_readfirstlane_b32 s41, v12
+; SI-NEXT: v_writelane_b32 v34, s15, 4
+; SI-NEXT: v_readfirstlane_b32 s43, v11
+; SI-NEXT: v_writelane_b32 v34, s41, 5
+; SI-NEXT: v_readfirstlane_b32 s45, v10
+; SI-NEXT: v_writelane_b32 v34, s43, 6
+; SI-NEXT: v_readfirstlane_b32 s47, v9
+; SI-NEXT: v_writelane_b32 v34, s45, 7
+; SI-NEXT: v_readfirstlane_b32 s57, v8
+; SI-NEXT: v_writelane_b32 v34, s47, 8
+; SI-NEXT: v_readfirstlane_b32 s59, v7
+; SI-NEXT: v_writelane_b32 v34, s57, 9
+; SI-NEXT: v_readfirstlane_b32 s61, v6
+; SI-NEXT: v_writelane_b32 v34, s59, 10
+; SI-NEXT: v_readfirstlane_b32 s63, v5
+; SI-NEXT: v_writelane_b32 v34, s61, 11
+; SI-NEXT: v_readfirstlane_b32 s73, v4
+; SI-NEXT: v_writelane_b32 v34, s63, 12
+; SI-NEXT: v_readfirstlane_b32 s6, v3
+; SI-NEXT: v_writelane_b32 v34, s73, 13
+; SI-NEXT: v_readfirstlane_b32 s5, v2
+; SI-NEXT: v_writelane_b32 v34, s6, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s5, 15
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: v_writelane_b32 v34, s4, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 17
; SI-NEXT: s_cbranch_scc0 .LBB61_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s4, 16
-; SI-NEXT: s_and_b32 s4, s50, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 2
-; SI-NEXT: s_lshl_b32 s4, s50, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 1
-; SI-NEXT: s_and_b32 s4, s51, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 4
-; SI-NEXT: s_lshl_b32 s4, s51, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 3
-; SI-NEXT: s_and_b32 s4, s52, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 6
-; SI-NEXT: s_lshl_b32 s4, s52, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 5
-; SI-NEXT: s_and_b32 s4, s53, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 8
-; SI-NEXT: s_lshl_b32 s4, s53, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 7
-; SI-NEXT: s_and_b32 s4, s55, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 10
-; SI-NEXT: s_lshl_b32 s4, s55, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 9
-; SI-NEXT: s_and_b32 s4, s64, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 12
-; SI-NEXT: s_lshl_b32 s4, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 11
-; SI-NEXT: s_and_b32 s4, s29, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 14
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 13
-; SI-NEXT: s_and_b32 s4, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 15
+; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s4, 16
; SI-NEXT: s_and_b32 s4, s27, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 18
+; SI-NEXT: v_writelane_b32 v34, s4, 19
; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 17
+; SI-NEXT: v_writelane_b32 v34, s4, 18
; SI-NEXT: s_and_b32 s4, s26, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: v_writelane_b32 v34, s4, 21
; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: s_and_b32 s10, s30, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s30, 16
-; SI-NEXT: s_and_b32 s12, s31, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_and_b32 s14, s34, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s34, 16
-; SI-NEXT: s_and_b32 s40, s35, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s35, 16
-; SI-NEXT: s_and_b32 s42, s36, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s36, 16
-; SI-NEXT: s_and_b32 s44, s37, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s37, 16
-; SI-NEXT: s_and_b32 s46, s38, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s38, 16
-; SI-NEXT: s_and_b32 s56, s39, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s39, 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s48, 16
-; SI-NEXT: s_and_b32 s60, s49, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s49, 16
-; SI-NEXT: s_and_b32 s78, s54, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 19
-; SI-NEXT: s_and_b32 s62, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s25, 16
-; SI-NEXT: s_and_b32 s72, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s24, 16
-; SI-NEXT: s_and_b32 s74, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s23, 16
-; SI-NEXT: s_and_b32 s76, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s75, s22, 16
-; SI-NEXT: s_and_b32 s88, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s21, 16
-; SI-NEXT: s_and_b32 s90, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s20, 16
-; SI-NEXT: s_and_b32 s92, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s19, 16
-; SI-NEXT: s_and_b32 s94, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s18, 16
-; SI-NEXT: s_and_b32 vcc_lo, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s95, s17, 16
-; SI-NEXT: s_and_b32 s6, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s16, 16
+; SI-NEXT: s_and_b32 s55, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_and_b32 s76, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s75, s6, 16
+; SI-NEXT: s_and_b32 s78, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s5, 16
+; SI-NEXT: s_and_b32 s90, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s8, 16
+; SI-NEXT: s_and_b32 s92, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s29, 16
+; SI-NEXT: s_and_b32 s94, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s4, 20
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s6, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s8, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s16, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB61_3
; SI-NEXT: .LBB61_2:
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr50
+; SI-NEXT: ; implicit-def: $sgpr49
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr95
@@ -93360,6 +93424,8 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr73
@@ -93368,8 +93434,6 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $sgpr57
@@ -93391,39 +93455,7 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; kill: killed $sgpr4
; SI-NEXT: ; implicit-def: $sgpr4
@@ -93435,68 +93467,43 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s65, s63
-; SI-NEXT: s_mov_b32 s66, s72
-; SI-NEXT: s_mov_b32 s67, s73
-; SI-NEXT: s_mov_b32 s68, s74
-; SI-NEXT: s_mov_b32 s69, s75
-; SI-NEXT: s_mov_b32 s70, s76
-; SI-NEXT: s_mov_b32 s71, s79
-; SI-NEXT: s_mov_b32 s80, s88
-; SI-NEXT: s_mov_b32 s81, s89
-; SI-NEXT: s_mov_b32 s82, s90
-; SI-NEXT: s_mov_b32 s83, s91
-; SI-NEXT: s_mov_b32 s84, s92
-; SI-NEXT: s_mov_b32 s85, s93
-; SI-NEXT: s_mov_b32 s86, s94
-; SI-NEXT: s_mov_b32 s87, s95
-; SI-NEXT: s_mov_b32 s96, vcc_lo
-; SI-NEXT: s_mov_b32 vcc_lo, s61
-; SI-NEXT: s_mov_b32 s97, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_hi, s62
-; SI-NEXT: s_mov_b32 s98, s6
+; SI-NEXT: s_mov_b32 s64, s30
+; SI-NEXT: s_mov_b32 s30, s6
+; SI-NEXT: s_mov_b32 s65, s8
; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s8
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s9, s11
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s11, s13
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s13, s15
-; SI-NEXT: s_mov_b32 s14, s40
-; SI-NEXT: s_mov_b32 s15, s41
-; SI-NEXT: s_mov_b32 s40, s42
-; SI-NEXT: s_mov_b32 s41, s43
-; SI-NEXT: s_mov_b32 s42, s44
-; SI-NEXT: s_mov_b32 s43, s45
-; SI-NEXT: s_mov_b32 s44, s46
-; SI-NEXT: s_mov_b32 s45, s47
-; SI-NEXT: s_mov_b32 s46, s56
-; SI-NEXT: s_mov_b32 s47, s57
-; SI-NEXT: s_mov_b32 s56, s58
-; SI-NEXT: s_mov_b32 s57, s59
-; SI-NEXT: s_mov_b32 s58, s60
-; SI-NEXT: v_readlane_b32 s59, v34, 1
-; SI-NEXT: v_readlane_b32 s60, v34, 2
-; SI-NEXT: v_readlane_b32 s61, v34, 3
-; SI-NEXT: v_readlane_b32 s62, v34, 4
-; SI-NEXT: v_readlane_b32 s63, v34, 5
-; SI-NEXT: v_readlane_b32 s72, v34, 6
-; SI-NEXT: v_readlane_b32 s73, v34, 7
-; SI-NEXT: v_readlane_b32 s74, v34, 8
-; SI-NEXT: v_readlane_b32 s6, v34, 9
-; SI-NEXT: v_readlane_b32 s75, v34, 10
-; SI-NEXT: v_readlane_b32 s76, v34, 11
-; SI-NEXT: v_readlane_b32 s79, v34, 12
-; SI-NEXT: v_readlane_b32 s88, v34, 13
-; SI-NEXT: v_readlane_b32 s89, v34, 14
-; SI-NEXT: v_readlane_b32 s90, v34, 15
-; SI-NEXT: v_readlane_b32 s91, v34, 16
-; SI-NEXT: v_readlane_b32 s92, v34, 17
-; SI-NEXT: v_readlane_b32 s93, v34, 18
-; SI-NEXT: v_readlane_b32 s94, v34, 19
-; SI-NEXT: v_readlane_b32 s95, v34, 20
+; SI-NEXT: s_mov_b32 s5, s55
+; SI-NEXT: s_mov_b32 s55, s9
+; SI-NEXT: s_mov_b32 s9, s10
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s12
+; SI-NEXT: s_mov_b32 s12, s13
+; SI-NEXT: s_mov_b32 s13, s14
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s40, s41
+; SI-NEXT: s_mov_b32 s41, s42
+; SI-NEXT: s_mov_b32 s42, s43
+; SI-NEXT: s_mov_b32 s43, s44
+; SI-NEXT: s_mov_b32 s44, s45
+; SI-NEXT: s_mov_b32 s45, s46
+; SI-NEXT: s_mov_b32 s46, s47
+; SI-NEXT: s_mov_b32 s47, s56
+; SI-NEXT: s_mov_b32 s56, s57
+; SI-NEXT: s_mov_b32 s57, s58
+; SI-NEXT: s_mov_b32 s58, s59
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s61
+; SI-NEXT: s_mov_b32 s61, s62
+; SI-NEXT: s_mov_b32 s62, s63
+; SI-NEXT: s_mov_b32 s63, s72
+; SI-NEXT: s_mov_b32 s72, s73
+; SI-NEXT: s_mov_b32 s73, s74
+; SI-NEXT: s_mov_b32 s74, s75
+; SI-NEXT: s_mov_b32 s75, s76
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: v_readlane_b32 s7, v34, 19
+; SI-NEXT: v_readlane_b32 s8, v34, 20
+; SI-NEXT: v_readlane_b32 s76, v34, 21
; SI-NEXT: s_cbranch_scc1 .LBB61_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_u32 s16, s16, 3
@@ -93509,241 +93516,241 @@ define inreg <64 x bfloat> @bitcast_v16i64_to_v64bf16_scalar(<16 x i64> inreg %a
; SI-NEXT: s_addc_u32 s23, s23, 0
; SI-NEXT: s_add_u32 s24, s24, 3
; SI-NEXT: s_addc_u32 s25, s25, 0
-; SI-NEXT: s_add_u32 s26, s26, 3
-; SI-NEXT: s_addc_u32 s27, s27, 0
-; SI-NEXT: s_add_u32 s28, s28, 3
-; SI-NEXT: s_addc_u32 s29, s29, 0
-; SI-NEXT: s_add_u32 s76, s64, 3
-; SI-NEXT: s_addc_u32 s6, s55, 0
-; SI-NEXT: s_add_u32 s75, s54, 3
-; SI-NEXT: s_addc_u32 s73, s53, 0
-; SI-NEXT: s_add_u32 s63, s52, 3
-; SI-NEXT: s_addc_u32 s61, s51, 0
-; SI-NEXT: s_add_u32 s59, s50, 3
-; SI-NEXT: s_addc_u32 s57, s49, 0
-; SI-NEXT: s_add_u32 s47, s48, 3
-; SI-NEXT: s_addc_u32 s45, s39, 0
-; SI-NEXT: s_add_u32 s43, s38, 3
-; SI-NEXT: s_addc_u32 s41, s37, 0
-; SI-NEXT: s_add_u32 s15, s36, 3
-; SI-NEXT: s_addc_u32 s13, s35, 0
-; SI-NEXT: s_add_u32 s11, s34, 3
-; SI-NEXT: s_addc_u32 s9, s31, 0
-; SI-NEXT: s_add_u32 s7, s30, 3
+; SI-NEXT: s_add_u32 s8, s26, 3
+; SI-NEXT: s_addc_u32 s6, s27, 0
+; SI-NEXT: s_add_u32 s7, s28, 3
+; SI-NEXT: s_addc_u32 s26, s29, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 17
+; SI-NEXT: s_add_u32 s27, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 16
+; SI-NEXT: s_addc_u32 s28, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 15
+; SI-NEXT: s_add_u32 s29, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 14
+; SI-NEXT: s_addc_u32 s74, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 13
+; SI-NEXT: s_add_u32 s72, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 12
+; SI-NEXT: s_addc_u32 s62, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 11
+; SI-NEXT: s_add_u32 s60, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 10
+; SI-NEXT: s_addc_u32 s58, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 9
+; SI-NEXT: s_add_u32 s56, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 8
+; SI-NEXT: s_addc_u32 s46, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 7
+; SI-NEXT: s_add_u32 s44, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 6
+; SI-NEXT: s_addc_u32 s42, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 5
+; SI-NEXT: s_add_u32 s40, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 4
+; SI-NEXT: s_addc_u32 s14, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 3
+; SI-NEXT: s_add_u32 s12, s4, 3
+; SI-NEXT: v_readlane_b32 s4, v34, 2
+; SI-NEXT: s_addc_u32 s10, s4, 0
+; SI-NEXT: v_readlane_b32 s4, v34, 1
+; SI-NEXT: s_add_u32 s11, s4, 3
; SI-NEXT: v_readlane_b32 s4, v34, 0
; SI-NEXT: s_addc_u32 s4, s4, 0
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_and_b32 s8, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s10, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s12, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s14, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_and_b32 s40, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_and_b32 s42, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_and_b32 s44, s43, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_and_b32 s46, s45, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s45, 16
-; SI-NEXT: s_and_b32 s56, s47, 0xffff0000
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_and_b32 s58, s57, 0xffff0000
-; SI-NEXT: s_lshl_b32 s57, s57, 16
-; SI-NEXT: s_and_b32 s60, s59, 0xffff0000
-; SI-NEXT: s_lshl_b32 s59, s59, 16
-; SI-NEXT: s_and_b32 s62, s61, 0xffff0000
-; SI-NEXT: s_lshl_b32 s61, s61, 16
-; SI-NEXT: s_and_b32 s72, s63, 0xffff0000
-; SI-NEXT: s_lshl_b32 s63, s63, 16
-; SI-NEXT: s_and_b32 s74, s73, 0xffff0000
-; SI-NEXT: s_lshl_b32 s73, s73, 16
-; SI-NEXT: s_and_b32 s78, s75, 0xffff0000
-; SI-NEXT: s_lshl_b32 s77, s75, 16
-; SI-NEXT: s_and_b32 s75, s6, 0xffff0000
+; SI-NEXT: s_and_b32 s9, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s55, s11, 16
+; SI-NEXT: s_and_b32 s11, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s13, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s15, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s41, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s43, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s45, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_and_b32 s47, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_and_b32 s57, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_and_b32 s59, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_and_b32 s61, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_and_b32 s63, s62, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_and_b32 s73, s72, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_and_b32 s75, s74, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s74, 16
+; SI-NEXT: s_and_b32 s78, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s29, 16
+; SI-NEXT: s_and_b32 s88, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s28, 16
+; SI-NEXT: s_and_b32 s90, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s27, 16
+; SI-NEXT: s_and_b32 s92, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s91, s26, 16
+; SI-NEXT: s_and_b32 s94, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s7, 16
+; SI-NEXT: s_and_b32 s7, s6, 0xffff0000
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_and_b32 s79, s76, 0xffff0000
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_and_b32 s89, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s29, 16
-; SI-NEXT: s_and_b32 s91, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s28, 16
-; SI-NEXT: s_and_b32 s93, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s27, 16
-; SI-NEXT: s_and_b32 s95, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s26, 16
-; SI-NEXT: s_and_b32 vcc_hi, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s25, 16
-; SI-NEXT: s_and_b32 s66, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s24, 16
-; SI-NEXT: s_and_b32 s68, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s23, 16
-; SI-NEXT: s_and_b32 s70, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s22, 16
-; SI-NEXT: s_and_b32 s80, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s21, 16
-; SI-NEXT: s_and_b32 s82, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s20, 16
-; SI-NEXT: s_and_b32 s84, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s19, 16
-; SI-NEXT: s_and_b32 s86, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s18, 16
-; SI-NEXT: s_and_b32 s96, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s17, 16
-; SI-NEXT: s_and_b32 s98, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s97, s16, 16
+; SI-NEXT: s_and_b32 s76, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_and_b32 vcc_lo, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s25, 16
+; SI-NEXT: s_and_b32 s30, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s24, 16
+; SI-NEXT: s_and_b32 s53, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s23, 16
+; SI-NEXT: s_and_b32 s51, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s22, 16
+; SI-NEXT: s_and_b32 s49, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s21, 16
+; SI-NEXT: s_and_b32 s39, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s20, 16
+; SI-NEXT: s_and_b32 s37, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s19, 16
+; SI-NEXT: s_and_b32 s35, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s18, 16
+; SI-NEXT: s_and_b32 s31, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s17, 16
+; SI-NEXT: s_and_b32 s65, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s16, 16
; SI-NEXT: .LBB61_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s64
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s34
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s36
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s38
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s48
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s50
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s52
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s54
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_hi
-; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
; SI-NEXT: v_mul_f32_e64 v9, 1.0, vcc_lo
+; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s95
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s8
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s93
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s91
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s89
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s79
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
; SI-NEXT: v_mul_f32_e64 v16, 1.0, s77
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s74
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s72
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s63
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s62
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s60
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s59
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s58
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s56
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s47
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s46
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s44
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s42
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s40
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s15
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s14
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s13
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s12
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s11
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s55
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v33, 33
+; SI-NEXT: v_readlane_b32 s30, v33, 16
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
-; SI-NEXT: v_readlane_b32 s31, v33, 34
-; SI-NEXT: v_readlane_b32 s98, v33, 32
-; SI-NEXT: v_readlane_b32 s97, v33, 31
-; SI-NEXT: v_readlane_b32 s96, v33, 30
-; SI-NEXT: v_readlane_b32 s87, v33, 29
-; SI-NEXT: v_readlane_b32 s86, v33, 28
-; SI-NEXT: v_readlane_b32 s85, v33, 27
-; SI-NEXT: v_readlane_b32 s84, v33, 26
-; SI-NEXT: v_readlane_b32 s83, v33, 25
-; SI-NEXT: v_readlane_b32 s82, v33, 24
-; SI-NEXT: v_readlane_b32 s81, v33, 23
-; SI-NEXT: v_readlane_b32 s80, v33, 22
-; SI-NEXT: v_readlane_b32 s71, v33, 21
-; SI-NEXT: v_readlane_b32 s70, v33, 20
-; SI-NEXT: v_readlane_b32 s69, v33, 19
-; SI-NEXT: v_readlane_b32 s68, v33, 18
-; SI-NEXT: v_readlane_b32 s67, v33, 17
-; SI-NEXT: v_readlane_b32 s66, v33, 16
+; SI-NEXT: v_readlane_b32 s31, v33, 17
; SI-NEXT: v_readlane_b32 s65, v33, 15
; SI-NEXT: v_readlane_b32 s64, v33, 14
; SI-NEXT: v_readlane_b32 s55, v33, 13
@@ -103002,183 +103009,213 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB67_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB67_3
; SI-NEXT: .LBB67_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB67_3: ; %Flow
@@ -103187,9 +103224,9 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB67_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -103201,7 +103238,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -103209,11 +103246,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -103224,7 +103261,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -103233,11 +103270,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -103248,7 +103285,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -103257,11 +103294,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -103272,7 +103309,7 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -103281,11 +103318,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -103293,47 +103330,56 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -103341,10 +103387,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -103352,12 +103399,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -103365,10 +103414,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -103376,12 +103426,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -103389,10 +103441,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -103400,12 +103453,14 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -103413,10 +103468,11 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -103432,7 +103488,8 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -103480,24 +103537,8 @@ define inreg <16 x i64> @bitcast_v64f16_to_v16i64_scalar(<64 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB67_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -106098,195 +106139,215 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB71_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB71_3
; SI-NEXT: .LBB71_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB71_3: ; %Flow
@@ -106297,168 +106358,188 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB71_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -106491,23 +106572,7 @@ define inreg <16 x i64> @bitcast_v64i16_to_v16i64_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -136225,183 +136290,213 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_writelane_b32 v34, s65, 15
; SI-NEXT: v_writelane_b32 v34, s66, 16
; SI-NEXT: v_writelane_b32 v34, s67, 17
-; SI-NEXT: v_writelane_b32 v34, s68, 18
-; SI-NEXT: v_writelane_b32 v34, s69, 19
-; SI-NEXT: v_writelane_b32 v34, s70, 20
-; SI-NEXT: v_writelane_b32 v34, s71, 21
-; SI-NEXT: v_writelane_b32 v34, s80, 22
-; SI-NEXT: v_writelane_b32 v34, s81, 23
-; SI-NEXT: v_writelane_b32 v34, s82, 24
-; SI-NEXT: v_writelane_b32 v34, s83, 25
-; SI-NEXT: v_writelane_b32 v34, s84, 26
-; SI-NEXT: v_writelane_b32 v34, s85, 27
-; SI-NEXT: v_writelane_b32 v34, s86, 28
-; SI-NEXT: v_writelane_b32 v34, s87, 29
-; SI-NEXT: v_writelane_b32 v34, s96, 30
-; SI-NEXT: v_writelane_b32 v34, s97, 31
-; SI-NEXT: v_writelane_b32 v34, s98, 32
-; SI-NEXT: v_writelane_b32 v34, s99, 33
-; SI-NEXT: v_writelane_b32 v34, s30, 34
-; SI-NEXT: v_writelane_b32 v34, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s6, v17
+; SI-NEXT: v_writelane_b32 v34, s30, 18
+; SI-NEXT: v_writelane_b32 v34, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: s_lshr_b32 s4, s5, 16
; SI-NEXT: v_readfirstlane_b32 s8, v16
+; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v35, s4, 0
+; SI-NEXT: s_lshr_b32 s31, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v14
+; SI-NEXT: v_writelane_b32 v35, s30, 1
+; SI-NEXT: s_lshr_b32 s34, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v13
+; SI-NEXT: v_writelane_b32 v35, s31, 2
+; SI-NEXT: s_lshr_b32 s63, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v12
+; SI-NEXT: v_writelane_b32 v35, s34, 3
+; SI-NEXT: s_lshr_b32 s62, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v11
+; SI-NEXT: v_writelane_b32 v35, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v10
+; SI-NEXT: v_writelane_b32 v35, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v9
+; SI-NEXT: v_writelane_b32 v35, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s78, 16
+; SI-NEXT: v_writelane_b32 v35, s60, 7
; SI-NEXT: v_readfirstlane_b32 s88, v8
+; SI-NEXT: v_readfirstlane_b32 s53, v3
+; SI-NEXT: v_writelane_b32 v35, s59, 8
+; SI-NEXT: s_lshr_b32 s58, s88, 16
+; SI-NEXT: v_writelane_b32 v35, s53, 9
+; SI-NEXT: s_lshr_b32 s54, s53, 16
+; SI-NEXT: v_writelane_b32 v35, s58, 10
; SI-NEXT: v_readfirstlane_b32 s90, v7
+; SI-NEXT: v_readfirstlane_b32 s52, v2
+; SI-NEXT: v_writelane_b32 v35, s54, 11
+; SI-NEXT: s_lshr_b32 s35, s90, 16
+; SI-NEXT: v_writelane_b32 v35, s52, 12
+; SI-NEXT: s_lshr_b32 s65, s52, 16
+; SI-NEXT: v_writelane_b32 v35, s35, 13
; SI-NEXT: v_readfirstlane_b32 s92, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v35, s65, 14
+; SI-NEXT: s_lshr_b32 s64, s92, 16
+; SI-NEXT: v_writelane_b32 v35, s51, 15
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v35, s64, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v35, s66, 17
; SI-NEXT: v_readfirstlane_b32 s94, v5
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
-; SI-NEXT: v_readfirstlane_b32 s4, v3
-; SI-NEXT: v_readfirstlane_b32 s31, v2
-; SI-NEXT: v_readfirstlane_b32 s69, v1
-; SI-NEXT: v_readfirstlane_b32 s80, v0
-; SI-NEXT: s_lshr_b32 s34, s29, 16
-; SI-NEXT: s_lshr_b32 s68, s28, 16
-; SI-NEXT: s_lshr_b32 s71, s27, 16
-; SI-NEXT: s_lshr_b32 s82, s26, 16
-; SI-NEXT: s_lshr_b32 s84, s25, 16
-; SI-NEXT: s_lshr_b32 s85, s24, 16
-; SI-NEXT: s_lshr_b32 s86, s23, 16
-; SI-NEXT: s_lshr_b32 s87, s22, 16
-; SI-NEXT: s_lshr_b32 s96, s21, 16
-; SI-NEXT: s_lshr_b32 s97, s20, 16
-; SI-NEXT: s_lshr_b32 s98, s19, 16
-; SI-NEXT: s_lshr_b32 s99, s18, 16
-; SI-NEXT: s_lshr_b32 s83, s17, 16
-; SI-NEXT: s_lshr_b32 s9, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s6, 16
-; SI-NEXT: s_lshr_b32 s66, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
-; SI-NEXT: s_lshr_b32 s95, s94, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_writelane_b32 v35, s67, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s57, s28, 16
+; SI-NEXT: s_lshr_b32 s7, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s9, s25, 16
+; SI-NEXT: s_lshr_b32 s11, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s13, s22, 16
+; SI-NEXT: s_lshr_b32 s15, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s41, s19, 16
+; SI-NEXT: s_lshr_b32 s42, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s94, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s30, s4, 16
-; SI-NEXT: s_lshr_b32 s35, s31, 16
-; SI-NEXT: s_lshr_b32 s70, s69, 16
-; SI-NEXT: s_lshr_b32 s81, s80, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr35 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v35, s67, 0
-; SI-NEXT: v_writelane_b32 v35, s66, 1
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v35, s50, 19
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v35, s55, 20
+; SI-NEXT: v_writelane_b32 v35, s5, 21
; SI-NEXT: s_cbranch_scc0 .LBB83_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s9, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s99, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s98, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s83, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s96, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s52, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s73, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s38, s44, s42
+; SI-NEXT: s_and_b32 s42, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s39, s42, s41
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s41, s41, s15
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s42, s42, s13
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_and_b32 s47, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s36, s47, s45
+; SI-NEXT: s_or_b32 s44, s44, s11
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s45, s45, s9
+; SI-NEXT: s_and_b32 s47, s26, 0xffff
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s47, s46
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s47, s47, s7
+; SI-NEXT: s_and_b32 s79, s28, 0xffff
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s48, s79, s57
+; SI-NEXT: s_and_b32 s57, s29, 0xffff
+; SI-NEXT: s_mov_b32 s79, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s57, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s65, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s54, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s57, vcc_hi, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s55, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s64, 16
+; SI-NEXT: s_or_b32 s56, s56, s57
+; SI-NEXT: s_and_b32 s57, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s57, s57, s35
+; SI-NEXT: s_and_b32 s35, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s35, s58
+; SI-NEXT: s_and_b32 s35, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s35, s59
+; SI-NEXT: s_and_b32 s35, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s35, s60
+; SI-NEXT: s_and_b32 s35, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s35, s61
+; SI-NEXT: s_and_b32 s35, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s35, s62
+; SI-NEXT: s_and_b32 s35, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s35, s63
+; SI-NEXT: s_and_b32 s35, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s64, s35, s34
+; SI-NEXT: s_and_b32 s34, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s65, s34, s31
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s66, s31, s30
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB83_3
; SI-NEXT: .LBB83_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s45
+; SI-NEXT: s_mov_b32 s95, s44
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s41
+; SI-NEXT: s_mov_b32 s73, s40
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s43
+; SI-NEXT: s_mov_b32 s91, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s46
+; SI-NEXT: s_mov_b32 s89, s7
+; SI-NEXT: s_mov_b32 s7, s57
+; SI-NEXT: s_mov_b32 s79, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB83_3: ; %Flow
@@ -136410,9 +136505,9 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB83_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s95
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -136424,7 +136519,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s99
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -136432,11 +136527,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s98
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -136447,7 +136542,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s93
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -136456,11 +136551,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s15
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s13
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -136471,7 +136566,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s91
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -136480,11 +136575,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_or_b32_e32 v7, v9, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s11
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v8
; SI-NEXT: v_or_b32_e32 v8, v10, v8
; SI-NEXT: v_cvt_f32_f16_e32 v10, s25
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s9
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
@@ -136495,7 +136590,7 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_or_b32_e32 v9, v10, v9
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v11
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s89
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
@@ -136504,11 +136599,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_or_b32_e32 v10, v12, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s7
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v11
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, s28
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s79
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
@@ -136516,47 +136611,56 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v35, 19
; SI-NEXT: v_cvt_f32_f16_e32 v15, s29
; SI-NEXT: v_or_b32_e32 v12, v13, v12
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s81
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 18
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
-; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
+; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
+; SI-NEXT: v_readlane_b32 s4, v35, 17
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v35, 15
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s69
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 14
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 12
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 11
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
+; SI-NEXT: v_readlane_b32 s4, v35, 9
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
-; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
+; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v35, 20
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, vcc_lo
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s95
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -136564,10 +136668,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v35, 16
; SI-NEXT: v_cvt_f32_f16_e32 v21, s94
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s92
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -136575,12 +136680,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v35, 13
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v35, 10
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -136588,10 +136695,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v35, 8
; SI-NEXT: v_cvt_f32_f16_e32 v24, s88
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s78
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -136599,12 +136707,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v35, 7
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v35, 6
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
@@ -136612,10 +136722,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v35, 5
; SI-NEXT: v_cvt_f32_f16_e32 v27, s74
; SI-NEXT: v_or_b32_e32 v24, v25, v24
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v28, s72
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
@@ -136623,12 +136734,14 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
+; SI-NEXT: v_readlane_b32 s4, v35, 4
; SI-NEXT: v_or_b32_e32 v25, v27, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v26
+; SI-NEXT: v_readlane_b32 s4, v35, 3
; SI-NEXT: v_or_b32_e32 v26, v28, v26
; SI-NEXT: v_cvt_f32_f16_e32 v28, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
@@ -136636,10 +136749,11 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v35, 2
; SI-NEXT: v_cvt_f32_f16_e32 v30, s12
; SI-NEXT: v_or_b32_e32 v27, v28, v27
; SI-NEXT: v_lshlrev_b32_e32 v28, 16, v29
-; SI-NEXT: v_cvt_f32_f16_e32 v29, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v29, s4
; SI-NEXT: v_cvt_f32_f16_e32 v31, s10
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
@@ -136655,7 +136769,8 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v29, v31, v29
; SI-NEXT: v_cvt_f32_f16_e32 v31, s8
; SI-NEXT: v_cvt_f32_f16_e32 v32, s4
-; SI-NEXT: v_cvt_f32_f16_e32 v33, s6
+; SI-NEXT: v_readlane_b32 s4, v35, 21
+; SI-NEXT: v_cvt_f32_f16_e32 v33, s4
; SI-NEXT: v_add_f32_e32 v30, 0x38000000, v30
; SI-NEXT: v_cvt_f16_f32_e32 v30, v30
; SI-NEXT: v_add_f32_e32 v31, 0x38000000, v31
@@ -136703,24 +136818,8 @@ define inreg <16 x double> @bitcast_v64f16_to_v16f64_scalar(<64 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB83_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v34, 34
-; SI-NEXT: v_readlane_b32 s31, v34, 35
-; SI-NEXT: v_readlane_b32 s99, v34, 33
-; SI-NEXT: v_readlane_b32 s98, v34, 32
-; SI-NEXT: v_readlane_b32 s97, v34, 31
-; SI-NEXT: v_readlane_b32 s96, v34, 30
-; SI-NEXT: v_readlane_b32 s87, v34, 29
-; SI-NEXT: v_readlane_b32 s86, v34, 28
-; SI-NEXT: v_readlane_b32 s85, v34, 27
-; SI-NEXT: v_readlane_b32 s84, v34, 26
-; SI-NEXT: v_readlane_b32 s83, v34, 25
-; SI-NEXT: v_readlane_b32 s82, v34, 24
-; SI-NEXT: v_readlane_b32 s81, v34, 23
-; SI-NEXT: v_readlane_b32 s80, v34, 22
-; SI-NEXT: v_readlane_b32 s71, v34, 21
-; SI-NEXT: v_readlane_b32 s70, v34, 20
-; SI-NEXT: v_readlane_b32 s69, v34, 19
-; SI-NEXT: v_readlane_b32 s68, v34, 18
+; SI-NEXT: v_readlane_b32 s30, v34, 18
+; SI-NEXT: v_readlane_b32 s31, v34, 19
; SI-NEXT: v_readlane_b32 s67, v34, 17
; SI-NEXT: v_readlane_b32 s66, v34, 16
; SI-NEXT: v_readlane_b32 s65, v34, 15
@@ -139311,195 +139410,215 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s98, 32
-; SI-NEXT: v_writelane_b32 v32, s99, 33
-; SI-NEXT: v_writelane_b32 v32, s30, 34
-; SI-NEXT: v_writelane_b32 v32, s31, 35
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v17
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v16
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v15
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s30, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v13
+; SI-NEXT: v_writelane_b32 v33, s30, 2
+; SI-NEXT: s_lshr_b32 s63, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v12
+; SI-NEXT: v_writelane_b32 v33, s31, 3
+; SI-NEXT: s_lshr_b32 s62, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 4
+; SI-NEXT: s_lshr_b32 s61, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 5
+; SI-NEXT: s_lshr_b32 s60, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 6
+; SI-NEXT: s_lshr_b32 s59, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 7
+; SI-NEXT: s_lshr_b32 s58, s89, 16
; SI-NEXT: v_readfirstlane_b32 s91, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 8
+; SI-NEXT: s_lshr_b32 s57, s91, 16
+; SI-NEXT: v_readfirstlane_b32 s54, v4
+; SI-NEXT: v_writelane_b32 v33, s58, 9
; SI-NEXT: v_readfirstlane_b32 s94, v6
+; SI-NEXT: s_lshr_b32 s55, s54, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 10
+; SI-NEXT: s_lshr_b32 s34, s94, 16
+; SI-NEXT: v_writelane_b32 v33, s55, 11
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_writelane_b32 v33, s34, 12
+; SI-NEXT: s_lshr_b32 s53, s64, 16
+; SI-NEXT: v_writelane_b32 v33, s54, 13
+; SI-NEXT: v_readfirstlane_b32 s65, v2
+; SI-NEXT: v_writelane_b32 v33, s53, 14
+; SI-NEXT: s_lshr_b32 s52, s65, 16
+; SI-NEXT: v_writelane_b32 v33, s64, 15
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 16
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v5
-; SI-NEXT: v_readfirstlane_b32 s31, v4
-; SI-NEXT: v_readfirstlane_b32 s69, v3
-; SI-NEXT: v_readfirstlane_b32 s80, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v1
-; SI-NEXT: v_readfirstlane_b32 s86, v0
-; SI-NEXT: s_lshr_b32 s92, s29, 16
-; SI-NEXT: s_lshr_b32 s95, s28, 16
-; SI-NEXT: s_lshr_b32 s4, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s68, s25, 16
-; SI-NEXT: s_lshr_b32 s70, s24, 16
-; SI-NEXT: s_lshr_b32 s81, s23, 16
-; SI-NEXT: s_lshr_b32 s84, s22, 16
-; SI-NEXT: s_lshr_b32 s87, s21, 16
-; SI-NEXT: s_lshr_b32 s96, s20, 16
-; SI-NEXT: s_lshr_b32 s97, s19, 16
-; SI-NEXT: s_lshr_b32 s98, s18, 16
-; SI-NEXT: s_lshr_b32 s99, s17, 16
-; SI-NEXT: s_lshr_b32 s6, s16, 16
-; SI-NEXT: s_lshr_b32 s67, s7, 16
-; SI-NEXT: s_lshr_b32 s10, s9, 16
-; SI-NEXT: s_lshr_b32 s12, s11, 16
-; SI-NEXT: s_lshr_b32 s14, s13, 16
-; SI-NEXT: s_lshr_b32 s63, s15, 16
-; SI-NEXT: s_lshr_b32 s62, s73, 16
-; SI-NEXT: s_lshr_b32 s61, s75, 16
-; SI-NEXT: s_lshr_b32 s60, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s90, s91, 16
-; SI-NEXT: s_lshr_b32 s93, s94, 16
-; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s69, 16
-; SI-NEXT: s_lshr_b32 s71, s80, 16
-; SI-NEXT: s_lshr_b32 s82, s83, 16
-; SI-NEXT: s_lshr_b32 s85, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_writelane_b32 v33, s63, 1
+; SI-NEXT: s_lshr_b32 s66, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s65, 17
+; SI-NEXT: s_lshr_b32 s35, vcc_hi, 16
+; SI-NEXT: v_readfirstlane_b32 s67, v0
+; SI-NEXT: v_writelane_b32 v33, s66, 18
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: s_lshr_b32 s6, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s10, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s14, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s8, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s41, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s50, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v18
+; SI-NEXT: v_writelane_b32 v33, s35, 19
+; SI-NEXT: s_cmp_lg_u32 s72, 0
+; SI-NEXT: v_writelane_b32 v33, s51, 20
+; SI-NEXT: v_writelane_b32 v33, s50, 21
; SI-NEXT: s_cbranch_scc0 .LBB87_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s6, 16
-; SI-NEXT: s_or_b32 s36, s5, s8
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s98, 16
-; SI-NEXT: s_or_b32 s38, s5, s8
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s97, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s99, 16
-; SI-NEXT: s_or_b32 s39, s5, s8
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s96, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s8
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s87, 16
-; SI-NEXT: s_or_b32 s41, s5, s8
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s84, 16
-; SI-NEXT: s_or_b32 s42, s5, s8
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s81, 16
-; SI-NEXT: s_or_b32 s43, s5, s8
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s70, 16
-; SI-NEXT: s_or_b32 s44, s5, s8
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s68, 16
-; SI-NEXT: s_or_b32 s45, s5, s8
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s8
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_and_b32 s76, s17, 0xffff
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s37, s76, s41
+; SI-NEXT: s_and_b32 s41, s18, 0xffff
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s41, s12
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s39, s41, s8
+; SI-NEXT: s_and_b32 s41, s20, 0xffff
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_and_b32 s74, s16, 0xffff
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s40, s41, s40
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s74, s43
+; SI-NEXT: s_or_b32 s41, s41, s14
+; SI-NEXT: s_and_b32 s43, s22, 0xffff
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s43, s42
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s43, s43, s10
+; SI-NEXT: s_and_b32 s76, s24, 0xffff
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s76, s44
+; SI-NEXT: s_and_b32 s76, s25, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s76, s45
+; SI-NEXT: s_and_b32 s76, s26, 0xffff
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s76, s46
+; SI-NEXT: s_and_b32 s76, s27, 0xffff
+; SI-NEXT: s_mov_b32 s90, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s76, s47
+; SI-NEXT: s_and_b32 s88, s28, 0xffff
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s48, s88, s6
+; SI-NEXT: s_and_b32 s88, s29, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s49, s88, s56
+; SI-NEXT: s_and_b32 s56, s67, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s50, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s66, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, s65, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s64, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s53, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s54, 0xffff
+; SI-NEXT: s_lshl_b32 s54, s55, 16
+; SI-NEXT: s_or_b32 s54, s56, s54
+; SI-NEXT: s_and_b32 s56, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s55, s56, s35
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s56, s56, s34
+; SI-NEXT: s_and_b32 s34, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s34, s57
+; SI-NEXT: s_and_b32 s34, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s34, s58
+; SI-NEXT: s_and_b32 s34, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s34, s59
+; SI-NEXT: s_and_b32 s34, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s34, s60
+; SI-NEXT: s_and_b32 s34, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s34, s61
+; SI-NEXT: s_and_b32 s34, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s34, s62
+; SI-NEXT: s_and_b32 s34, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s34, s63
+; SI-NEXT: s_and_b32 s34, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s64, s34, s31
+; SI-NEXT: s_and_b32 s31, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s65, s31, s30
+; SI-NEXT: s_and_b32 s30, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s66, s30, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s47, s5, s4
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s60, 16
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s61, 16
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s62, 16
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s63, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s66, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_or_b32 s67, s4, s5
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_or_b32 s67, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB87_3
; SI-NEXT: .LBB87_2:
-; SI-NEXT: s_mov_b32 s74, s60
-; SI-NEXT: s_mov_b32 s76, s61
-; SI-NEXT: s_mov_b32 s72, s62
-; SI-NEXT: s_mov_b32 s14, s12
-; SI-NEXT: s_mov_b32 s12, s10
-; SI-NEXT: s_mov_b32 s10, s67
-; SI-NEXT: s_mov_b32 s8, s4
+; SI-NEXT: s_mov_b32 s76, s6
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s78, s43
+; SI-NEXT: s_mov_b32 s72, s41
+; SI-NEXT: s_mov_b32 vcc_lo, s12
+; SI-NEXT: s_mov_b32 s12, s8
+; SI-NEXT: s_mov_b32 s8, s40
+; SI-NEXT: s_mov_b32 s95, s14
+; SI-NEXT: s_mov_b32 s88, s67
+; SI-NEXT: s_mov_b32 s14, s42
+; SI-NEXT: s_mov_b32 s74, s10
+; SI-NEXT: s_mov_b32 s10, s44
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s92, s46
+; SI-NEXT: s_mov_b32 s90, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB87_3: ; %Flow
@@ -139510,168 +139629,188 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s99, 16
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_add_i32 s37, s5, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
+; SI-NEXT: s_lshl_b32 s5, s12, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s96, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s93, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s34, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s86, s86, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s88, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 21
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 20
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s52, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s31, s31, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 15
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 14
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s53, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: s_add_i32 s94, s94, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 12
+; SI-NEXT: s_and_b32 s5, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s91, s91, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: s_and_b32 s5, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 1
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s5, v33, 0
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s66, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v33, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s67, s4, 0x30000
; SI-NEXT: .LBB87_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 34
+; SI-NEXT: v_readlane_b32 s30, v32, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -139704,23 +139843,7 @@ define inreg <16 x double> @bitcast_v64i16_to_v16f64_scalar(<64 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v29, s65
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
-; SI-NEXT: v_readlane_b32 s31, v32, 35
-; SI-NEXT: v_readlane_b32 s99, v32, 33
-; SI-NEXT: v_readlane_b32 s98, v32, 32
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -146655,1522 +146778,1576 @@ define inreg <64 x bfloat> @bitcast_v128i8_to_v64bf16_scalar(<128 x i8> inreg %a
; SI-NEXT: v_writelane_b32 v45, s99, 33
; SI-NEXT: v_writelane_b32 v45, s30, 34
; SI-NEXT: v_writelane_b32 v45, s31, 35
-; SI-NEXT: ; implicit-def: $vgpr56 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s4, v30
-; SI-NEXT: s_waitcnt expcnt(5)
-; SI-NEXT: v_writelane_b32 v56, s29, 0
-; SI-NEXT: v_writelane_b32 v56, s28, 1
-; SI-NEXT: v_writelane_b32 v56, s27, 2
-; SI-NEXT: v_writelane_b32 v56, s26, 3
-; SI-NEXT: v_writelane_b32 v56, s25, 4
-; SI-NEXT: v_writelane_b32 v56, s24, 5
-; SI-NEXT: v_writelane_b32 v56, s23, 6
-; SI-NEXT: v_writelane_b32 v56, s22, 7
-; SI-NEXT: v_writelane_b32 v56, s21, 8
-; SI-NEXT: v_writelane_b32 v56, s4, 9
-; SI-NEXT: v_readfirstlane_b32 s4, v29
-; SI-NEXT: v_writelane_b32 v56, s4, 10
-; SI-NEXT: v_readfirstlane_b32 s4, v28
-; SI-NEXT: v_writelane_b32 v56, s4, 11
-; SI-NEXT: v_readfirstlane_b32 s4, v27
-; SI-NEXT: v_writelane_b32 v56, s4, 12
-; SI-NEXT: v_readfirstlane_b32 s4, v26
-; SI-NEXT: v_writelane_b32 v56, s4, 13
-; SI-NEXT: v_readfirstlane_b32 s4, v25
-; SI-NEXT: v_writelane_b32 v56, s4, 14
-; SI-NEXT: v_readfirstlane_b32 s4, v24
-; SI-NEXT: v_writelane_b32 v56, s4, 15
-; SI-NEXT: v_readfirstlane_b32 s4, v23
+; SI-NEXT: v_readfirstlane_b32 s91, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:332
-; SI-NEXT: s_mov_b32 s6, s19
-; SI-NEXT: v_writelane_b32 v56, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s4, v22
-; SI-NEXT: v_readfirstlane_b32 s19, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:328
-; SI-NEXT: v_writelane_b32 v56, s4, 17
-; SI-NEXT: v_readfirstlane_b32 s4, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
-; SI-NEXT: v_writelane_b32 v56, s4, 18
-; SI-NEXT: v_readfirstlane_b32 s4, v20
-; SI-NEXT: v_writelane_b32 v56, s4, 19
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:320
+; SI-NEXT: v_readfirstlane_b32 s31, v22
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:328
+; SI-NEXT: v_readfirstlane_b32 s94, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:324
+; SI-NEXT: v_readfirstlane_b32 s63, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:320
+; SI-NEXT: v_readfirstlane_b32 s75, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:316
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:312
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:308
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:304
-; SI-NEXT: v_writelane_b32 v56, s4, 20
-; SI-NEXT: v_readfirstlane_b32 s75, v15
-; SI-NEXT: v_readfirstlane_b32 s44, v16
-; SI-NEXT: v_readfirstlane_b32 s27, v12
-; SI-NEXT: v_readfirstlane_b32 s52, v0
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:244
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s89, v10
-; SI-NEXT: v_readfirstlane_b32 s62, v9
-; SI-NEXT: v_readfirstlane_b32 s92, v17
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:300
-; SI-NEXT: v_readfirstlane_b32 s88, v14
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:296
-; SI-NEXT: v_readfirstlane_b32 s59, v13
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:292
-; SI-NEXT: v_readfirstlane_b32 s77, v11
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:288
-; SI-NEXT: s_mov_b32 s7, s20
-; SI-NEXT: v_readfirstlane_b32 s79, v8
-; SI-NEXT: v_readfirstlane_b32 s12, v7
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:284
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:264
-; SI-NEXT: v_readfirstlane_b32 s46, v6
+; SI-NEXT: ; implicit-def: $vgpr56 : SGPR spill to VGPR lane
+; SI-NEXT: v_readfirstlane_b32 s80, v21
+; SI-NEXT: s_waitcnt expcnt(5)
+; SI-NEXT: v_writelane_b32 v56, s19, 0
+; SI-NEXT: v_writelane_b32 v56, s17, 1
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:312
+; SI-NEXT: v_writelane_b32 v56, s16, 2
+; SI-NEXT: v_readfirstlane_b32 s35, v18
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:308
+; SI-NEXT: v_readfirstlane_b32 s4, v7
+; SI-NEXT: v_readfirstlane_b32 s39, v16
+; SI-NEXT: v_readfirstlane_b32 s37, v15
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:304
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:300
+; SI-NEXT: v_readfirstlane_b32 s54, v11
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:296
+; SI-NEXT: v_writelane_b32 v56, s4, 3
+; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_readfirstlane_b32 s96, v9
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
+; SI-NEXT: v_writelane_b32 v56, s4, 4
+; SI-NEXT: v_readfirstlane_b32 s4, v5
+; SI-NEXT: v_readfirstlane_b32 s95, v8
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
-; SI-NEXT: v_readfirstlane_b32 s22, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
-; SI-NEXT: v_readfirstlane_b32 s36, v4
+; SI-NEXT: v_writelane_b32 v56, s4, 5
+; SI-NEXT: v_readfirstlane_b32 s4, v4
+; SI-NEXT: v_writelane_b32 v56, s4, 6
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_readfirstlane_b32 s20, v3
+; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s34, v10
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:264
+; SI-NEXT: v_readfirstlane_b32 s38, v13
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:260
+; SI-NEXT: v_writelane_b32 v56, s4, 7
+; SI-NEXT: v_readfirstlane_b32 s48, v14
+; SI-NEXT: v_readfirstlane_b32 s92, v12
+; SI-NEXT: v_readfirstlane_b32 s90, v0
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:244
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
+; SI-NEXT: v_readfirstlane_b32 s79, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v28
+; SI-NEXT: v_readfirstlane_b32 s77, v17
+; SI-NEXT: s_mov_b32 s60, s26
+; SI-NEXT: v_readfirstlane_b32 s47, v30
+; SI-NEXT: v_readfirstlane_b32 s58, v29
+; SI-NEXT: v_readfirstlane_b32 s72, v27
+; SI-NEXT: v_readfirstlane_b32 s73, v26
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
+; SI-NEXT: s_mov_b32 s57, s21
+; SI-NEXT: s_mov_b32 s89, s20
+; SI-NEXT: s_mov_b32 s61, s27
+; SI-NEXT: s_mov_b32 s45, s28
; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v23
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v1
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:256
+; SI-NEXT: v_readfirstlane_b32 s4, v2
+; SI-NEXT: v_writelane_b32 v56, s4, 8
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:248
-; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
-; SI-NEXT: v_readfirstlane_b32 s83, v18
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:232
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
-; SI-NEXT: v_readfirstlane_b32 s4, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:228
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:224
-; SI-NEXT: v_readfirstlane_b32 s51, v19
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:220
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:248
+; SI-NEXT: v_writelane_b32 v56, s4, 9
+; SI-NEXT: v_readfirstlane_b32 s4, v22
+; SI-NEXT: v_writelane_b32 v56, s4, 10
+; SI-NEXT: v_readfirstlane_b32 s4, v19
+; SI-NEXT: v_writelane_b32 v56, s4, 11
+; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:220
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:216
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:232
+; SI-NEXT: v_writelane_b32 v56, s4, 12
+; SI-NEXT: v_readfirstlane_b32 s4, v25
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:212
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:208
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:228
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:224
+; SI-NEXT: v_writelane_b32 v56, s4, 13
+; SI-NEXT: v_readfirstlane_b32 s4, v21
+; SI-NEXT: v_writelane_b32 v56, s4, 14
+; SI-NEXT: v_readfirstlane_b32 s4, v18
+; SI-NEXT: v_writelane_b32 v56, s4, 15
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s8, v22
-; SI-NEXT: v_writelane_b32 v56, s8, 21
-; SI-NEXT: v_readfirstlane_b32 s8, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 22
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:184
-; SI-NEXT: v_readfirstlane_b32 s39, v12
-; SI-NEXT: v_readfirstlane_b32 s31, v0
-; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60
-; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:56
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:68
-; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s5, v24
-; SI-NEXT: v_readfirstlane_b32 s8, v17
-; SI-NEXT: v_writelane_b32 v56, s8, 23
-; SI-NEXT: v_readfirstlane_b32 s8, v14
-; SI-NEXT: v_writelane_b32 v56, s8, 24
-; SI-NEXT: v_readfirstlane_b32 s8, v13
-; SI-NEXT: v_writelane_b32 v56, s8, 25
-; SI-NEXT: v_readfirstlane_b32 s8, v11
-; SI-NEXT: v_writelane_b32 v56, s8, 26
-; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:156
-; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:152
-; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:172
+; SI-NEXT: v_readfirstlane_b32 s4, v11
+; SI-NEXT: v_writelane_b32 v56, s4, 16
+; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: v_readfirstlane_b32 s8, v8
-; SI-NEXT: v_readfirstlane_b32 s37, v7
-; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:108
-; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:104
-; SI-NEXT: v_writelane_b32 v56, s8, 27
-; SI-NEXT: v_readfirstlane_b32 s8, v6
-; SI-NEXT: v_writelane_b32 v56, s8, 28
-; SI-NEXT: v_readfirstlane_b32 s8, v5
-; SI-NEXT: v_writelane_b32 v56, s8, 29
-; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s8, v4
-; SI-NEXT: v_writelane_b32 v56, s8, 30
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
-; SI-NEXT: v_readfirstlane_b32 s8, v3
-; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
-; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
+; SI-NEXT: v_readfirstlane_b32 s4, v9
+; SI-NEXT: v_writelane_b32 v56, s4, 17
+; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
+; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
+; SI-NEXT: v_readfirstlane_b32 s26, v15
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v8
+; SI-NEXT: v_readfirstlane_b32 s50, v7
+; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 18
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
+; SI-NEXT: v_readfirstlane_b32 s85, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
-; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:148
+; SI-NEXT: v_readfirstlane_b32 s69, v4
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s4, v3
+; SI-NEXT: v_writelane_b32 v56, s4, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v10
+; SI-NEXT: v_writelane_b32 v56, s4, 20
+; SI-NEXT: s_waitcnt vmcnt(14)
+; SI-NEXT: v_readfirstlane_b32 s4, v13
+; SI-NEXT: v_writelane_b32 v56, s4, 21
+; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:108
+; SI-NEXT: buffer_load_dword v39, off, s[0:3], s32 offset:104
+; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:100
+; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
+; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
+; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
+; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:164
-; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:180
+; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:152
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:196
+; SI-NEXT: v_readfirstlane_b32 s93, v16
+; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:184
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
-; SI-NEXT: v_writelane_b32 v56, s8, 31
-; SI-NEXT: v_readfirstlane_b32 s85, v9
-; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:92
-; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s35, v10
-; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:76
-; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s48, v18
-; SI-NEXT: v_readfirstlane_b32 s63, v15
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:44
-; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s38, v21
-; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:36
-; SI-NEXT: s_waitcnt expcnt(4)
-; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s30, v16
-; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:28
-; SI-NEXT: s_waitcnt expcnt(3)
-; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:52
-; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s82, v1
+; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
+; SI-NEXT: v_readfirstlane_b32 s4, v14
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v1
+; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
-; SI-NEXT: v_readfirstlane_b32 s78, v2
-; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
-; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s86, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:12
+; SI-NEXT: v_readfirstlane_b32 s4, v2
+; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:76
+; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
+; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:92
+; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
+; SI-NEXT: v_writelane_b32 v56, s4, 24
+; SI-NEXT: v_readfirstlane_b32 s4, v12
+; SI-NEXT: v_writelane_b32 v56, s4, 25
+; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v56, s4, 26
+; SI-NEXT: v_readfirstlane_b32 s4, v22
+; SI-NEXT: v_readfirstlane_b32 s70, v20
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s61, v28
+; SI-NEXT: v_readfirstlane_b32 s71, v28
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s84, v23
-; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:20
+; SI-NEXT: v_writelane_b32 v56, s4, 27
+; SI-NEXT: v_readfirstlane_b32 s4, v17
+; SI-NEXT: v_writelane_b32 v56, s4, 28
+; SI-NEXT: v_readfirstlane_b32 s4, v23
+; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_readfirstlane_b32 s4, v19
+; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:28
+; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
+; SI-NEXT: v_readfirstlane_b32 s4, v24
+; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:20
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s11, v0
-; SI-NEXT: v_readfirstlane_b32 s8, v53
-; SI-NEXT: v_readfirstlane_b32 s24, v12
-; SI-NEXT: v_readfirstlane_b32 s29, v52
-; SI-NEXT: v_readfirstlane_b32 s97, v29
-; SI-NEXT: v_readfirstlane_b32 s98, v20
-; SI-NEXT: v_readfirstlane_b32 s49, v25
-; SI-NEXT: v_readfirstlane_b32 s73, v11
-; SI-NEXT: v_readfirstlane_b32 s96, v33
-; SI-NEXT: v_readfirstlane_b32 s26, v31
-; SI-NEXT: v_readfirstlane_b32 s94, v14
-; SI-NEXT: v_readfirstlane_b32 s72, v7
-; SI-NEXT: v_readfirstlane_b32 s58, v39
-; SI-NEXT: v_readfirstlane_b32 s81, v4
-; SI-NEXT: v_readfirstlane_b32 s60, v37
-; SI-NEXT: v_readfirstlane_b32 s47, v3
-; SI-NEXT: v_readfirstlane_b32 s76, v38
-; SI-NEXT: v_readfirstlane_b32 s91, v5
-; SI-NEXT: v_readfirstlane_b32 s21, v35
-; SI-NEXT: v_readfirstlane_b32 s57, v36
-; SI-NEXT: v_readfirstlane_b32 s87, v8
-; SI-NEXT: v_readfirstlane_b32 s99, v34
-; SI-NEXT: v_readfirstlane_b32 s90, v6
-; SI-NEXT: v_readfirstlane_b32 s28, v13
-; SI-NEXT: v_readfirstlane_b32 s25, v32
-; SI-NEXT: v_readfirstlane_b32 s74, v17
-; SI-NEXT: v_readfirstlane_b32 s93, v30
-; SI-NEXT: v_readfirstlane_b32 s23, v22
-; SI-NEXT: v_readfirstlane_b32 s50, v26
-; SI-NEXT: v_readfirstlane_b32 s95, v27
-; SI-NEXT: v_readfirstlane_b32 s34, v24
-; SI-NEXT: v_readfirstlane_b32 s45, v9
-; SI-NEXT: v_readfirstlane_b32 s15, v49
-; SI-NEXT: v_readfirstlane_b32 s56, v10
-; SI-NEXT: v_readfirstlane_b32 s13, v51
+; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:44
+; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
+; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:36
+; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
+; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:60
+; SI-NEXT: buffer_load_dword v53, off, s[0:3], s32 offset:56
+; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
+; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:68
+; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s19, v5
+; SI-NEXT: v_readfirstlane_b32 s66, v37
+; SI-NEXT: v_readfirstlane_b32 s86, v35
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v36
+; SI-NEXT: v_readfirstlane_b32 s87, v4
+; SI-NEXT: v_readfirstlane_b32 s36, v6
+; SI-NEXT: v_writelane_b32 v56, s96, 32
+; SI-NEXT: v_readfirstlane_b32 s55, v10
+; SI-NEXT: v_readfirstlane_b32 s5, v13
+; SI-NEXT: v_readfirstlane_b32 s4, v39
+; SI-NEXT: v_readfirstlane_b32 s6, v48
+; SI-NEXT: v_readfirstlane_b32 s64, v3
+; SI-NEXT: v_readfirstlane_b32 s65, v38
+; SI-NEXT: v_readfirstlane_b32 s99, v7
+; SI-NEXT: v_readfirstlane_b32 s49, v33
+; SI-NEXT: v_readfirstlane_b32 s59, v34
+; SI-NEXT: v_writelane_b32 v56, s95, 33
+; SI-NEXT: v_readfirstlane_b32 s56, v9
+; SI-NEXT: v_writelane_b32 v56, s92, 34
+; SI-NEXT: v_readfirstlane_b32 s30, v32
+; SI-NEXT: v_writelane_b32 v56, s54, 35
+; SI-NEXT: v_readfirstlane_b32 s67, v31
+; SI-NEXT: v_writelane_b32 v56, s38, 36
+; SI-NEXT: v_readfirstlane_b32 s21, v8
+; SI-NEXT: v_writelane_b32 v56, s34, 37
+; SI-NEXT: v_readfirstlane_b32 s16, v15
+; SI-NEXT: v_writelane_b32 v56, s39, 38
+; SI-NEXT: v_readfirstlane_b32 s20, v30
+; SI-NEXT: v_writelane_b32 v56, s90, 39
+; SI-NEXT: v_readfirstlane_b32 s27, v11
+; SI-NEXT: v_writelane_b32 v56, s37, 40
; SI-NEXT: s_waitcnt vmcnt(14)
-; SI-NEXT: v_readfirstlane_b32 s9, v15
-; SI-NEXT: v_readfirstlane_b32 s69, v55
-; SI-NEXT: v_readfirstlane_b32 s68, v21
-; SI-NEXT: v_writelane_b32 v47, s68, 0
-; SI-NEXT: v_readfirstlane_b32 s70, v40
-; SI-NEXT: v_writelane_b32 v47, s69, 1
+; SI-NEXT: v_readfirstlane_b32 s17, v29
+; SI-NEXT: v_writelane_b32 v56, s77, 41
+; SI-NEXT: v_readfirstlane_b32 s98, v18
+; SI-NEXT: v_writelane_b32 v56, s48, 42
+; SI-NEXT: v_readfirstlane_b32 s28, v27
+; SI-NEXT: v_writelane_b32 v56, s63, 43
+; SI-NEXT: v_readfirstlane_b32 s68, v16
+; SI-NEXT: v_writelane_b32 v56, s80, 44
+; SI-NEXT: v_readfirstlane_b32 s84, v26
+; SI-NEXT: v_writelane_b32 v56, s94, 45
+; SI-NEXT: v_writelane_b32 v56, s35, 46
+; SI-NEXT: v_writelane_b32 v56, s79, 47
+; SI-NEXT: v_readfirstlane_b32 s81, v25
+; SI-NEXT: v_writelane_b32 v56, s91, 48
+; SI-NEXT: v_readfirstlane_b32 s82, v21
+; SI-NEXT: v_writelane_b32 v56, s75, 49
+; SI-NEXT: v_writelane_b32 v56, s31, 50
+; SI-NEXT: v_writelane_b32 v56, s62, 51
+; SI-NEXT: v_writelane_b32 v56, s72, 52
+; SI-NEXT: v_readfirstlane_b32 s10, v50
+; SI-NEXT: v_readfirstlane_b32 s12, v2
+; SI-NEXT: v_writelane_b32 v47, s12, 0
+; SI-NEXT: v_writelane_b32 v47, s10, 1
+; SI-NEXT: v_writelane_b32 v47, s5, 2
+; SI-NEXT: v_writelane_b32 v47, s4, 3
+; SI-NEXT: v_readfirstlane_b32 s13, v14
+; SI-NEXT: v_writelane_b32 v47, s6, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s64, 6
+; SI-NEXT: v_writelane_b32 v47, s66, 7
+; SI-NEXT: v_readfirstlane_b32 s15, v43
+; SI-NEXT: v_readfirstlane_b32 s41, v28
+; SI-NEXT: v_writelane_b32 v47, s41, 8
+; SI-NEXT: v_readfirstlane_b32 s40, v44
+; SI-NEXT: v_writelane_b32 v47, s15, 9
+; SI-NEXT: v_writelane_b32 v47, s40, 10
+; SI-NEXT: v_writelane_b32 v47, s65, 11
+; SI-NEXT: v_readfirstlane_b32 s46, v20
+; SI-NEXT: v_writelane_b32 v56, s58, 53
; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s71, v16
-; SI-NEXT: v_writelane_b32 v47, s70, 2
+; SI-NEXT: v_readfirstlane_b32 s88, v19
+; SI-NEXT: s_waitcnt vmcnt(12)
+; SI-NEXT: v_readfirstlane_b32 s78, v41
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s80, v18
-; SI-NEXT: v_writelane_b32 v47, s71, 3
-; SI-NEXT: v_writelane_b32 v47, s80, 4
-; SI-NEXT: v_writelane_b32 v47, s8, 5
-; SI-NEXT: v_writelane_b32 v47, s9, 6
-; SI-NEXT: v_writelane_b32 v47, s24, 7
-; SI-NEXT: v_writelane_b32 v47, s13, 8
-; SI-NEXT: v_writelane_b32 v47, s29, 9
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s14, v1
-; SI-NEXT: v_writelane_b32 v47, s11, 10
-; SI-NEXT: v_writelane_b32 v47, s14, 11
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s40, v50
-; SI-NEXT: v_writelane_b32 v47, s15, 12
-; SI-NEXT: v_writelane_b32 v47, s40, 13
+; SI-NEXT: v_readfirstlane_b32 s9, v24
+; SI-NEXT: v_writelane_b32 v47, s9, 12
+; SI-NEXT: s_waitcnt vmcnt(10)
+; SI-NEXT: v_readfirstlane_b32 s42, v42
+; SI-NEXT: v_writelane_b32 v47, s78, 13
+; SI-NEXT: v_writelane_b32 v47, s42, 14
+; SI-NEXT: v_writelane_b32 v47, s55, 15
+; SI-NEXT: v_writelane_b32 v47, s19, 16
+; SI-NEXT: v_writelane_b32 v47, s86, 17
+; SI-NEXT: v_writelane_b32 v47, vcc_hi, 18
+; SI-NEXT: v_writelane_b32 v47, s87, 19
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_readfirstlane_b32 s42, v2
-; SI-NEXT: v_writelane_b32 v47, s56, 14
-; SI-NEXT: v_writelane_b32 v47, s42, 15
+; SI-NEXT: v_readfirstlane_b32 s97, v23
+; SI-NEXT: v_writelane_b32 v47, s46, 20
+; SI-NEXT: v_readfirstlane_b32 s83, v55
+; SI-NEXT: v_writelane_b32 v47, s97, 21
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s43, v48
-; SI-NEXT: v_writelane_b32 v47, s58, 16
-; SI-NEXT: v_writelane_b32 v47, s43, 17
-; SI-NEXT: v_writelane_b32 v47, s45, 18
-; SI-NEXT: v_writelane_b32 v47, s47, 19
-; SI-NEXT: v_writelane_b32 v47, s60, 20
-; SI-NEXT: v_writelane_b32 v47, s76, 21
-; SI-NEXT: v_writelane_b32 v47, s72, 22
-; SI-NEXT: v_writelane_b32 v47, s91, 23
-; SI-NEXT: v_writelane_b32 v47, s21, 24
-; SI-NEXT: v_writelane_b32 v47, s57, 25
-; SI-NEXT: v_readfirstlane_b32 s41, v54
-; SI-NEXT: v_writelane_b32 v47, s81, 26
-; SI-NEXT: v_writelane_b32 v56, s41, 32
-; SI-NEXT: v_writelane_b32 v47, s87, 27
-; SI-NEXT: v_writelane_b32 v56, s18, 33
-; SI-NEXT: v_writelane_b32 v47, s96, 28
-; SI-NEXT: v_writelane_b32 v56, s6, 34
-; SI-NEXT: v_writelane_b32 v47, s99, 29
-; SI-NEXT: v_writelane_b32 v56, s17, 35
-; SI-NEXT: v_writelane_b32 v47, s90, 30
-; SI-NEXT: v_writelane_b32 v56, s16, 36
-; SI-NEXT: v_writelane_b32 v47, s28, 31
-; SI-NEXT: v_writelane_b32 v56, s7, 37
-; SI-NEXT: v_writelane_b32 v47, s26, 32
-; SI-NEXT: v_writelane_b32 v56, s92, 38
-; SI-NEXT: v_writelane_b32 v47, s37, 33
-; SI-NEXT: v_writelane_b32 v56, s44, 39
-; SI-NEXT: v_writelane_b32 v47, s25, 34
-; SI-NEXT: v_writelane_b32 v56, s59, 40
-; SI-NEXT: v_writelane_b32 v47, s78, 35
-; SI-NEXT: v_writelane_b32 v56, s27, 41
-; SI-NEXT: v_writelane_b32 v47, s73, 36
-; SI-NEXT: v_writelane_b32 v56, s62, 42
-; SI-NEXT: v_writelane_b32 v47, s74, 37
-; SI-NEXT: v_writelane_b32 v56, s79, 43
-; SI-NEXT: v_writelane_b32 v47, s97, 38
-; SI-NEXT: v_writelane_b32 v56, s22, 44
-; SI-NEXT: v_writelane_b32 v47, s35, 39
-; SI-NEXT: v_writelane_b32 v56, s75, 45
-; SI-NEXT: v_writelane_b32 v47, s39, 40
-; SI-NEXT: v_writelane_b32 v56, s36, 46
-; SI-NEXT: v_writelane_b32 v47, s85, 41
-; SI-NEXT: v_writelane_b32 v56, s19, 47
-; SI-NEXT: v_writelane_b32 v47, s93, 42
-; SI-NEXT: v_writelane_b32 v56, s77, 48
-; SI-NEXT: v_writelane_b32 v47, s63, 43
-; SI-NEXT: v_writelane_b32 v56, s88, 49
-; SI-NEXT: v_writelane_b32 v47, s86, 44
-; SI-NEXT: v_writelane_b32 v56, s12, 50
-; SI-NEXT: v_writelane_b32 v47, s94, 45
-; SI-NEXT: v_writelane_b32 v56, s89, 51
-; SI-NEXT: v_writelane_b32 v47, s38, 46
-; SI-NEXT: v_writelane_b32 v56, s46, 52
-; SI-NEXT: v_writelane_b32 v47, s61, 47
-; SI-NEXT: v_writelane_b32 v56, s20, 53
-; SI-NEXT: v_writelane_b32 v47, s23, 48
-; SI-NEXT: v_writelane_b32 v56, vcc_lo, 54
-; SI-NEXT: v_writelane_b32 v47, s50, 49
-; SI-NEXT: v_writelane_b32 v56, vcc_hi, 55
-; SI-NEXT: v_writelane_b32 v47, s82, 50
+; SI-NEXT: v_readfirstlane_b32 s76, v40
+; SI-NEXT: v_writelane_b32 v47, s83, 22
+; SI-NEXT: v_writelane_b32 v47, s76, 23
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s53, v28
-; SI-NEXT: v_writelane_b32 v56, s52, 56
-; SI-NEXT: v_writelane_b32 v47, s31, 51
-; SI-NEXT: v_readfirstlane_b32 s54, v43
-; SI-NEXT: v_writelane_b32 v56, s53, 57
-; SI-NEXT: v_writelane_b32 v47, s95, 52
+; SI-NEXT: v_readfirstlane_b32 s53, v22
+; SI-NEXT: v_writelane_b32 v47, s88, 24
+; SI-NEXT: v_readfirstlane_b32 s74, v53
+; SI-NEXT: v_writelane_b32 v47, s53, 25
+; SI-NEXT: v_writelane_b32 v47, s74, 26
+; SI-NEXT: v_writelane_b32 v47, s99, 27
+; SI-NEXT: v_writelane_b32 v47, s36, 28
+; SI-NEXT: v_writelane_b32 v47, s49, 29
+; SI-NEXT: v_writelane_b32 v47, s59, 30
+; SI-NEXT: v_writelane_b32 v47, s56, 31
+; SI-NEXT: v_writelane_b32 v47, s30, 32
+; SI-NEXT: v_writelane_b32 v47, s67, 33
+; SI-NEXT: v_writelane_b32 v47, s21, 34
+; SI-NEXT: v_writelane_b32 v47, s16, 35
+; SI-NEXT: v_writelane_b32 v47, s20, 36
+; SI-NEXT: v_writelane_b32 v47, s27, 37
+; SI-NEXT: v_writelane_b32 v47, s17, 38
+; SI-NEXT: v_writelane_b32 v47, s98, 39
+; SI-NEXT: v_writelane_b32 v47, s28, 40
+; SI-NEXT: v_writelane_b32 v47, s68, 41
+; SI-NEXT: v_writelane_b32 v47, s84, 42
+; SI-NEXT: v_writelane_b32 v47, s71, 43
+; SI-NEXT: v_writelane_b32 v47, s70, 44
+; SI-NEXT: v_writelane_b32 v47, s81, 45
+; SI-NEXT: v_writelane_b32 v47, s82, 46
+; SI-NEXT: v_writelane_b32 v47, s85, 47
+; SI-NEXT: v_writelane_b32 v47, s69, 48
+; SI-NEXT: v_writelane_b32 v47, s18, 49
+; SI-NEXT: v_writelane_b32 v56, s73, 54
+; SI-NEXT: v_writelane_b32 v47, s22, 50
+; SI-NEXT: v_readfirstlane_b32 s52, v17
+; SI-NEXT: v_writelane_b32 v56, s47, 55
+; SI-NEXT: v_writelane_b32 v47, s23, 51
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_readfirstlane_b32 s55, v44
-; SI-NEXT: v_writelane_b32 v56, s54, 58
-; SI-NEXT: v_writelane_b32 v47, s30, 53
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s64, v23
-; SI-NEXT: v_writelane_b32 v56, s55, 59
-; SI-NEXT: v_writelane_b32 v47, s98, 54
-; SI-NEXT: v_readfirstlane_b32 s65, v41
-; SI-NEXT: v_writelane_b32 v56, s64, 60
-; SI-NEXT: v_writelane_b32 v47, s48, 55
-; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_readfirstlane_b32 s66, v42
-; SI-NEXT: v_writelane_b32 v56, s65, 61
-; SI-NEXT: v_writelane_b32 v47, s84, 56
-; SI-NEXT: v_readfirstlane_b32 s67, v19
-; SI-NEXT: v_writelane_b32 v56, s66, 62
-; SI-NEXT: v_writelane_b32 v47, s49, 57
-; SI-NEXT: v_writelane_b32 v56, s67, 63
-; SI-NEXT: v_writelane_b32 v47, s34, 58
+; SI-NEXT: v_readfirstlane_b32 s51, v54
+; SI-NEXT: v_writelane_b32 v56, s52, 56
+; SI-NEXT: v_writelane_b32 v47, s57, 52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s7, v12
+; SI-NEXT: v_writelane_b32 v56, s51, 57
+; SI-NEXT: v_writelane_b32 v47, s89, 53
+; SI-NEXT: s_waitcnt vmcnt(0)
+; SI-NEXT: v_readfirstlane_b32 s8, v52
+; SI-NEXT: v_writelane_b32 v56, s7, 58
+; SI-NEXT: v_writelane_b32 v47, s60, 54
+; SI-NEXT: v_readfirstlane_b32 s44, v51
+; SI-NEXT: v_writelane_b32 v56, s8, 59
+; SI-NEXT: v_writelane_b32 v47, s61, 55
+; SI-NEXT: v_readfirstlane_b32 s11, v0
+; SI-NEXT: v_writelane_b32 v56, s44, 60
+; SI-NEXT: v_writelane_b32 v47, s25, 56
+; SI-NEXT: v_readfirstlane_b32 s43, v1
+; SI-NEXT: v_writelane_b32 v56, s11, 61
+; SI-NEXT: v_writelane_b32 v47, s24, 57
+; SI-NEXT: v_readfirstlane_b32 s14, v49
+; SI-NEXT: v_writelane_b32 v56, s43, 62
+; SI-NEXT: v_writelane_b32 v47, s29, 58
+; SI-NEXT: v_writelane_b32 v56, s14, 63
+; SI-NEXT: v_writelane_b32 v47, s45, 59
; SI-NEXT: s_cbranch_scc0 .LBB89_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: v_writelane_b32 v46, s83, 7
-; SI-NEXT: v_writelane_b32 v46, s4, 8
-; SI-NEXT: s_and_b32 s4, s16, 0xff
+; SI-NEXT: v_writelane_b32 v46, s50, 19
+; SI-NEXT: s_mov_b32 s50, s35
+; SI-NEXT: s_mov_b32 s35, s77
+; SI-NEXT: s_mov_b32 s77, s45
+; SI-NEXT: s_mov_b32 s45, s29
+; SI-NEXT: v_readlane_b32 s29, v56, 2
+; SI-NEXT: s_mov_b32 s69, s80
+; SI-NEXT: s_and_b32 s80, s29, 0xff
+; SI-NEXT: s_mov_b32 s85, s47
+; SI-NEXT: s_lshl_b32 s47, s80, 16
+; SI-NEXT: s_mov_b32 s80, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s58
+; SI-NEXT: v_readlane_b32 s58, v56, 1
+; SI-NEXT: s_lshl_b32 s58, s58, 24
+; SI-NEXT: v_writelane_b32 v46, s93, 20
+; SI-NEXT: v_writelane_b32 v46, s26, 21
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s26, v56, 0
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s26, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 60
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s89, 0xff
+; SI-NEXT: s_lshl_b32 s58, s57, 8
+; SI-NEXT: s_or_b32 s18, s47, s58
+; SI-NEXT: s_and_b32 s47, s22, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s23, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 61
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s24, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s25, 24
+; SI-NEXT: v_writelane_b32 v47, s93, 63
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s60, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s61, 24
+; SI-NEXT: v_writelane_b32 v46, s18, 22
+; SI-NEXT: v_writelane_b32 v47, s93, 62
+; SI-NEXT: s_or_b32 s93, s58, s47
+; SI-NEXT: s_and_b32 s47, s77, 0xff
+; SI-NEXT: s_lshl_b32 s58, s45, 8
+; SI-NEXT: v_writelane_b32 v46, s93, 0
+; SI-NEXT: s_or_b32 s93, s47, s58
+; SI-NEXT: s_and_b32 s47, s90, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 9
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 8
+; SI-NEXT: s_or_b32 s90, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 7
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 6
+; SI-NEXT: s_or_b32 s89, s58, s47
+; SI-NEXT: s_and_b32 s47, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 5
+; SI-NEXT: s_lshl_b32 s58, s18, 24
+; SI-NEXT: v_readlane_b32 s18, v56, 4
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_and_b32 s45, s18, 0xff
+; SI-NEXT: v_readlane_b32 s18, v56, 3
+; SI-NEXT: s_or_b32 s61, s58, s47
+; SI-NEXT: s_lshl_b32 s47, s18, 8
+; SI-NEXT: v_writelane_b32 v46, s90, 2
+; SI-NEXT: s_or_b32 s90, s45, s47
+; SI-NEXT: s_and_b32 s47, s95, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s58, s96, 24
+; SI-NEXT: v_writelane_b32 v46, s89, 1
+; SI-NEXT: s_or_b32 s60, s58, s47
+; SI-NEXT: s_and_b32 s47, s34, 0xff
+; SI-NEXT: v_writelane_b32 v46, s61, 3
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s54, 24
+; SI-NEXT: v_writelane_b32 v46, s60, 5
+; SI-NEXT: s_or_b32 s57, s57, s47
+; SI-NEXT: s_and_b32 s47, s92, 0xff
+; SI-NEXT: v_writelane_b32 v46, s57, 4
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s38, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 6
+; SI-NEXT: s_and_b32 s47, s48, 0xff
+; SI-NEXT: s_lshl_b32 s57, s37, 8
+; SI-NEXT: s_or_b32 s58, s47, s57
+; SI-NEXT: s_and_b32 s47, s39, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s35, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 7
+; SI-NEXT: s_and_b32 s47, s50, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s94, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 14
+; SI-NEXT: s_and_b32 s47, s63, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s69, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 13
+; SI-NEXT: s_and_b32 s47, s31, 0xff
+; SI-NEXT: s_lshl_b32 s57, s91, 8
+; SI-NEXT: s_or_b32 s45, s47, s57
+; SI-NEXT: s_and_b32 s47, s79, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s75, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 15
+; SI-NEXT: s_and_b32 s47, s73, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, s72, 24
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: v_writelane_b32 v46, s47, 17
+; SI-NEXT: s_and_b32 s47, s62, 0xff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_lshl_b32 s57, vcc_lo, 24
+; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_or_b32 s47, s57, s47
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: v_writelane_b32 v46, s47, 16
+; SI-NEXT: s_or_b32 s15, s15, s41
+; SI-NEXT: v_writelane_b32 v46, s15, 18
+; SI-NEXT: s_and_b32 s15, s46, 0xff
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_lshl_b32 s41, s42, 24
+; SI-NEXT: s_or_b32 s15, s41, s15
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: v_writelane_b32 v46, s15, 8
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s15, s78, 24
+; SI-NEXT: s_or_b32 s9, s15, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 9
+; SI-NEXT: s_and_b32 s9, s88, 0xff
+; SI-NEXT: s_lshl_b32 s15, s76, 8
+; SI-NEXT: s_or_b32 s15, s9, s15
+; SI-NEXT: s_and_b32 s9, s97, 0xff
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s83, 24
+; SI-NEXT: s_or_b32 s9, s41, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 11
+; SI-NEXT: s_and_b32 s9, s52, 0xff
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s51, 24
+; SI-NEXT: s_or_b32 s9, s41, s9
+; SI-NEXT: v_writelane_b32 v46, s9, 10
+; SI-NEXT: s_and_b32 s9, s53, 0xff
+; SI-NEXT: s_and_b32 s47, s85, 0xff
+; SI-NEXT: s_lshl_b32 s40, s40, 8
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_lshl_b32 s41, s74, 24
+; SI-NEXT: s_or_b32 s40, s47, s40
+; SI-NEXT: s_mov_b32 s47, s49
+; SI-NEXT: s_or_b32 s49, s41, s9
+; SI-NEXT: s_and_b32 s9, s11, 0xff
+; SI-NEXT: s_lshl_b32 s8, s8, 8
+; SI-NEXT: s_and_b32 s7, s7, 0xff
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_or_b32 s51, s9, s8
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s44, 24
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_or_b32 s7, s8, s7
+; SI-NEXT: s_or_b32 s11, s4, s5
+; SI-NEXT: s_and_b32 s4, s55, 0xff
+; SI-NEXT: v_writelane_b32 v46, s7, 12
+; SI-NEXT: s_and_b32 s7, s12, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_mov_b32 s83, s51
-; SI-NEXT: s_mov_b32 s51, s10
-; SI-NEXT: s_mov_b32 s10, s5
-; SI-NEXT: s_lshl_b32 s5, s17, 24
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s18, 0xff
+; SI-NEXT: s_lshl_b32 s5, s65, 24
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s10, 24
+; SI-NEXT: s_or_b32 s55, s5, s4
+; SI-NEXT: s_and_b32 s4, s64, 0xff
+; SI-NEXT: s_mov_b32 s42, s16
+; SI-NEXT: s_or_b32 s16, s8, s7
+; SI-NEXT: s_and_b32 s7, s43, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s5, s6, 24
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v56, 8
-; SI-NEXT: v_writelane_b32 v47, s4, 60
-; SI-NEXT: s_and_b32 s4, s7, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v46, s4, 9
-; SI-NEXT: v_readlane_b32 s4, v56, 7
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 6
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 5
-; SI-NEXT: v_writelane_b32 v47, s5, 62
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 4
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 3
-; SI-NEXT: v_writelane_b32 v47, s5, 61
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 2
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s4, 24
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s4, v56, 1
-; SI-NEXT: v_writelane_b32 v47, s5, 63
-; SI-NEXT: s_and_b32 s5, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 0
-; SI-NEXT: s_lshl_b32 s6, s4, 8
-; SI-NEXT: s_or_b32 s4, s5, s6
-; SI-NEXT: s_and_b32 s6, s52, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 10
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 1
-; SI-NEXT: s_and_b32 s6, vcc_hi, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s20, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 0
-; SI-NEXT: s_and_b32 s6, s36, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s22, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 2
-; SI-NEXT: s_and_b32 s6, s46, 0xff
-; SI-NEXT: s_lshl_b32 s7, s12, 8
-; SI-NEXT: s_or_b32 s4, s6, s7
-; SI-NEXT: s_and_b32 s6, s79, 0xff
+; SI-NEXT: s_lshl_b32 s5, s66, 24
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s8, s14, 24
+; SI-NEXT: s_or_b32 s22, s5, s4
+; SI-NEXT: s_and_b32 s4, s87, 0xff
+; SI-NEXT: s_lshl_b32 s5, vcc_hi, 8
+; SI-NEXT: s_or_b32 s7, s8, s7
+; SI-NEXT: s_and_b32 s8, s13, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s5, s4, s5
+; SI-NEXT: s_and_b32 s4, s19, 0xff
+; SI-NEXT: s_or_b32 s10, s8, s6
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s86, 24
+; SI-NEXT: s_or_b32 s64, s6, s4
+; SI-NEXT: s_and_b32 s4, s36, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s59, 24
+; SI-NEXT: s_or_b32 s65, s6, s4
+; SI-NEXT: s_and_b32 s4, s99, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s47, 24
+; SI-NEXT: s_or_b32 s94, s6, s4
+; SI-NEXT: s_and_b32 s4, s21, 0xff
+; SI-NEXT: s_lshl_b32 s6, s30, 8
+; SI-NEXT: s_or_b32 s29, s4, s6
+; SI-NEXT: s_and_b32 s4, s56, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s67, 24
+; SI-NEXT: s_or_b32 s92, s6, s4
+; SI-NEXT: s_and_b32 s4, s27, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s20, 24
+; SI-NEXT: s_or_b32 s88, s6, s4
+; SI-NEXT: s_and_b32 s4, s42, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s17, 24
+; SI-NEXT: s_or_b32 s89, s6, s4
+; SI-NEXT: s_and_b32 s4, s68, 0xff
+; SI-NEXT: s_lshl_b32 s6, s28, 8
+; SI-NEXT: s_or_b32 s23, s4, s6
+; SI-NEXT: s_and_b32 s4, s98, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s84, 24
+; SI-NEXT: s_or_b32 s78, s6, s4
+; SI-NEXT: s_and_b32 s4, s82, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s81, 24
+; SI-NEXT: s_or_b32 s62, s6, s4
+; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s70, 24
+; SI-NEXT: s_or_b32 s76, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 31
+; SI-NEXT: v_readlane_b32 s6, v56, 30
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s24, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v56, 29
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 28
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s61, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 27
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 26
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s47, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 25
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 24
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s57, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 23
+; SI-NEXT: v_readlane_b32 s6, v56, 22
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s25, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v56, 21
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v56, 20
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s46, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 19
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: v_readlane_b32 s6, v47, 48
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_or_b32 s14, s6, s4
+; SI-NEXT: v_readlane_b32 s4, v47, 47
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s4, 16
+; SI-NEXT: v_readlane_b32 s4, v56, 18
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_or_b32 s44, s4, s6
+; SI-NEXT: v_readlane_b32 s4, v46, 19
+; SI-NEXT: s_mov_b32 s75, s4
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s80, 8
+; SI-NEXT: s_or_b32 s4, s4, s6
+; SI-NEXT: v_readlane_b32 s6, v56, 17
+; SI-NEXT: v_readlane_b32 s77, v56, 16
+; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s62, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 11
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 4
-; SI-NEXT: s_and_b32 s6, s89, 0xff
+; SI-NEXT: s_lshl_b32 s8, s77, 24
+; SI-NEXT: s_or_b32 s12, s8, s6
+; SI-NEXT: v_readlane_b32 s6, v46, 21
+; SI-NEXT: s_mov_b32 s20, s6
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: v_readlane_b32 s8, v46, 20
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s77, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 3
-; SI-NEXT: s_and_b32 s6, s27, 0xff
+; SI-NEXT: s_mov_b32 s91, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 24
+; SI-NEXT: s_or_b32 s8, s8, s6
+; SI-NEXT: v_readlane_b32 s6, v56, 15
+; SI-NEXT: s_mov_b32 s72, s6
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 14
+; SI-NEXT: v_readlane_b32 s60, v56, 12
+; SI-NEXT: v_readlane_b32 s18, v56, 13
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s59, 24
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s6, 5
-; SI-NEXT: s_and_b32 s6, s88, 0xff
-; SI-NEXT: s_lshl_b32 s7, s75, 8
-; SI-NEXT: s_or_b32 s4, s6, s7
-; SI-NEXT: s_and_b32 s6, s44, 0xff
+; SI-NEXT: s_mov_b32 s17, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 24
+; SI-NEXT: v_readlane_b32 s63, v56, 11
+; SI-NEXT: s_or_b32 s9, s9, s6
+; SI-NEXT: s_and_b32 s6, s18, 0xff
+; SI-NEXT: s_lshl_b32 s13, s60, 8
+; SI-NEXT: s_or_b32 s26, s6, s13
+; SI-NEXT: s_and_b32 s6, s63, 0xff
+; SI-NEXT: v_readlane_b32 s13, v56, 10
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s92, 24
-; SI-NEXT: v_writelane_b32 v46, s4, 12
-; SI-NEXT: s_or_b32 s4, s7, s6
-; SI-NEXT: v_writelane_b32 v46, s4, 6
-; SI-NEXT: s_and_b32 s7, s19, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 20
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s12, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 19
-; SI-NEXT: s_or_b32 s7, s12, s7
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 18
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s18, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 17
-; SI-NEXT: s_or_b32 s18, s18, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 16
-; SI-NEXT: s_lshl_b32 s19, s4, 8
-; SI-NEXT: v_readlane_b32 s4, v56, 15
-; SI-NEXT: s_or_b32 s6, s12, s19
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 14
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s19, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 13
-; SI-NEXT: s_or_b32 s16, s19, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 12
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s20, s4, 24
-; SI-NEXT: v_readlane_b32 s4, v56, 11
-; SI-NEXT: s_or_b32 s5, s20, s12
-; SI-NEXT: s_and_b32 s12, s4, 0xff
-; SI-NEXT: v_readlane_b32 s4, v56, 10
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s20, s4, 24
-; SI-NEXT: s_or_b32 s4, s20, s12
-; SI-NEXT: v_readlane_b32 s12, v56, 9
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s27, s55, 8
-; SI-NEXT: s_or_b32 s52, s12, s27
-; SI-NEXT: s_and_b32 s12, s53, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s27, s54, 24
-; SI-NEXT: s_or_b32 s17, s27, s12
-; SI-NEXT: s_and_b32 s12, s67, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s27, s66, 24
-; SI-NEXT: s_or_b32 s19, s27, s12
-; SI-NEXT: s_and_b32 s12, s64, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_lshl_b32 s44, s65, 24
-; SI-NEXT: s_or_b32 s12, s44, s12
-; SI-NEXT: s_and_b32 s44, s71, 0xff
-; SI-NEXT: s_lshl_b32 s46, s70, 8
-; SI-NEXT: s_or_b32 s53, s44, s46
-; SI-NEXT: s_and_b32 s44, s68, 0xff
-; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_lshl_b32 s46, s69, 24
-; SI-NEXT: s_or_b32 s44, s46, s44
-; SI-NEXT: s_and_b32 s46, s9, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s41, s41, 24
-; SI-NEXT: s_or_b32 s41, s41, s46
-; SI-NEXT: s_and_b32 s46, s80, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s59, s8, 24
-; SI-NEXT: s_or_b32 s59, s59, s46
-; SI-NEXT: s_and_b32 s46, s11, 0xff
-; SI-NEXT: s_lshl_b32 s62, s29, 8
-; SI-NEXT: s_or_b32 s54, s46, s62
-; SI-NEXT: s_and_b32 s46, s24, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s62, s13, 24
-; SI-NEXT: s_or_b32 s27, s62, s46
-; SI-NEXT: s_and_b32 s46, s56, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s62, s40, 24
-; SI-NEXT: s_or_b32 s62, s62, s46
-; SI-NEXT: s_and_b32 s46, s14, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s75, s15, 24
-; SI-NEXT: s_or_b32 s75, s75, s46
-; SI-NEXT: s_and_b32 s46, s45, 0xff
-; SI-NEXT: s_lshl_b32 s77, s43, 8
-; SI-NEXT: s_or_b32 s55, s46, s77
-; SI-NEXT: s_and_b32 s46, s42, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s77, s58, 24
-; SI-NEXT: s_or_b32 s29, s77, s46
-; SI-NEXT: s_and_b32 s46, s72, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s79, s76, 24
-; SI-NEXT: s_or_b32 s46, s79, s46
-; SI-NEXT: s_and_b32 s79, s47, 0xff
-; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s60, 24
-; SI-NEXT: s_or_b32 s24, s88, s79
-; SI-NEXT: s_and_b32 s88, s81, 0xff
-; SI-NEXT: s_lshl_b32 s89, s57, 8
-; SI-NEXT: s_mov_b32 s42, s16
-; SI-NEXT: s_or_b32 s16, s88, s89
-; SI-NEXT: s_and_b32 s88, s91, 0xff
-; SI-NEXT: s_lshl_b32 s88, s88, 16
-; SI-NEXT: s_lshl_b32 s89, s21, 24
-; SI-NEXT: s_or_b32 s21, s89, s88
-; SI-NEXT: s_and_b32 s88, s90, 0xff
-; SI-NEXT: s_lshl_b32 s88, s88, 16
-; SI-NEXT: s_lshl_b32 s90, s99, 24
-; SI-NEXT: s_or_b32 s88, s90, s88
-; SI-NEXT: s_and_b32 s90, s87, 0xff
-; SI-NEXT: s_lshl_b32 s90, s90, 16
-; SI-NEXT: s_lshl_b32 s91, s96, 24
-; SI-NEXT: s_or_b32 s40, s91, s90
-; SI-NEXT: s_and_b32 s91, s73, 0xff
-; SI-NEXT: s_lshl_b32 s92, s25, 8
-; SI-NEXT: s_mov_b32 s45, s19
-; SI-NEXT: s_or_b32 s19, s91, s92
-; SI-NEXT: s_and_b32 s91, s28, 0xff
-; SI-NEXT: s_lshl_b32 s91, s91, 16
-; SI-NEXT: s_lshl_b32 s92, s26, 24
-; SI-NEXT: s_or_b32 s92, s92, s91
-; SI-NEXT: s_and_b32 s91, s94, 0xff
-; SI-NEXT: s_lshl_b32 s91, s91, 16
-; SI-NEXT: s_lshl_b32 s93, s93, 24
-; SI-NEXT: s_or_b32 s91, s93, s91
-; SI-NEXT: s_and_b32 s93, s74, 0xff
-; SI-NEXT: s_lshl_b32 s93, s93, 16
-; SI-NEXT: s_lshl_b32 s94, s97, 24
-; SI-NEXT: s_or_b32 s93, s94, s93
-; SI-NEXT: s_and_b32 s94, s98, 0xff
-; SI-NEXT: s_lshl_b32 s95, s95, 8
-; SI-NEXT: s_or_b32 s22, s94, s95
-; SI-NEXT: s_and_b32 s94, s23, 0xff
-; SI-NEXT: s_lshl_b32 s94, s94, 16
-; SI-NEXT: s_lshl_b32 s95, s50, 24
-; SI-NEXT: s_or_b32 s95, s95, s94
-; SI-NEXT: s_and_b32 s94, s34, 0xff
-; SI-NEXT: s_lshl_b32 s94, s94, 16
-; SI-NEXT: s_lshl_b32 vcc_lo, s49, 24
-; SI-NEXT: s_or_b32 s94, vcc_lo, s94
-; SI-NEXT: s_and_b32 vcc_lo, s61, 0xff
-; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 vcc_hi, s86, 24
-; SI-NEXT: s_or_b32 vcc_lo, vcc_hi, vcc_lo
-; SI-NEXT: s_and_b32 vcc_hi, s84, 0xff
-; SI-NEXT: s_lshl_b32 s30, s30, 8
-; SI-NEXT: s_mov_b32 s43, s17
-; SI-NEXT: s_or_b32 s17, vcc_hi, s30
-; SI-NEXT: s_and_b32 vcc_hi, s38, 0xff
-; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s30, s63, 24
-; SI-NEXT: s_or_b32 s30, s30, vcc_hi
-; SI-NEXT: s_and_b32 vcc_hi, s48, 0xff
-; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s31, s31, 24
-; SI-NEXT: s_or_b32 vcc_hi, s31, vcc_hi
-; SI-NEXT: s_and_b32 s31, s39, 0xff
-; SI-NEXT: s_lshl_b32 s31, s31, 16
-; SI-NEXT: s_lshl_b32 s34, s35, 24
-; SI-NEXT: s_or_b32 s31, s34, s31
-; SI-NEXT: s_and_b32 s34, s82, 0xff
-; SI-NEXT: s_lshl_b32 s35, s85, 8
-; SI-NEXT: s_or_b32 s20, s34, s35
-; SI-NEXT: s_and_b32 s34, s78, 0xff
-; SI-NEXT: s_lshl_b32 s34, s34, 16
-; SI-NEXT: s_lshl_b32 s35, s37, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 31
-; SI-NEXT: s_or_b32 s35, s35, s34
-; SI-NEXT: s_and_b32 s34, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 30
-; SI-NEXT: s_lshl_b32 s34, s34, 16
-; SI-NEXT: s_lshl_b32 s36, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 29
-; SI-NEXT: s_or_b32 s34, s36, s34
-; SI-NEXT: s_and_b32 s36, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 28
-; SI-NEXT: s_lshl_b32 s36, s36, 16
-; SI-NEXT: s_lshl_b32 s37, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 27
-; SI-NEXT: s_or_b32 s36, s37, s36
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 26
-; SI-NEXT: s_lshl_b32 s38, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v56, 25
-; SI-NEXT: s_or_b32 s23, s37, s38
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 24
-; SI-NEXT: s_lshl_b32 s37, s37, 16
-; SI-NEXT: s_lshl_b32 s38, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 23
-; SI-NEXT: s_or_b32 s38, s38, s37
-; SI-NEXT: s_mov_b32 s90, s8
-; SI-NEXT: s_and_b32 s37, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 22
-; SI-NEXT: s_lshl_b32 s37, s37, 16
-; SI-NEXT: s_mov_b32 s28, s8
-; SI-NEXT: s_lshl_b32 s39, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 21
-; SI-NEXT: s_or_b32 s37, s39, s37
-; SI-NEXT: s_and_b32 s39, s8, 0xff
-; SI-NEXT: s_lshl_b32 s39, s39, 16
-; SI-NEXT: s_lshl_b32 s48, s10, 24
-; SI-NEXT: s_or_b32 s39, s48, s39
-; SI-NEXT: s_and_b32 s48, s51, 0xff
-; SI-NEXT: s_lshl_b32 s64, s83, 8
-; SI-NEXT: v_readlane_b32 s8, v46, 8
-; SI-NEXT: s_or_b32 s25, s48, s64
-; SI-NEXT: s_mov_b32 s57, s8
-; SI-NEXT: s_and_b32 s48, s8, 0xff
-; SI-NEXT: v_readlane_b32 s8, v46, 7
-; SI-NEXT: s_mov_b32 s76, s8
-; SI-NEXT: s_lshl_b32 s64, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v46, 9
-; SI-NEXT: s_lshl_b32 s80, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 10
-; SI-NEXT: s_lshl_b32 s71, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 11
-; SI-NEXT: s_lshl_b32 s48, s48, 16
-; SI-NEXT: s_lshl_b32 s70, s8, 16
-; SI-NEXT: v_readlane_b32 s8, v46, 12
-; SI-NEXT: s_mov_b32 s74, s10
-; SI-NEXT: s_mov_b32 s26, s51
-; SI-NEXT: s_mov_b32 s73, s83
-; SI-NEXT: s_or_b32 s48, s64, s48
-; SI-NEXT: s_lshl_b32 s69, s8, 16
-; SI-NEXT: s_lshl_b32 s68, s6, 16
-; SI-NEXT: s_lshl_b32 s67, s52, 16
-; SI-NEXT: s_lshl_b32 s66, s53, 16
-; SI-NEXT: s_lshl_b32 s65, s54, 16
-; SI-NEXT: s_lshl_b32 s64, s55, 16
-; SI-NEXT: s_lshl_b32 s55, s16, 16
-; SI-NEXT: s_lshl_b32 s54, s19, 16
-; SI-NEXT: s_mov_b32 s19, s5
-; SI-NEXT: s_lshl_b32 s53, s22, 16
-; SI-NEXT: s_lshl_b32 s52, s17, 16
-; SI-NEXT: s_lshl_b32 s51, s20, 16
-; SI-NEXT: s_mov_b32 s20, s4
-; SI-NEXT: s_lshl_b32 s50, s23, 16
-; SI-NEXT: s_lshl_b32 s49, s25, 16
+; SI-NEXT: s_mov_b32 s41, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_or_b32 s74, s13, s6
+; SI-NEXT: v_readlane_b32 s6, v46, 22
+; SI-NEXT: s_mov_b32 s79, s80
+; SI-NEXT: s_mov_b32 s59, s77
+; SI-NEXT: s_mov_b32 s21, s18
+; SI-NEXT: s_mov_b32 s18, s60
+; SI-NEXT: s_mov_b32 vcc_lo, s63
+; SI-NEXT: s_lshl_b32 s13, s6, 16
+; SI-NEXT: s_lshl_b32 s42, s93, 16
+; SI-NEXT: s_lshl_b32 s6, s90, 16
+; SI-NEXT: s_lshl_b32 s63, s58, 16
+; SI-NEXT: s_lshl_b32 s73, s45, 16
+; SI-NEXT: s_lshl_b32 s54, s40, 16
+; SI-NEXT: s_lshl_b32 s50, s15, 16
+; SI-NEXT: s_lshl_b32 s38, s51, 16
+; SI-NEXT: s_lshl_b32 s34, s10, 16
+; SI-NEXT: s_lshl_b32 s95, s5, 16
+; SI-NEXT: s_lshl_b32 s90, s29, 16
+; SI-NEXT: s_lshl_b32 s77, s23, 16
+; SI-NEXT: s_lshl_b32 s58, s24, 16
+; SI-NEXT: s_lshl_b32 s45, s25, 16
+; SI-NEXT: s_lshl_b32 s10, s4, 16
+; SI-NEXT: s_lshl_b32 s56, s26, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB89_3
; SI-NEXT: .LBB89_2:
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: v_readlane_b32 s90, v56, 23
+; SI-NEXT: s_mov_b32 s75, s50
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: v_readlane_b32 s28, v56, 22
+; SI-NEXT: s_mov_b32 s79, vcc_lo
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s26, s10
+; SI-NEXT: s_mov_b32 s20, s26
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s74, s5
+; SI-NEXT: s_mov_b32 s91, s93
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s73, s51
+; SI-NEXT: v_readlane_b32 s59, v56, 16
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s57, s4
+; SI-NEXT: v_readlane_b32 s21, v56, 13
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: v_readlane_b32 s72, v56, 15
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s17, v56, 14
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s18, v56, 12
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 vcc_lo, v56, 11
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: v_readlane_b32 s41, v56, 10
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s76, s83
; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr13
+; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr18
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr42
-; SI-NEXT: ; implicit-def: $sgpr19
-; SI-NEXT: ; implicit-def: $sgpr20
-; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr41
-; SI-NEXT: ; implicit-def: $sgpr59
-; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr27
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr75
-; SI-NEXT: ; implicit-def: $sgpr64
-; SI-NEXT: ; implicit-def: $sgpr29
-; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr24
-; SI-NEXT: ; implicit-def: $sgpr55
-; SI-NEXT: ; implicit-def: $sgpr21
-; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr73
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr53
-; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr94
-; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr52
-; SI-NEXT: ; implicit-def: $sgpr30
-; SI-NEXT: ; implicit-def: $vcc_hi
-; SI-NEXT: ; implicit-def: $sgpr31
-; SI-NEXT: ; implicit-def: $sgpr51
-; SI-NEXT: ; implicit-def: $sgpr35
-; SI-NEXT: ; implicit-def: $sgpr34
-; SI-NEXT: ; implicit-def: $sgpr36
; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr38
-; SI-NEXT: ; implicit-def: $sgpr37
-; SI-NEXT: ; implicit-def: $sgpr39
; SI-NEXT: ; implicit-def: $sgpr49
-; SI-NEXT: ; implicit-def: $sgpr48
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr16
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr22
+; SI-NEXT: ; implicit-def: $sgpr95
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr90
+; SI-NEXT: ; implicit-def: $sgpr92
+; SI-NEXT: ; implicit-def: $sgpr88
+; SI-NEXT: ; implicit-def: $sgpr89
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr58
+; SI-NEXT: ; implicit-def: $sgpr61
+; SI-NEXT: ; implicit-def: $sgpr47
+; SI-NEXT: ; implicit-def: $sgpr57
+; SI-NEXT: ; implicit-def: $sgpr45
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr44
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr56
+; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; kill: killed $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: .LBB89_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: v_readlane_b32 s81, v47, 59
-; SI-NEXT: v_readlane_b32 s82, v47, 60
-; SI-NEXT: v_readlane_b32 s84, v47, 61
-; SI-NEXT: v_readlane_b32 s83, v47, 62
-; SI-NEXT: v_readlane_b32 s85, v47, 63
-; SI-NEXT: v_readlane_b32 s87, v46, 0
-; SI-NEXT: v_readlane_b32 s86, v46, 1
-; SI-NEXT: v_readlane_b32 s96, v46, 2
-; SI-NEXT: v_readlane_b32 s98, v46, 3
-; SI-NEXT: v_readlane_b32 s97, v46, 4
-; SI-NEXT: v_readlane_b32 s99, v46, 5
-; SI-NEXT: v_readlane_b32 s6, v46, 6
+; SI-NEXT: v_readlane_b32 s4, v46, 1
+; SI-NEXT: v_readlane_b32 s15, v46, 7
+; SI-NEXT: s_mov_b32 s37, s7
+; SI-NEXT: s_mov_b32 s7, s73
+; SI-NEXT: v_readlane_b32 s84, v47, 60
+; SI-NEXT: v_readlane_b32 s85, v47, 61
+; SI-NEXT: v_readlane_b32 s81, v47, 62
+; SI-NEXT: v_readlane_b32 s83, v47, 63
+; SI-NEXT: v_readlane_b32 s82, v46, 0
+; SI-NEXT: v_readlane_b32 s43, v46, 2
+; SI-NEXT: v_readlane_b32 s80, v46, 3
+; SI-NEXT: v_readlane_b32 s69, v46, 4
+; SI-NEXT: v_readlane_b32 s71, v46, 5
+; SI-NEXT: v_readlane_b32 s70, v46, 6
+; SI-NEXT: v_readlane_b32 s52, v46, 8
+; SI-NEXT: v_readlane_b32 s53, v46, 9
+; SI-NEXT: v_readlane_b32 s48, v46, 10
+; SI-NEXT: v_readlane_b32 s51, v46, 11
+; SI-NEXT: v_readlane_b32 s39, v46, 12
+; SI-NEXT: s_mov_b32 s30, s55
+; SI-NEXT: s_mov_b32 s35, s11
+; SI-NEXT: s_mov_b32 s93, s65
+; SI-NEXT: s_mov_b32 vcc_hi, s64
+; SI-NEXT: s_mov_b32 s5, s74
+; SI-NEXT: s_mov_b32 s74, s4
+; SI-NEXT: s_mov_b32 s4, s56
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: v_readlane_b32 s68, v46, 13
+; SI-NEXT: v_readlane_b32 s67, v46, 14
+; SI-NEXT: v_readlane_b32 s66, v46, 15
+; SI-NEXT: s_mov_b32 s40, s42
+; SI-NEXT: s_mov_b32 s42, s7
+; SI-NEXT: v_readlane_b32 s65, v46, 16
+; SI-NEXT: v_readlane_b32 s64, v46, 17
+; SI-NEXT: v_readlane_b32 s55, v46, 18
+; SI-NEXT: s_mov_b32 s56, s13
+; SI-NEXT: v_readlane_b32 s60, v47, 50
+; SI-NEXT: v_readlane_b32 s96, v47, 51
+; SI-NEXT: v_readlane_b32 s97, v47, 52
+; SI-NEXT: v_readlane_b32 s98, v47, 53
+; SI-NEXT: v_readlane_b32 s29, v47, 54
+; SI-NEXT: v_readlane_b32 s99, v47, 55
+; SI-NEXT: v_readlane_b32 s28, v47, 58
; SI-NEXT: s_cbranch_scc1 .LBB89_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s4, s26, 3
+; SI-NEXT: s_add_i32 s4, s21, 3
; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s73, 8
-; SI-NEXT: s_add_i32 s6, s57, 3
+; SI-NEXT: s_lshl_b32 s5, s18, 8
+; SI-NEXT: s_add_i32 s6, vcc_lo, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s76, 24
+; SI-NEXT: s_lshl_b32 s5, s41, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s5, s90, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 21
-; SI-NEXT: s_and_b32 s5, s5, 0xff
-; SI-NEXT: s_lshl_b32 s6, s28, 8
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: s_or_b32 s5, s6, s5
+; SI-NEXT: s_or_b32 s5, s5, s4
+; SI-NEXT: s_add_i32 s4, s20, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s6, s91, 8
+; SI-NEXT: s_add_i32 s7, s72, 3
+; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s7, s7, 0xff
-; SI-NEXT: s_lshl_b32 s6, s74, 24
+; SI-NEXT: s_lshl_b32 s6, s17, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_addk_i32 s5, 0x300
+; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s6, s6, s7
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 27
-; SI-NEXT: s_add_i32 s21, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 26
-; SI-NEXT: v_readlane_b32 s8, v56, 25
-; SI-NEXT: s_and_b32 s6, s21, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s8, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 24
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 31
-; SI-NEXT: s_add_i32 s10, s7, 3
-; SI-NEXT: v_readlane_b32 s8, v56, 30
-; SI-NEXT: s_and_b32 s7, s10, 0xff
-; SI-NEXT: s_lshl_b32 s10, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v56, 28
-; SI-NEXT: s_or_b32 s7, s10, s7
-; SI-NEXT: s_lshl_b32 s10, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v56, 29
-; SI-NEXT: s_add_i32 s12, s8, 3
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_or_b32 s6, s6, s4
+; SI-NEXT: s_add_i32 s4, s75, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s7, s79, 8
+; SI-NEXT: s_or_b32 s7, s7, s4
+; SI-NEXT: v_readlane_b32 s4, v56, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s8, s59, 24
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s7, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 50
-; SI-NEXT: s_or_b32 s10, s10, s12
+; SI-NEXT: s_or_b32 s4, s8, s4
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_add_i32 s57, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 41
-; SI-NEXT: s_or_b32 s7, s10, s7
-; SI-NEXT: s_and_b32 s10, s57, 0xff
-; SI-NEXT: s_lshl_b32 s12, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 33
-; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_lshl_b32 s12, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 35
-; SI-NEXT: s_add_i32 s26, s8, 3
-; SI-NEXT: s_and_b32 s13, s26, 0xff
-; SI-NEXT: v_readlane_b32 s8, v47, 55
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_add_i32 s73, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 51
-; SI-NEXT: s_or_b32 s12, s12, s13
+; SI-NEXT: s_or_b32 s7, s4, s7
+; SI-NEXT: v_readlane_b32 s4, v56, 19
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s8, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 48
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s8, s4, s8
+; SI-NEXT: v_readlane_b32 s4, v56, 18
+; SI-NEXT: s_lshl_b32 s9, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 47
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s8, 0x300
+; SI-NEXT: s_or_b32 s4, s9, s4
+; SI-NEXT: s_and_b32 s8, s8, 0xffff
+; SI-NEXT: s_or_b32 s8, s4, s8
+; SI-NEXT: v_readlane_b32 s4, v56, 23
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s9, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 22
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s9, s4, s9
+; SI-NEXT: v_readlane_b32 s4, v56, 20
+; SI-NEXT: s_lshl_b32 s10, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 21
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s9, 0x300
+; SI-NEXT: s_or_b32 s4, s10, s4
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_or_b32 s9, s4, s9
+; SI-NEXT: v_readlane_b32 s4, v56, 27
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s10, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 26
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s10, s4, s10
+; SI-NEXT: v_readlane_b32 s4, v56, 24
+; SI-NEXT: s_lshl_b32 s11, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 25
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s13, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 39
+; SI-NEXT: s_or_b32 s4, s11, s4
; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 40
-; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s12, s73, 0xff
-; SI-NEXT: s_add_i32 s11, s8, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_addk_i32 s12, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 56
-; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: s_and_b32 s11, s12, 0xffff
-; SI-NEXT: s_add_i32 s74, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 53
-; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: s_and_b32 s11, s74, 0xff
-; SI-NEXT: s_lshl_b32 s12, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 43
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_lshl_b32 s12, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 46
-; SI-NEXT: s_add_i32 s43, s8, 3
-; SI-NEXT: s_and_b32 s13, s43, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s10, s4, s10
+; SI-NEXT: v_readlane_b32 s4, v56, 31
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s11, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v56, 30
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s11, s4, s11
+; SI-NEXT: v_readlane_b32 s4, v56, 28
+; SI-NEXT: s_lshl_b32 s12, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v56, 29
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s11, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 58
-; SI-NEXT: s_or_b32 s12, s12, s13
+; SI-NEXT: s_or_b32 s4, s12, s4
; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_add_i32 s78, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 57
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s78, 0xff
-; SI-NEXT: s_lshl_b32 s13, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_lshl_b32 s13, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 47
-; SI-NEXT: s_add_i32 s45, s8, 3
-; SI-NEXT: s_and_b32 s15, s45, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s11, s4, s11
+; SI-NEXT: v_readlane_b32 s4, v47, 46
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s12, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 45
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s12, s4, s12
+; SI-NEXT: v_readlane_b32 s4, v47, 44
+; SI-NEXT: s_lshl_b32 s13, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 43
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s12, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 54
-; SI-NEXT: s_or_b32 s13, s13, s15
+; SI-NEXT: s_or_b32 s4, s13, s4
; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_add_i32 s72, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 52
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s72, 0xff
-; SI-NEXT: s_lshl_b32 s15, s8, 8
-; SI-NEXT: v_readlane_b32 s8, v47, 49
-; SI-NEXT: s_or_b32 s13, s15, s13
-; SI-NEXT: s_lshl_b32 s15, s8, 24
-; SI-NEXT: v_readlane_b32 s8, v47, 48
-; SI-NEXT: s_add_i32 s47, s8, 3
-; SI-NEXT: s_and_b32 s18, s47, 0xff
-; SI-NEXT: s_lshl_b32 s18, s18, 16
+; SI-NEXT: s_or_b32 s12, s4, s12
+; SI-NEXT: v_readlane_b32 s4, v47, 41
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s13, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 40
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s13, s4, s13
+; SI-NEXT: v_readlane_b32 s4, v47, 42
+; SI-NEXT: s_lshl_b32 s14, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 39
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s13, 0x300
-; SI-NEXT: v_readlane_b32 s8, v47, 45
-; SI-NEXT: s_or_b32 s15, s15, s18
+; SI-NEXT: s_or_b32 s4, s14, s4
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_add_i32 s42, s8, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 42
-; SI-NEXT: s_or_b32 s13, s15, s13
-; SI-NEXT: s_and_b32 s15, s42, 0xff
-; SI-NEXT: s_lshl_b32 s14, s8, 8
-; SI-NEXT: s_or_b32 s14, s14, s15
-; SI-NEXT: v_readlane_b32 s15, v47, 37
-; SI-NEXT: s_add_i32 s29, s15, 3
-; SI-NEXT: v_readlane_b32 s8, v47, 38
-; SI-NEXT: s_and_b32 s15, s29, 0xff
-; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s13, s4, s13
+; SI-NEXT: v_readlane_b32 s4, v47, 37
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s14, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 36
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s14, s4, s14
+; SI-NEXT: v_readlane_b32 s4, v47, 38
+; SI-NEXT: s_lshl_b32 s15, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 35
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_addk_i32 s14, 0x300
-; SI-NEXT: s_or_b32 s8, s8, s15
+; SI-NEXT: s_or_b32 s4, s15, s4
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_or_b32 s14, s8, s14
-; SI-NEXT: v_readlane_b32 s8, v47, 36
-; SI-NEXT: s_add_i32 s24, s8, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 34
-; SI-NEXT: v_readlane_b32 s16, v47, 31
-; SI-NEXT: s_and_b32 s8, s24, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s17, s16, 3
-; SI-NEXT: s_or_b32 s8, s15, s8
-; SI-NEXT: v_readlane_b32 s15, v47, 32
-; SI-NEXT: s_and_b32 s16, s17, 0xff
-; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: s_or_b32 s14, s4, s14
+; SI-NEXT: v_readlane_b32 s4, v47, 34
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s15, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s15, s4, s15
+; SI-NEXT: v_readlane_b32 s4, v47, 33
+; SI-NEXT: s_lshl_b32 s16, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 31
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s15, 0x300
+; SI-NEXT: s_or_b32 s4, s16, s4
+; SI-NEXT: s_and_b32 s15, s15, 0xffff
+; SI-NEXT: s_or_b32 s15, s4, s15
+; SI-NEXT: v_readlane_b32 s4, v47, 28
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s16, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v47, 30
+; SI-NEXT: s_lshl_b32 s4, s4, 8
+; SI-NEXT: s_or_b32 s16, s4, s16
+; SI-NEXT: v_readlane_b32 s4, v47, 29
+; SI-NEXT: s_lshl_b32 s17, s4, 24
+; SI-NEXT: v_readlane_b32 s4, v47, 27
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_or_b32 s4, s17, s4
+; SI-NEXT: s_and_b32 s16, s16, 0xffff
+; SI-NEXT: s_or_b32 s16, s4, s16
+; SI-NEXT: s_add_i32 s4, s5, 0x3000000
+; SI-NEXT: s_add_i32 s5, s6, 0x3000000
+; SI-NEXT: s_add_i32 s6, s7, 0x3000000
+; SI-NEXT: s_add_i32 s7, s8, 0x3000000
+; SI-NEXT: s_add_i32 s8, s9, 0x3000000
+; SI-NEXT: s_add_i32 s9, s10, 0x3000000
+; SI-NEXT: s_add_i32 s10, s11, 0x3000000
+; SI-NEXT: s_add_i32 s11, s12, 0x3000000
+; SI-NEXT: s_add_i32 s12, s13, 0x3000000
+; SI-NEXT: s_add_i32 s13, s14, 0x3000000
+; SI-NEXT: s_add_i32 s14, s15, 0x3000000
+; SI-NEXT: s_add_i32 s15, s16, 0x3000000
+; SI-NEXT: v_readlane_b32 s16, v47, 19
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 18
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 17
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 16
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_or_b32 s15, s15, s16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s15, s15, s8
-; SI-NEXT: v_readlane_b32 s8, v47, 30
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 29
-; SI-NEXT: v_readlane_b32 s17, v47, 27
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s40, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 15
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 11
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s41, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 4
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 3
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 2
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s44, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 1
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v56, 63
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v56, 62
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s45, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v56, 60
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v56, 58
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s46, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v56, 56
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v56, 57
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 26
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 25
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s47, s16, s17
+; SI-NEXT: v_readlane_b32 s16, v47, 24
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s17, s16, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 23
+; SI-NEXT: s_lshl_b32 s16, s16, 8
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: s_addk_i32 s16, 0x300
+; SI-NEXT: s_and_b32 s17, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 22
+; SI-NEXT: s_lshl_b32 s18, s16, 24
+; SI-NEXT: v_readlane_b32 s16, v47, 21
+; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s16, s16, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_or_b32 s16, s16, s17
+; SI-NEXT: v_readlane_b32 s17, v47, 20
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s18, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v47, 14
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s18, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v47, 13
+; SI-NEXT: s_lshl_b32 s20, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v47, 12
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_or_b32 s8, s16, s8
-; SI-NEXT: v_readlane_b32 s16, v47, 28
; SI-NEXT: s_and_b32 s17, s17, 0xff
-; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_add_i32 s23, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v47, 26
-; SI-NEXT: s_or_b32 s16, s16, s8
-; SI-NEXT: s_add_i32 s22, s7, 0x3000000
-; SI-NEXT: s_add_i32 s8, s10, 0x3000000
-; SI-NEXT: s_add_i32 s10, s11, 0x3000000
-; SI-NEXT: s_add_i32 s11, s12, 0x3000000
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 25
-; SI-NEXT: v_readlane_b32 s12, v47, 23
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 24
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s17, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 22
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 21
-; SI-NEXT: v_readlane_b32 s12, v47, 19
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 20
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s21, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 18
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 17
-; SI-NEXT: v_readlane_b32 s12, v47, 15
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 16
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s24, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 14
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 13
-; SI-NEXT: v_readlane_b32 s12, v47, 11
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s25, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 10
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 9
-; SI-NEXT: v_readlane_b32 s12, v47, 7
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 8
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s26, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 6
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_and_b32 s7, s6, 0xff
-; SI-NEXT: v_readlane_b32 s6, v56, 32
-; SI-NEXT: v_readlane_b32 s12, v47, 4
-; SI-NEXT: s_lshl_b32 s6, s6, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s6, s7
-; SI-NEXT: v_readlane_b32 s7, v47, 5
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s29, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v47, 3
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 2
-; SI-NEXT: v_readlane_b32 s12, v47, 0
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 1
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s40, s7, s6
-; SI-NEXT: v_readlane_b32 s6, v56, 63
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 62
-; SI-NEXT: v_readlane_b32 s12, v56, 60
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 61
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s27, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 9
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 59
-; SI-NEXT: v_readlane_b32 s12, v56, 57
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 58
-; SI-NEXT: s_and_b32 s12, s12, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s12
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s12, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 13
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 12
-; SI-NEXT: v_readlane_b32 s18, v56, 11
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 10
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s28, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 17
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 16
-; SI-NEXT: v_readlane_b32 s18, v56, 15
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 14
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s20, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 47
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v56, 20
-; SI-NEXT: v_readlane_b32 s18, v56, 19
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v56, 18
-; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s7, 24
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s18
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 49
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 45
-; SI-NEXT: v_readlane_b32 s19, v56, 39
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s18, s18, 8
-; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: v_readlane_b32 s18, v56, 38
-; SI-NEXT: s_and_b32 s19, s19, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s18, s18, 24
-; SI-NEXT: s_lshl_b32 s19, s19, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s18, s18, s19
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: s_add_i32 s18, s6, 0x3000000
-; SI-NEXT: v_readlane_b32 s6, v56, 51
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s19, v56, 48
-; SI-NEXT: v_readlane_b32 s41, v56, 41
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s19, s19, 8
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: v_readlane_b32 s19, v56, 40
-; SI-NEXT: s_and_b32 s41, s41, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s19, s19, 24
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s19, s19, s41
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: v_readlane_b32 s19, v56, 52
-; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: v_readlane_b32 s41, v56, 50
-; SI-NEXT: v_readlane_b32 s42, v56, 43
-; SI-NEXT: s_and_b32 s19, s19, 0xff
-; SI-NEXT: s_lshl_b32 s41, s41, 8
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: s_or_b32 s19, s41, s19
-; SI-NEXT: v_readlane_b32 s41, v56, 42
+; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_add_i32 s18, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 55
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s20, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v47, 10
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s21, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v47, 9
+; SI-NEXT: s_lshl_b32 s20, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v47, 8
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_add_i32 s57, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s20, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 52
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s20, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 53
+; SI-NEXT: s_lshl_b32 s21, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 51
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s21, s17
+; SI-NEXT: s_or_b32 s17, s17, s20
+; SI-NEXT: s_add_i32 s20, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 50
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s21, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s21, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 49
+; SI-NEXT: s_lshl_b32 s22, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s22, s17
+; SI-NEXT: s_or_b32 s17, s17, s21
+; SI-NEXT: s_add_i32 s21, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s22, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s22
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s22, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 44
+; SI-NEXT: s_lshl_b32 s23, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 43
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s23, s17
+; SI-NEXT: s_or_b32 s17, s17, s22
+; SI-NEXT: s_add_i32 s22, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 42
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s23, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s23
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s23, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 41
+; SI-NEXT: s_lshl_b32 s24, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 38
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s24, s17
+; SI-NEXT: s_or_b32 s17, s17, s23
+; SI-NEXT: s_add_i32 s23, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s24, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s24
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s24, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 36
+; SI-NEXT: s_lshl_b32 s25, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s25, s17
+; SI-NEXT: s_or_b32 s17, s17, s24
+; SI-NEXT: s_add_i32 s24, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 4
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s25, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 3
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s25
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s25, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 32
+; SI-NEXT: s_lshl_b32 s26, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 33
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s26, s17
+; SI-NEXT: s_or_b32 s17, s17, s25
+; SI-NEXT: s_add_i32 s25, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v56, 8
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s26, s17, 0xff
+; SI-NEXT: v_readlane_b32 s17, v56, 7
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_or_b32 s17, s17, s26
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s26, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 5
+; SI-NEXT: s_lshl_b32 s27, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 6
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s27, s17
+; SI-NEXT: s_or_b32 s17, s17, s26
+; SI-NEXT: s_add_i32 s26, s17, 0x3000000
+; SI-NEXT: v_readlane_b32 s17, v47, 59
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s27, s28, 8
+; SI-NEXT: s_or_b32 s17, s27, s17
+; SI-NEXT: s_addk_i32 s17, 0x300
+; SI-NEXT: s_and_b32 s27, s17, 0xffff
+; SI-NEXT: v_readlane_b32 s17, v56, 9
+; SI-NEXT: s_lshl_b32 s28, s17, 24
+; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_and_b32 s17, s17, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_or_b32 s17, s28, s17
+; SI-NEXT: s_or_b32 s17, s17, s27
+; SI-NEXT: v_readlane_b32 s27, v47, 57
+; SI-NEXT: s_add_i32 s27, s27, 3
+; SI-NEXT: v_readlane_b32 s28, v47, 56
+; SI-NEXT: s_and_b32 s27, s27, 0xff
+; SI-NEXT: s_lshl_b32 s28, s28, 8
+; SI-NEXT: s_add_i32 s29, s29, 3
+; SI-NEXT: s_or_b32 s27, s28, s27
+; SI-NEXT: s_and_b32 s29, s29, 0xff
+; SI-NEXT: s_addk_i32 s27, 0x300
+; SI-NEXT: s_lshl_b32 s28, s99, 24
+; SI-NEXT: s_lshl_b32 s29, s29, 16
+; SI-NEXT: s_and_b32 s27, s27, 0xffff
+; SI-NEXT: s_or_b32 s28, s28, s29
+; SI-NEXT: s_or_b32 s27, s28, s27
+; SI-NEXT: s_add_i32 s28, s98, 3
+; SI-NEXT: s_and_b32 s28, s28, 0xff
+; SI-NEXT: s_lshl_b32 s29, s97, 8
+; SI-NEXT: s_add_i32 s42, s60, 3
+; SI-NEXT: s_or_b32 s28, s29, s28
; SI-NEXT: s_and_b32 s42, s42, 0xff
-; SI-NEXT: s_addk_i32 s19, 0x300
-; SI-NEXT: s_lshl_b32 s41, s41, 24
+; SI-NEXT: s_addk_i32 s28, 0x300
+; SI-NEXT: s_lshl_b32 s29, s96, 24
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_and_b32 s19, s19, 0xffff
-; SI-NEXT: s_or_b32 s41, s41, s42
-; SI-NEXT: s_or_b32 s19, s41, s19
-; SI-NEXT: v_readlane_b32 s41, v56, 55
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: v_readlane_b32 s42, v56, 53
-; SI-NEXT: v_readlane_b32 s43, v56, 46
-; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_and_b32 s28, s28, 0xffff
+; SI-NEXT: s_or_b32 s29, s29, s42
+; SI-NEXT: s_or_b32 s28, s29, s28
+; SI-NEXT: v_readlane_b32 s29, v56, 2
+; SI-NEXT: s_add_i32 s29, s29, 3
+; SI-NEXT: v_readlane_b32 s42, v56, 1
+; SI-NEXT: v_readlane_b32 s43, v47, 49
+; SI-NEXT: s_and_b32 s29, s29, 0xff
; SI-NEXT: s_lshl_b32 s42, s42, 8
; SI-NEXT: s_add_i32 s43, s43, 3
-; SI-NEXT: s_or_b32 s41, s42, s41
-; SI-NEXT: v_readlane_b32 s42, v56, 44
+; SI-NEXT: s_or_b32 s29, s42, s29
+; SI-NEXT: v_readlane_b32 s19, v56, 0
; SI-NEXT: s_and_b32 s43, s43, 0xff
-; SI-NEXT: s_addk_i32 s41, 0x300
-; SI-NEXT: s_lshl_b32 s42, s42, 24
+; SI-NEXT: s_addk_i32 s29, 0x300
+; SI-NEXT: s_lshl_b32 s42, s19, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_and_b32 s41, s41, 0xffff
+; SI-NEXT: s_and_b32 s29, s29, 0xffff
; SI-NEXT: s_or_b32 s42, s42, s43
-; SI-NEXT: s_or_b32 s41, s42, s41
-; SI-NEXT: v_readlane_b32 s42, v56, 1
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: v_readlane_b32 s43, v56, 0
-; SI-NEXT: v_readlane_b32 s44, v56, 56
-; SI-NEXT: s_and_b32 s42, s42, 0xff
-; SI-NEXT: s_lshl_b32 s43, s43, 8
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: s_or_b32 s42, s43, s42
-; SI-NEXT: v_readlane_b32 s43, v56, 54
-; SI-NEXT: s_and_b32 s44, s44, 0xff
-; SI-NEXT: s_addk_i32 s42, 0x300
-; SI-NEXT: s_lshl_b32 s43, s43, 24
-; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_and_b32 s42, s42, 0xffff
-; SI-NEXT: s_or_b32 s43, s43, s44
-; SI-NEXT: s_or_b32 s42, s43, s42
-; SI-NEXT: v_readlane_b32 s43, v56, 5
-; SI-NEXT: s_add_i32 s43, s43, 3
-; SI-NEXT: v_readlane_b32 s44, v56, 4
-; SI-NEXT: v_readlane_b32 s45, v56, 3
-; SI-NEXT: s_and_b32 s43, s43, 0xff
-; SI-NEXT: s_lshl_b32 s44, s44, 8
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: s_or_b32 s43, s44, s43
-; SI-NEXT: v_readlane_b32 s44, v56, 2
-; SI-NEXT: s_and_b32 s45, s45, 0xff
-; SI-NEXT: s_addk_i32 s43, 0x300
-; SI-NEXT: s_lshl_b32 s44, s44, 24
-; SI-NEXT: s_lshl_b32 s45, s45, 16
-; SI-NEXT: s_and_b32 s43, s43, 0xffff
-; SI-NEXT: s_or_b32 s44, s44, s45
-; SI-NEXT: s_or_b32 s43, s44, s43
-; SI-NEXT: v_readlane_b32 s44, v56, 37
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: v_readlane_b32 s45, v56, 8
-; SI-NEXT: v_readlane_b32 s46, v56, 7
-; SI-NEXT: s_and_b32 s44, s44, 0xff
-; SI-NEXT: s_lshl_b32 s45, s45, 8
-; SI-NEXT: s_add_i32 s46, s46, 3
-; SI-NEXT: s_or_b32 s44, s45, s44
-; SI-NEXT: v_readlane_b32 s45, v56, 6
-; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_addk_i32 s44, 0x300
-; SI-NEXT: s_lshl_b32 s45, s45, 24
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_or_b32 s45, s45, s46
-; SI-NEXT: s_or_b32 s44, s45, s44
-; SI-NEXT: v_readlane_b32 s45, v56, 36
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: v_readlane_b32 s46, v56, 35
-; SI-NEXT: v_readlane_b32 s47, v56, 33
-; SI-NEXT: s_and_b32 s45, s45, 0xff
-; SI-NEXT: s_lshl_b32 s46, s46, 8
-; SI-NEXT: s_add_i32 s47, s47, 3
-; SI-NEXT: s_or_b32 s45, s46, s45
-; SI-NEXT: v_readlane_b32 s46, v56, 34
-; SI-NEXT: s_and_b32 s47, s47, 0xff
-; SI-NEXT: s_addk_i32 s45, 0x300
-; SI-NEXT: s_lshl_b32 s46, s46, 24
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_and_b32 s45, s45, 0xffff
-; SI-NEXT: s_or_b32 s46, s46, s47
-; SI-NEXT: s_or_b32 s45, s46, s45
-; SI-NEXT: s_add_i32 s4, s4, 0x3000000
-; SI-NEXT: s_add_i32 s5, s5, 0x3000000
-; SI-NEXT: s_add_i32 s9, s9, 0x3000000
-; SI-NEXT: s_add_i32 s13, s13, 0x3000000
-; SI-NEXT: s_add_i32 s14, s14, 0x3000000
-; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_add_i32 s16, s16, 0x3000000
; SI-NEXT: s_add_i32 s17, s17, 0x3000000
-; SI-NEXT: s_add_i32 s21, s21, 0x3000000
-; SI-NEXT: s_add_i32 s24, s24, 0x3000000
-; SI-NEXT: s_add_i32 s25, s25, 0x3000000
-; SI-NEXT: s_add_i32 s26, s26, 0x3000000
-; SI-NEXT: s_add_i32 s29, s29, 0x3000000
+; SI-NEXT: s_or_b32 s29, s42, s29
; SI-NEXT: s_add_i32 s40, s40, 0x3000000
-; SI-NEXT: s_add_i32 s6, s6, 0x3000000
-; SI-NEXT: s_add_i32 s19, s19, 0x3000000
; SI-NEXT: s_add_i32 s41, s41, 0x3000000
-; SI-NEXT: s_add_i32 s42, s42, 0x3000000
-; SI-NEXT: s_add_i32 s43, s43, 0x3000000
; SI-NEXT: s_add_i32 s44, s44, 0x3000000
; SI-NEXT: s_add_i32 s45, s45, 0x3000000
-; SI-NEXT: s_and_b32 s82, s45, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s45, 16
-; SI-NEXT: s_and_b32 s83, s44, 0xffff0000
-; SI-NEXT: s_lshl_b32 s80, s44, 16
-; SI-NEXT: s_and_b32 s85, s43, 0xffff0000
-; SI-NEXT: s_lshl_b32 s84, s43, 16
-; SI-NEXT: s_and_b32 s86, s42, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s42, 16
-; SI-NEXT: s_and_b32 s96, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s87, s41, 16
-; SI-NEXT: s_and_b32 s97, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s70, s19, 16
-; SI-NEXT: s_and_b32 s99, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s98, s6, 16
-; SI-NEXT: s_and_b32 s6, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s18, 16
-; SI-NEXT: s_and_b32 s18, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s42, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s68, s20, 16
-; SI-NEXT: s_and_b32 s20, s28, 0xffff0000
-; SI-NEXT: s_lshl_b32 s19, s28, 16
-; SI-NEXT: s_and_b32 s43, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s12, 16
-; SI-NEXT: s_and_b32 s12, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s27, 16
-; SI-NEXT: s_and_b32 s44, s40, 0xffff0000
-; SI-NEXT: s_lshl_b32 s66, s40, 16
-; SI-NEXT: s_and_b32 s59, s29, 0xffff0000
-; SI-NEXT: s_lshl_b32 s41, s29, 16
-; SI-NEXT: s_and_b32 s27, s26, 0xffff0000
-; SI-NEXT: s_lshl_b32 s65, s26, 16
-; SI-NEXT: s_and_b32 s75, s25, 0xffff0000
-; SI-NEXT: s_lshl_b32 s62, s25, 16
-; SI-NEXT: s_and_b32 s29, s24, 0xffff0000
-; SI-NEXT: s_lshl_b32 s64, s24, 16
-; SI-NEXT: s_and_b32 s24, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s46, s21, 16
-; SI-NEXT: s_and_b32 s21, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s55, s17, 16
-; SI-NEXT: s_and_b32 s40, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s88, s16, 16
-; SI-NEXT: s_and_b32 s92, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s54, s15, 16
-; SI-NEXT: s_and_b32 s93, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s91, s14, 16
-; SI-NEXT: s_and_b32 s95, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s53, s13, 16
-; SI-NEXT: s_and_b32 vcc_lo, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s11, 16
-; SI-NEXT: s_and_b32 s30, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s52, s10, 16
-; SI-NEXT: s_and_b32 s31, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_hi, s9, 16
-; SI-NEXT: s_and_b32 s35, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s51, s8, 16
-; SI-NEXT: s_and_b32 s36, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s34, s22, 16
-; SI-NEXT: s_and_b32 s38, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s50, s23, 16
-; SI-NEXT: s_and_b32 s39, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s37, s5, 16
-; SI-NEXT: s_and_b32 s48, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s49, s4, 16
+; SI-NEXT: s_add_i32 s46, s46, 0x3000000
+; SI-NEXT: s_add_i32 s47, s47, 0x3000000
+; SI-NEXT: s_add_i32 s16, s16, 0x3000000
+; SI-NEXT: s_add_i32 s27, s27, 0x3000000
+; SI-NEXT: s_add_i32 s28, s28, 0x3000000
+; SI-NEXT: s_add_i32 s29, s29, 0x3000000
+; SI-NEXT: s_and_b32 s43, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s17, s17, 16
+; SI-NEXT: s_and_b32 s71, s25, 0xffff0000
+; SI-NEXT: s_lshl_b32 s25, s25, 16
+; SI-NEXT: s_and_b32 s85, s29, 0xffff0000
+; SI-NEXT: s_lshl_b32 s84, s29, 16
+; SI-NEXT: s_and_b32 s83, s28, 0xffff0000
+; SI-NEXT: s_lshl_b32 s56, s28, 16
+; SI-NEXT: s_and_b32 s82, s27, 0xffff0000
+; SI-NEXT: s_lshl_b32 s81, s27, 16
+; SI-NEXT: s_and_b32 s80, s26, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s26, 16
+; SI-NEXT: s_and_b32 s70, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 s69, s24, 16
+; SI-NEXT: s_and_b32 s73, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s23, 16
+; SI-NEXT: s_and_b32 s68, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s67, s22, 16
+; SI-NEXT: s_and_b32 s66, s21, 0xffff0000
+; SI-NEXT: s_lshl_b32 s42, s21, 16
+; SI-NEXT: s_and_b32 s65, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s20, 16
+; SI-NEXT: s_and_b32 s55, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s57, 16
+; SI-NEXT: s_and_b32 s53, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s18, 16
+; SI-NEXT: s_and_b32 s51, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s50, s16, 16
+; SI-NEXT: s_and_b32 s49, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s48, s47, 16
+; SI-NEXT: s_and_b32 s39, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s46, 16
+; SI-NEXT: s_and_b32 s37, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s16, s45, 16
+; SI-NEXT: s_and_b32 s35, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s44, 16
+; SI-NEXT: s_and_b32 s22, s41, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s41, 16
+; SI-NEXT: s_and_b32 vcc_hi, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s95, s40, 16
+; SI-NEXT: s_mov_b32 s40, s17
+; SI-NEXT: s_and_b32 s94, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s15, 16
+; SI-NEXT: s_and_b32 s92, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s90, s14, 16
+; SI-NEXT: s_and_b32 s89, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s88, s13, 16
+; SI-NEXT: s_and_b32 s78, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s12, 16
+; SI-NEXT: s_and_b32 s76, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s62, s11, 16
+; SI-NEXT: s_and_b32 s61, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s58, s10, 16
+; SI-NEXT: s_and_b32 s57, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s9, 16
+; SI-NEXT: s_and_b32 s46, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s45, s8, 16
+; SI-NEXT: s_and_b32 s44, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s7, 16
+; SI-NEXT: s_and_b32 s12, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s6, 16
+; SI-NEXT: s_mov_b32 s6, s25
+; SI-NEXT: s_and_b32 s9, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s8, s5, 16
+; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: .LBB89_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s85
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s84
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
; SI-NEXT: v_mul_f32_e64 v1, 1.0, s83
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s56
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s82
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s81
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s40
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s96
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s80
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:340 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:344 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:348 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:352 ; 4-byte Folded Reload
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s74
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s97
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s71
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s6
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s99
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s70
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s98
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s69
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s63
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s18
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s68
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s67
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s42
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s66
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s42
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s20
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s19
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s64
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s55
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s54
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s52
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s50
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s49
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s41
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s48
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s27
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s38
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s75
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s16
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s29
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s64
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s34
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s24
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s22
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s30
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s21
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, vcc_hi
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s55
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s95
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s93
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
; SI-NEXT: v_mul_f32_e64 v21, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s54
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s90
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s89
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s88
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s53
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s77
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s76
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s62
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s30
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s61
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s52
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s58
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s57
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, vcc_hi
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s47
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s35
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s46
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s51
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s45
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s36
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s44
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s14
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s38
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s50
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s10
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s39
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s9
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s37
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s8
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s48
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s49
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
; SI-NEXT: v_readlane_b32 s30, v45, 34
; SI-NEXT: v_lshr_b64 v[31:32], v[31:32], 16
; SI-NEXT: v_readlane_b32 s31, v45, 35
@@ -173291,13 +173468,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s4, v21
; SI-NEXT: v_writelane_b32 v56, s4, 21
; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:332
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:328
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:320
-; SI-NEXT: v_writelane_b32 v56, s4, 22
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:316
; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: v_readfirstlane_b32 s4, v18
; SI-NEXT: v_writelane_b32 v56, s4, 24
@@ -173307,28 +173484,26 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_writelane_b32 v56, s4, 26
; SI-NEXT: v_readfirstlane_b32 s4, v15
; SI-NEXT: v_writelane_b32 v56, s4, 27
-; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v11
-; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v9
-; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: v_writelane_b32 v56, s4, 29
; SI-NEXT: v_readfirstlane_b32 s4, v8
-; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_writelane_b32 v56, s4, 30
; SI-NEXT: v_readfirstlane_b32 s4, v7
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:264
-; SI-NEXT: v_writelane_b32 v56, s4, 32
-; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s87, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
; SI-NEXT: v_readfirstlane_b32 s98, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
; SI-NEXT: v_readfirstlane_b32 s99, v4
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s86, v14
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:312
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:308
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
@@ -173338,34 +173513,35 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
; SI-NEXT: v_readfirstlane_b32 s76, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:296
-; SI-NEXT: v_writelane_b32 v56, s4, 34
+; SI-NEXT: v_writelane_b32 v56, s4, 32
; SI-NEXT: v_readfirstlane_b32 s77, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v22
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s81, v21
+; SI-NEXT: v_readfirstlane_b32 s83, v21
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:228
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:224
; SI-NEXT: v_readfirstlane_b32 s4, v1
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:248
-; SI-NEXT: v_writelane_b32 v56, s4, 35
+; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v0
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:244
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s82, v23
+; SI-NEXT: v_readfirstlane_b32 s90, v23
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:220
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s80, v20
+; SI-NEXT: v_readfirstlane_b32 s82, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:232
-; SI-NEXT: v_readfirstlane_b32 s83, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:232
+; SI-NEXT: v_readfirstlane_b32 s94, v20
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:208
-; SI-NEXT: v_writelane_b32 v56, s4, 36
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:208
+; SI-NEXT: v_writelane_b32 v47, s94, 0
+; SI-NEXT: v_writelane_b32 v56, s4, 34
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s21, v8
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
@@ -173376,56 +173552,56 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s46, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_readfirstlane_b32 s63, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
-; SI-NEXT: v_readfirstlane_b32 s73, v5
-; SI-NEXT: v_readfirstlane_b32 s40, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
+; SI-NEXT: v_readfirstlane_b32 s40, v4
; SI-NEXT: v_readfirstlane_b32 s69, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
; SI-NEXT: v_readfirstlane_b32 s47, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
; SI-NEXT: v_readfirstlane_b32 s55, v9
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s56, v14
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
; SI-NEXT: v_readfirstlane_b32 s16, v12
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
; SI-NEXT: v_readfirstlane_b32 s54, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v21
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s59, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v21
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt expcnt(3)
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: v_readfirstlane_b32 s84, v17
+; SI-NEXT: v_readfirstlane_b32 s74, v17
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s90, v18
+; SI-NEXT: v_readfirstlane_b32 s29, v18
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s23, v25
+; SI-NEXT: v_readfirstlane_b32 s72, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt expcnt(2)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s65, v23
+; SI-NEXT: v_readfirstlane_b32 s64, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
@@ -173435,190 +173611,187 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s37, v22
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:44
+; SI-NEXT: v_readfirstlane_b32 s71, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s17, v16
+; SI-NEXT: v_readfirstlane_b32 s58, v16
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s62, v1
+; SI-NEXT: v_readfirstlane_b32 s60, v1
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:36
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s94, v29
+; SI-NEXT: v_readfirstlane_b32 s65, v29
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s13, v11
-; SI-NEXT: v_readfirstlane_b32 s66, v33
+; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_readfirstlane_b32 s66, v33
; SI-NEXT: v_readfirstlane_b32 s14, v34
; SI-NEXT: v_readfirstlane_b32 s15, v6
; SI-NEXT: v_readfirstlane_b32 s93, v39
-; SI-NEXT: v_readfirstlane_b32 s39, v35
-; SI-NEXT: v_readfirstlane_b32 s49, v4
-; SI-NEXT: v_readfirstlane_b32 s51, v37
-; SI-NEXT: v_writelane_b32 v47, s51, 0
-; SI-NEXT: v_writelane_b32 v47, s54, 1
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s12, v38
-; SI-NEXT: v_writelane_b32 v47, s83, 2
+; SI-NEXT: v_writelane_b32 v47, s12, 1
+; SI-NEXT: v_readfirstlane_b32 s57, v26
; SI-NEXT: v_readfirstlane_b32 s52, v9
-; SI-NEXT: v_writelane_b32 v47, s12, 3
-; SI-NEXT: v_writelane_b32 v47, s52, 4
+; SI-NEXT: v_writelane_b32 v47, s52, 2
; SI-NEXT: v_readfirstlane_b32 s67, v32
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v26
-; SI-NEXT: v_readfirstlane_b32 s61, v27
+; SI-NEXT: v_readfirstlane_b32 s80, v14
+; SI-NEXT: v_readfirstlane_b32 s17, v27
+; SI-NEXT: v_readfirstlane_b32 s61, v12
+; SI-NEXT: v_readfirstlane_b32 s73, v28
; SI-NEXT: v_readfirstlane_b32 s53, v31
-; SI-NEXT: v_writelane_b32 v47, s53, 5
-; SI-NEXT: v_writelane_b32 v47, s55, 6
-; SI-NEXT: v_writelane_b32 v47, s13, 7
-; SI-NEXT: v_writelane_b32 v47, s67, 8
-; SI-NEXT: v_writelane_b32 v47, s88, 9
-; SI-NEXT: v_writelane_b32 v47, s44, 10
-; SI-NEXT: v_writelane_b32 v47, s45, 11
-; SI-NEXT: v_writelane_b32 v47, s46, 12
-; SI-NEXT: v_writelane_b32 v47, s78, 13
-; SI-NEXT: v_writelane_b32 v47, s66, 14
-; SI-NEXT: v_writelane_b32 v47, s47, 15
-; SI-NEXT: v_writelane_b32 v47, s56, 16
-; SI-NEXT: v_writelane_b32 v47, s57, 17
-; SI-NEXT: v_writelane_b32 v47, s14, 18
-; SI-NEXT: v_writelane_b32 v47, s58, 19
-; SI-NEXT: v_writelane_b32 v47, s59, 20
-; SI-NEXT: v_writelane_b32 v47, s17, 21
-; SI-NEXT: v_writelane_b32 v47, s60, 22
-; SI-NEXT: v_writelane_b32 v47, s15, 23
-; SI-NEXT: v_writelane_b32 v47, s61, 24
-; SI-NEXT: v_readfirstlane_b32 s63, v12
-; SI-NEXT: v_writelane_b32 v47, s62, 25
-; SI-NEXT: v_writelane_b32 v47, s63, 26
-; SI-NEXT: v_writelane_b32 v47, s16, 27
+; SI-NEXT: v_writelane_b32 v47, s53, 3
+; SI-NEXT: v_writelane_b32 v47, s55, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s67, 6
+; SI-NEXT: v_writelane_b32 v47, s88, 7
+; SI-NEXT: v_writelane_b32 v47, s44, 8
+; SI-NEXT: v_writelane_b32 v47, s45, 9
+; SI-NEXT: v_writelane_b32 v47, s46, 10
+; SI-NEXT: v_writelane_b32 v47, s78, 11
+; SI-NEXT: v_writelane_b32 v47, s66, 12
+; SI-NEXT: v_writelane_b32 v47, s71, 13
+; SI-NEXT: v_writelane_b32 v47, s14, 14
+; SI-NEXT: v_writelane_b32 v47, s80, 15
+; SI-NEXT: v_writelane_b32 v47, s57, 16
+; SI-NEXT: v_writelane_b32 v47, s58, 17
+; SI-NEXT: v_writelane_b32 v47, s59, 18
+; SI-NEXT: v_writelane_b32 v47, s15, 19
+; SI-NEXT: v_writelane_b32 v47, s17, 20
+; SI-NEXT: v_writelane_b32 v47, s60, 21
; SI-NEXT: v_readfirstlane_b32 s92, v2
-; SI-NEXT: v_writelane_b32 v47, vcc_lo, 28
-; SI-NEXT: v_writelane_b32 v47, s72, 29
-; SI-NEXT: v_writelane_b32 v47, s73, 30
-; SI-NEXT: v_writelane_b32 v47, s23, 31
-; SI-NEXT: v_readfirstlane_b32 s74, v13
+; SI-NEXT: v_writelane_b32 v47, s61, 22
+; SI-NEXT: v_writelane_b32 v47, s16, 23
+; SI-NEXT: v_writelane_b32 v47, s62, 24
+; SI-NEXT: v_writelane_b32 v47, s63, 25
; SI-NEXT: v_readfirstlane_b32 s95, v48
-; SI-NEXT: v_writelane_b32 v47, s18, 32
-; SI-NEXT: v_readfirstlane_b32 s64, v28
-; SI-NEXT: v_writelane_b32 v47, s74, 33
-; SI-NEXT: v_writelane_b32 v47, s64, 34
+; SI-NEXT: v_writelane_b32 v47, vcc_lo, 26
+; SI-NEXT: v_writelane_b32 v47, s72, 27
+; SI-NEXT: v_readfirstlane_b32 s23, v13
+; SI-NEXT: v_writelane_b32 v47, s18, 28
+; SI-NEXT: v_writelane_b32 v47, s23, 29
+; SI-NEXT: v_writelane_b32 v47, s73, 30
; SI-NEXT: v_readfirstlane_b32 s41, v30
+; SI-NEXT: v_writelane_b32 v47, s40, 31
+; SI-NEXT: v_writelane_b32 v47, s41, 32
+; SI-NEXT: v_readfirstlane_b32 s68, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v5
+; SI-NEXT: v_writelane_b32 v47, s74, 33
+; SI-NEXT: v_readfirstlane_b32 s39, v35
+; SI-NEXT: v_writelane_b32 v47, s68, 34
+; SI-NEXT: v_writelane_b32 v47, s64, 35
; SI-NEXT: v_readfirstlane_b32 s19, v41
-; SI-NEXT: v_writelane_b32 v56, s19, 37
+; SI-NEXT: v_writelane_b32 v56, s19, 35
; SI-NEXT: v_readfirstlane_b32 s43, v51
; SI-NEXT: v_readfirstlane_b32 s42, v18
-; SI-NEXT: v_writelane_b32 v56, s42, 38
-; SI-NEXT: v_writelane_b32 v56, s43, 39
+; SI-NEXT: v_writelane_b32 v56, s42, 36
+; SI-NEXT: v_writelane_b32 v56, s43, 37
; SI-NEXT: v_readfirstlane_b32 s6, v42
; SI-NEXT: v_readfirstlane_b32 s7, v23
-; SI-NEXT: v_writelane_b32 v56, s6, 40
+; SI-NEXT: v_writelane_b32 v56, s6, 38
; SI-NEXT: v_readfirstlane_b32 s75, v52
-; SI-NEXT: v_writelane_b32 v56, s7, 41
+; SI-NEXT: v_writelane_b32 v56, s7, 39
; SI-NEXT: s_waitcnt vmcnt(13)
; SI-NEXT: v_readfirstlane_b32 s79, v0
-; SI-NEXT: v_writelane_b32 v56, s75, 42
+; SI-NEXT: v_writelane_b32 v56, s75, 40
; SI-NEXT: s_waitcnt vmcnt(11)
; SI-NEXT: v_readfirstlane_b32 s89, v22
-; SI-NEXT: v_writelane_b32 v56, s79, 43
+; SI-NEXT: v_writelane_b32 v56, s79, 41
; SI-NEXT: v_readfirstlane_b32 s91, v53
-; SI-NEXT: v_writelane_b32 v56, s89, 44
-; SI-NEXT: v_writelane_b32 v56, s91, 45
-; SI-NEXT: v_writelane_b32 v56, s92, 46
+; SI-NEXT: v_writelane_b32 v56, s89, 42
+; SI-NEXT: v_writelane_b32 v56, s91, 43
+; SI-NEXT: v_writelane_b32 v56, s92, 44
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_readfirstlane_b32 s8, v54
-; SI-NEXT: v_writelane_b32 v56, s93, 47
+; SI-NEXT: v_writelane_b32 v56, s93, 45
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s9, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 48
-; SI-NEXT: v_writelane_b32 v56, s9, 49
+; SI-NEXT: v_readfirstlane_b32 s9, v19
+; SI-NEXT: v_writelane_b32 v56, s8, 46
+; SI-NEXT: v_writelane_b32 v56, s9, 47
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_writelane_b32 v56, s95, 50
-; SI-NEXT: v_writelane_b32 v56, s31, 51
+; SI-NEXT: v_writelane_b32 v56, s95, 48
+; SI-NEXT: v_writelane_b32 v56, s31, 49
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_writelane_b32 v56, s80, 52
-; SI-NEXT: v_writelane_b32 v47, s40, 35
+; SI-NEXT: v_writelane_b32 v56, s82, 50
; SI-NEXT: v_readfirstlane_b32 s36, v49
-; SI-NEXT: v_writelane_b32 v56, s34, 53
-; SI-NEXT: v_writelane_b32 v47, s41, 36
-; SI-NEXT: v_readfirstlane_b32 s68, v10
-; SI-NEXT: v_writelane_b32 v56, s36, 54
-; SI-NEXT: v_writelane_b32 v47, s84, 37
+; SI-NEXT: v_writelane_b32 v56, s34, 51
+; SI-NEXT: v_writelane_b32 v56, s36, 52
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_readfirstlane_b32 s10, v50
-; SI-NEXT: v_writelane_b32 v56, s81, 55
-; SI-NEXT: v_writelane_b32 v47, s68, 38
-; SI-NEXT: v_readfirstlane_b32 s38, v5
-; SI-NEXT: v_writelane_b32 v56, s10, 56
-; SI-NEXT: v_writelane_b32 v47, s65, 39
-; SI-NEXT: v_writelane_b32 v56, s38, 57
-; SI-NEXT: v_writelane_b32 v47, s90, 40
+; SI-NEXT: v_writelane_b32 v56, s83, 53
+; SI-NEXT: v_writelane_b32 v56, s10, 54
+; SI-NEXT: v_writelane_b32 v56, s38, 55
; SI-NEXT: v_readfirstlane_b32 s11, v17
-; SI-NEXT: v_writelane_b32 v56, s39, 58
-; SI-NEXT: v_writelane_b32 v47, s69, 41
+; SI-NEXT: v_writelane_b32 v56, s39, 56
; SI-NEXT: v_readfirstlane_b32 s48, v36
-; SI-NEXT: v_writelane_b32 v56, s11, 59
-; SI-NEXT: v_writelane_b32 v47, s94, 42
-; SI-NEXT: v_writelane_b32 v56, s48, 60
-; SI-NEXT: v_writelane_b32 v47, s70, 43
-; SI-NEXT: v_readfirstlane_b32 s35, v19
-; SI-NEXT: v_writelane_b32 v56, s49, 61
-; SI-NEXT: v_writelane_b32 v47, s37, 44
-; SI-NEXT: v_readfirstlane_b32 s71, v15
+; SI-NEXT: v_writelane_b32 v56, s11, 57
+; SI-NEXT: v_writelane_b32 v47, s29, 36
+; SI-NEXT: v_readfirstlane_b32 s49, v4
+; SI-NEXT: v_writelane_b32 v56, s48, 58
+; SI-NEXT: v_writelane_b32 v47, s69, 37
+; SI-NEXT: v_writelane_b32 v56, s49, 59
+; SI-NEXT: v_writelane_b32 v47, s65, 38
; SI-NEXT: v_readfirstlane_b32 s50, v3
+; SI-NEXT: v_writelane_b32 v56, s90, 60
+; SI-NEXT: v_writelane_b32 v47, s70, 39
+; SI-NEXT: v_readfirstlane_b32 s35, v20
+; SI-NEXT: v_readfirstlane_b32 s51, v37
+; SI-NEXT: v_writelane_b32 v56, s50, 61
+; SI-NEXT: v_writelane_b32 v47, s37, 40
+; SI-NEXT: v_readfirstlane_b32 s81, v15
; SI-NEXT: v_readfirstlane_b32 s22, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s85, v24
+; SI-NEXT: v_readfirstlane_b32 s30, v24
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s25, v40
; SI-NEXT: v_readfirstlane_b32 s28, v21
-; SI-NEXT: v_readfirstlane_b32 s87, v25
+; SI-NEXT: v_readfirstlane_b32 s85, v25
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v43
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_readfirstlane_b32 s26, v29
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s27, v44
-; SI-NEXT: v_writelane_b32 v56, s82, 62
-; SI-NEXT: v_writelane_b32 v47, s35, 45
-; SI-NEXT: v_writelane_b32 v56, s50, 63
-; SI-NEXT: v_writelane_b32 v47, s71, 46
+; SI-NEXT: v_writelane_b32 v56, s51, 62
+; SI-NEXT: v_writelane_b32 v47, s35, 41
+; SI-NEXT: v_writelane_b32 v56, s54, 63
+; SI-NEXT: v_writelane_b32 v47, s81, 42
; SI-NEXT: s_cbranch_scc0 .LBB93_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: v_readlane_b32 s4, v56, 11
; SI-NEXT: v_readlane_b32 s5, v56, 10
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: v_writelane_b32 v47, s21, 47
+; SI-NEXT: v_writelane_b32 v47, s21, 57
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 4
-; SI-NEXT: v_writelane_b32 v47, s4, 48
+; SI-NEXT: v_writelane_b32 v47, s4, 43
; SI-NEXT: s_and_b32 s4, s24, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 34
-; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_readlane_b32 s5, v56, 32
+; SI-NEXT: v_writelane_b32 v47, s4, 45
; SI-NEXT: s_and_b32 s4, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 29
-; SI-NEXT: v_writelane_b32 v47, s4, 50
+; SI-NEXT: v_readlane_b32 s5, v56, 28
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s4, 49
; SI-NEXT: v_readlane_b32 s4, v56, 24
; SI-NEXT: s_and_b32 s5, s4, 0xff
; SI-NEXT: v_readlane_b32 s4, v56, 23
; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 52
+; SI-NEXT: v_writelane_b32 v47, s4, 51
; SI-NEXT: v_readlane_b32 s4, v56, 16
; SI-NEXT: v_readlane_b32 s5, v56, 15
; SI-NEXT: s_and_b32 s4, s4, 0xff
@@ -173628,41 +173801,45 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s4, s7, 0xff
; SI-NEXT: s_lshl_b32 s5, s6, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 54
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_and_b32 s4, s9, 0xff
; SI-NEXT: s_lshl_b32 s5, s8, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s4, 58
; SI-NEXT: s_and_b32 s4, s11, 0xff
; SI-NEXT: s_lshl_b32 s5, s10, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 56
+; SI-NEXT: v_writelane_b32 v47, s4, 59
; SI-NEXT: s_and_b32 s4, s13, 0xff
; SI-NEXT: s_lshl_b32 s5, s12, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 57
+; SI-NEXT: v_writelane_b32 v47, s4, 60
; SI-NEXT: s_and_b32 s4, s15, 0xff
; SI-NEXT: s_lshl_b32 s5, s14, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 58
+; SI-NEXT: v_writelane_b32 v47, s4, 61
; SI-NEXT: s_and_b32 s4, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s41, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: v_writelane_b32 v47, s4, 62
+; SI-NEXT: s_and_b32 s4, s81, 0xff
; SI-NEXT: s_lshl_b32 s5, s35, 8
-; SI-NEXT: s_or_b32 s71, s4, s5
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v47, s4, 63
; SI-NEXT: s_and_b32 s4, s37, 0xff
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: s_or_b32 s70, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_lshl_b32 s5, s40, 8
-; SI-NEXT: s_or_b32 s69, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 0
+; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 1
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s16, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 7
-; SI-NEXT: v_writelane_b32 v47, s4, 60
+; SI-NEXT: v_writelane_b32 v46, s4, 2
; SI-NEXT: s_and_b32 s4, s20, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s5, s4, s5
@@ -173671,57 +173848,52 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s6, v56, 5
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s7, s6, s4
+; SI-NEXT: s_or_b32 s81, s6, s4
; SI-NEXT: v_readlane_b32 s4, v56, 9
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: v_readlane_b32 s6, v56, 8
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s7, 61
; SI-NEXT: s_or_b32 s4, s6, s4
-; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: v_readlane_b32 s6, v56, 1
; SI-NEXT: v_readlane_b32 s7, v56, 0
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_or_b32 s7, s6, s7
-; SI-NEXT: v_readlane_b32 s6, v56, 36
+; SI-NEXT: v_readlane_b32 s6, v56, 34
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 35
+; SI-NEXT: v_readlane_b32 s8, v56, 33
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_or_b32 s9, s8, s6
; SI-NEXT: v_readlane_b32 s6, v56, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: v_readlane_b32 s8, v56, 2
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s9, 3
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s9, 62
-; SI-NEXT: s_or_b32 s6, s8, s6
; SI-NEXT: s_or_b32 s7, s7, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 33
-; SI-NEXT: v_readlane_b32 s9, v56, 32
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 31
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_and_b32 s8, s87, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
; SI-NEXT: s_or_b32 s9, s8, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 31
+; SI-NEXT: v_readlane_b32 s8, v56, 30
; SI-NEXT: s_and_b32 s8, s8, 0xff
-; SI-NEXT: v_readlane_b32 s10, v56, 30
+; SI-NEXT: v_readlane_b32 s10, v56, 29
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s10, 24
; SI-NEXT: s_or_b32 s11, s10, s8
; SI-NEXT: s_and_b32 s8, s99, 0xff
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s11, 4
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s98, 24
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s11, 63
-; SI-NEXT: s_or_b32 s8, s10, s8
; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: v_readlane_b32 s10, v56, 28
; SI-NEXT: v_readlane_b32 s11, v56, 27
-; SI-NEXT: s_and_b32 s10, s10, 0xff
+; SI-NEXT: s_or_b32 s8, s10, s8
+; SI-NEXT: s_and_b32 s10, s86, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 8
; SI-NEXT: s_or_b32 s11, s10, s11
; SI-NEXT: v_readlane_b32 s10, v56, 26
@@ -173729,16 +173901,12 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s12, v56, 25
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s13, s12, s10
+; SI-NEXT: s_or_b32 s84, s12, s10
; SI-NEXT: s_and_b32 s10, s97, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s96, 24
-; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: v_readlane_b32 s12, v56, 20
-; SI-NEXT: v_writelane_b32 v46, s13, 0
-; SI-NEXT: s_or_b32 s11, s11, s13
; SI-NEXT: s_and_b32 s13, s12, 0xff
; SI-NEXT: v_readlane_b32 s12, v56, 19
; SI-NEXT: s_lshl_b32 s12, s12, 8
@@ -173748,7 +173916,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s14, s12, 16
; SI-NEXT: v_readlane_b32 s12, v56, 17
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s86, s12, s14
+; SI-NEXT: s_or_b32 s69, s12, s14
; SI-NEXT: v_readlane_b32 s12, v56, 22
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s14, s12, 16
@@ -173762,7 +173930,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s14, s26, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_lshl_b32 s16, vcc_hi, 24
-; SI-NEXT: s_or_b32 s29, s16, s14
+; SI-NEXT: s_or_b32 s70, s16, s14
; SI-NEXT: v_readlane_b32 s14, v56, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s16, s14, 16
@@ -173772,16 +173940,16 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s16, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s25, 8
; SI-NEXT: s_or_b32 s18, s16, s18
-; SI-NEXT: s_and_b32 s16, s85, 0xff
-; SI-NEXT: v_writelane_b32 v46, s20, 1
+; SI-NEXT: s_and_b32 s16, s30, 0xff
+; SI-NEXT: v_writelane_b32 v46, s20, 5
; SI-NEXT: s_lshl_b32 s16, s16, 16
; SI-NEXT: s_lshl_b32 s20, s22, 24
-; SI-NEXT: v_writelane_b32 v46, vcc_hi, 2
-; SI-NEXT: v_writelane_b32 v46, s28, 3
+; SI-NEXT: v_writelane_b32 v46, vcc_hi, 6
+; SI-NEXT: v_writelane_b32 v46, s28, 7
; SI-NEXT: s_or_b32 s28, s20, s16
-; SI-NEXT: s_and_b32 s20, s87, 0xff
+; SI-NEXT: s_and_b32 s20, s85, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: v_writelane_b32 v46, s24, 4
+; SI-NEXT: v_writelane_b32 v46, s24, 8
; SI-NEXT: s_lshl_b32 s24, s19, 24
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s40, s24, s20
@@ -173795,7 +173963,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s19, s24, s18
; SI-NEXT: s_and_b32 s24, s89, 0xff
; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: v_writelane_b32 v46, s25, 5
+; SI-NEXT: v_writelane_b32 v46, s25, 9
; SI-NEXT: s_lshl_b32 s25, s91, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s42, s25, s24
@@ -173820,7 +173988,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s25, s38, 0xff
; SI-NEXT: s_lshl_b32 s25, s25, 16
; SI-NEXT: s_lshl_b32 s26, s39, 24
-; SI-NEXT: v_writelane_b32 v46, s27, 6
+; SI-NEXT: v_writelane_b32 v46, s27, 10
; SI-NEXT: s_or_b32 s27, s26, s25
; SI-NEXT: s_and_b32 s26, s50, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
@@ -173841,44 +174009,44 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s38, s75, s26
; SI-NEXT: s_or_b32 s39, s20, s18
-; SI-NEXT: s_and_b32 s20, s63, 0xff
-; SI-NEXT: s_lshl_b32 s26, s61, 8
+; SI-NEXT: s_and_b32 s20, s61, 0xff
+; SI-NEXT: s_lshl_b32 s26, s17, 8
; SI-NEXT: s_or_b32 s20, s20, s26
-; SI-NEXT: s_and_b32 s26, s58, 0xff
+; SI-NEXT: s_and_b32 s26, s80, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_lshl_b32 s75, s59, 24
+; SI-NEXT: s_lshl_b32 s75, s57, 24
; SI-NEXT: s_or_b32 s26, s75, s26
-; SI-NEXT: s_and_b32 s75, s74, 0xff
+; SI-NEXT: s_and_b32 s75, s23, 0xff
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s78, s64, 24
+; SI-NEXT: s_lshl_b32 s78, s73, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s48, s78, s75
; SI-NEXT: s_or_b32 s49, s20, s26
-; SI-NEXT: s_and_b32 s20, s23, 0xff
-; SI-NEXT: s_lshl_b32 s75, vcc_lo, 8
+; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_lshl_b32 s75, s62, 8
; SI-NEXT: s_or_b32 s75, s20, s75
-; SI-NEXT: s_and_b32 s20, s60, 0xff
+; SI-NEXT: s_and_b32 s20, s59, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_lshl_b32 s78, s56, 24
+; SI-NEXT: s_lshl_b32 s78, s71, 24
; SI-NEXT: s_or_b32 s20, s78, s20
-; SI-NEXT: s_and_b32 s78, s94, 0xff
+; SI-NEXT: s_and_b32 s78, s65, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s79, s65, 24
+; SI-NEXT: s_lshl_b32 s79, s64, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_or_b32 s50, s79, s78
; SI-NEXT: s_or_b32 s51, s75, s20
-; SI-NEXT: s_and_b32 s75, s62, 0xff
-; SI-NEXT: s_lshl_b32 s78, s17, 8
+; SI-NEXT: s_and_b32 s75, s60, 0xff
+; SI-NEXT: s_lshl_b32 s78, s58, 8
; SI-NEXT: s_or_b32 s75, s75, s78
; SI-NEXT: s_and_b32 s78, s45, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
; SI-NEXT: s_lshl_b32 s79, s44, 24
; SI-NEXT: s_or_b32 s78, s79, s78
-; SI-NEXT: s_and_b32 s79, s90, 0xff
+; SI-NEXT: s_and_b32 s79, s29, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s84, 24
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 47
+; SI-NEXT: v_readlane_b32 s16, v47, 57
; SI-NEXT: s_or_b32 s52, s88, s79
; SI-NEXT: s_or_b32 s53, s75, s78
; SI-NEXT: s_and_b32 s75, s46, 0xff
@@ -173888,167 +174056,188 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_lshl_b32 s88, s54, 24
; SI-NEXT: s_or_b32 s88, s88, s79
-; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_and_b32 s79, vcc_lo, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s72, 24
+; SI-NEXT: s_lshl_b32 s89, s63, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 48
+; SI-NEXT: v_readlane_b32 s16, v47, 43
; SI-NEXT: s_or_b32 s54, s89, s79
; SI-NEXT: s_or_b32 s55, s75, s88
-; SI-NEXT: s_and_b32 s75, s83, 0xff
-; SI-NEXT: s_lshl_b32 s79, s82, 8
+; SI-NEXT: s_and_b32 s75, s94, 0xff
+; SI-NEXT: s_lshl_b32 s79, s90, 8
; SI-NEXT: s_and_b32 s44, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 49
+; SI-NEXT: v_readlane_b32 s16, v47, 45
; SI-NEXT: s_or_b32 s75, s75, s79
-; SI-NEXT: s_and_b32 s79, s81, 0xff
+; SI-NEXT: s_and_b32 s79, s83, 0xff
; SI-NEXT: s_and_b32 s45, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 50
+; SI-NEXT: v_readlane_b32 s16, v47, 47
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s80, 24
+; SI-NEXT: s_lshl_b32 s89, s82, 24
; SI-NEXT: s_and_b32 s46, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: v_readlane_b32 s16, v47, 49
; SI-NEXT: s_or_b32 s90, s89, s79
+; SI-NEXT: s_mov_b32 s71, s47
; SI-NEXT: s_lshl_b32 s89, s47, 24
; SI-NEXT: s_and_b32 s47, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 52
-; SI-NEXT: s_and_b32 s79, s57, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: s_and_b32 s79, s56, 0xff
; SI-NEXT: s_and_b32 s56, s16, 0xffff
; SI-NEXT: v_readlane_b32 s16, v47, 53
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_and_b32 s57, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 54
+; SI-NEXT: v_readlane_b32 s16, v47, 55
; SI-NEXT: s_or_b32 s64, s89, s79
; SI-NEXT: s_and_b32 s79, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 55
+; SI-NEXT: v_readlane_b32 s16, v47, 58
; SI-NEXT: s_and_b32 s89, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 56
+; SI-NEXT: v_readlane_b32 s16, v47, 59
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_and_b32 s17, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 57
+; SI-NEXT: v_readlane_b32 s16, v47, 60
+; SI-NEXT: s_or_b32 s5, s5, s81
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_and_b32 s91, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 58
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: s_or_b32 s65, s75, s90
; SI-NEXT: s_and_b32 s92, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 59
-; SI-NEXT: s_or_b32 s13, s13, s86
+; SI-NEXT: v_readlane_b32 s16, v47, 62
+; SI-NEXT: s_or_b32 s74, s44, s4
+; SI-NEXT: s_mov_b32 s75, s5
+; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[4:5], s[6:7], 16
; SI-NEXT: s_and_b32 s93, s16, 0xffff
-; SI-NEXT: s_mov_b32 s16, s85
-; SI-NEXT: s_lshr_b64 s[84:85], s[40:41], 16
-; SI-NEXT: s_and_b32 s94, s71, 0xffff
-; SI-NEXT: s_and_b32 s95, s70, 0xffff
-; SI-NEXT: s_or_b32 s58, s56, s12
-; SI-NEXT: s_mov_b32 s59, s13
-; SI-NEXT: s_lshr_b64 s[70:71], s[12:13], 16
-; SI-NEXT: s_or_b32 s12, s93, s48
-; SI-NEXT: s_mov_b32 s13, s49
-; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s85, s16
-; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: v_readlane_b32 s16, v47, 63
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
+; SI-NEXT: s_or_b32 s11, s11, s84
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 16
+; SI-NEXT: s_or_b32 s13, s13, s69
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_mov_b32 s25, s22
-; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v47, 60
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: s_lshr_b64 s[4:5], s[12:13], 16
+; SI-NEXT: s_or_b32 s15, s15, s70
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: s_lshr_b64 s[4:5], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: s_lshr_b64 s[4:5], s[40:41], 16
+; SI-NEXT: s_mov_b32 s23, s21
+; SI-NEXT: v_readlane_b32 s21, v46, 1
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: s_and_b32 s94, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_and_b32 vcc_lo, s21, 0xffff
+; SI-NEXT: v_readlane_b32 s21, v46, 2
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: s_lshr_b64 s[4:5], s[42:43], 16
+; SI-NEXT: s_and_b32 s95, s16, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s21, 0xffff
; SI-NEXT: s_or_b32 s60, s47, s10
; SI-NEXT: s_mov_b32 s61, s11
-; SI-NEXT: s_lshr_b64 vcc, s[10:11], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_or_b32 s10, s94, s50
; SI-NEXT: s_mov_b32 s11, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 16
-; SI-NEXT: s_lshr_b32 s49, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 62
-; SI-NEXT: s_or_b32 s15, s15, s29
-; SI-NEXT: s_mov_b32 s23, s21
-; SI-NEXT: s_or_b32 s65, s75, s90
-; SI-NEXT: s_and_b32 s21, s69, 0xffff
-; SI-NEXT: s_and_b32 s22, s22, 0xffff
-; SI-NEXT: s_or_b32 s62, s46, s8
-; SI-NEXT: s_mov_b32 s63, s9
-; SI-NEXT: s_lshr_b64 s[66:67], s[8:9], 16
-; SI-NEXT: s_or_b32 s8, s95, s52
-; SI-NEXT: s_mov_b32 s9, s53
-; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_lshr_b32 s51, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 63
-; SI-NEXT: s_or_b32 s74, s44, s4
-; SI-NEXT: s_mov_b32 s75, s5
-; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: v_readlane_b32 s16, v46, 3
; SI-NEXT: s_or_b32 s72, s45, s6
; SI-NEXT: s_mov_b32 s73, s7
-; SI-NEXT: s_lshr_b64 s[30:31], s[6:7], 16
+; SI-NEXT: s_or_b32 s62, s46, s8
+; SI-NEXT: s_mov_b32 s63, s9
+; SI-NEXT: s_or_b32 s58, s56, s12
+; SI-NEXT: s_mov_b32 s59, s13
; SI-NEXT: s_or_b32 s56, s57, s14
; SI-NEXT: s_mov_b32 s57, s15
-; SI-NEXT: s_lshr_b64 s[80:81], s[14:15], 16
-; SI-NEXT: s_or_b32 s44, s89, s42
-; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_lshr_b64 s[68:69], s[42:43], 16
-; SI-NEXT: s_or_b32 s42, s17, s34
-; SI-NEXT: s_mov_b32 s43, s35
-; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
+; SI-NEXT: v_writelane_b32 v47, s5, 56
; SI-NEXT: s_or_b32 s14, s92, s38
; SI-NEXT: s_mov_b32 s15, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[38:39], 16
-; SI-NEXT: s_or_b32 s6, s21, s54
+; SI-NEXT: s_or_b32 s12, s93, s48
+; SI-NEXT: s_mov_b32 s13, s49
+; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
+; SI-NEXT: s_or_b32 s8, s95, s52
+; SI-NEXT: s_mov_b32 s9, s53
+; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
+; SI-NEXT: s_or_b32 s6, vcc_lo, s54
; SI-NEXT: s_mov_b32 s7, s55
; SI-NEXT: s_lshr_b64 s[54:55], s[54:55], 16
-; SI-NEXT: s_or_b32 s4, s22, s64
+; SI-NEXT: s_or_b32 s4, vcc_hi, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_lshr_b32 s53, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_lshr_b32 s51, s16, 16
+; SI-NEXT: v_readlane_b32 s16, v46, 4
; SI-NEXT: s_or_b32 s46, s79, s40
; SI-NEXT: s_mov_b32 s47, s41
+; SI-NEXT: s_or_b32 s44, s89, s42
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_or_b32 s42, s17, s34
+; SI-NEXT: s_mov_b32 s43, s35
+; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
; SI-NEXT: s_or_b32 s40, s91, s36
; SI-NEXT: s_mov_b32 s41, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 16
-; SI-NEXT: s_lshr_b32 s55, s16, 16
-; SI-NEXT: s_lshr_b32 s65, s86, 16
-; SI-NEXT: s_lshr_b32 s35, s29, 16
-; SI-NEXT: s_mov_b32 s94, s66
+; SI-NEXT: s_lshr_b32 s49, s81, 16
+; SI-NEXT: s_lshr_b32 s53, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s84, 16
+; SI-NEXT: s_lshr_b32 s65, s69, 16
+; SI-NEXT: s_lshr_b32 s69, s70, 16
; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_readlane_b32 s28, v46, 3
-; SI-NEXT: v_readlane_b32 vcc_hi, v46, 2
+; SI-NEXT: v_readlane_b32 s28, v46, 7
+; SI-NEXT: v_readlane_b32 vcc_hi, v46, 6
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s91, s24, 16
-; SI-NEXT: v_readlane_b32 s24, v46, 4
+; SI-NEXT: v_readlane_b32 s24, v46, 8
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: v_readlane_b32 s27, v46, 6
+; SI-NEXT: v_readlane_b32 s27, v46, 10
; SI-NEXT: s_lshr_b32 s95, s18, 16
; SI-NEXT: s_lshr_b32 s31, s26, 16
; SI-NEXT: s_lshr_b32 s17, s20, 16
-; SI-NEXT: v_readlane_b32 s20, v46, 1
+; SI-NEXT: v_readlane_b32 s20, v46, 5
; SI-NEXT: s_lshr_b32 s92, s78, 16
-; SI-NEXT: s_lshr_b32 s78, s88, 16
-; SI-NEXT: v_readlane_b32 s21, v47, 47
+; SI-NEXT: s_lshr_b32 s68, s88, 16
+; SI-NEXT: v_readlane_b32 s21, v47, 57
; SI-NEXT: s_mov_b32 s26, s23
-; SI-NEXT: s_mov_b32 s22, s25
-; SI-NEXT: v_readlane_b32 s25, v46, 5
+; SI-NEXT: v_readlane_b32 s25, v46, 9
; SI-NEXT: s_lshr_b32 s39, s90, 16
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB93_3
; SI-NEXT: .LBB93_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s71, s47
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v47, s4, 47
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr49
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr70
; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
@@ -174070,41 +174259,62 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s5, 56
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB93_3: ; %Flow
; SI-NEXT: s_and_b64 s[18:19], s[66:67], exec
; SI-NEXT: s_cselect_b32 s16, 1, 0
; SI-NEXT: s_cmp_lg_u32 s16, 1
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: v_readlane_b32 s18, v47, 7
+; SI-NEXT: v_readlane_b32 s19, v47, 8
+; SI-NEXT: v_readlane_b32 s88, v47, 9
+; SI-NEXT: v_readlane_b32 s90, v47, 10
+; SI-NEXT: v_readlane_b32 s78, v47, 11
+; SI-NEXT: v_readlane_b32 s94, v47, 16
+; SI-NEXT: v_readlane_b32 vcc_lo, v47, 27
+; SI-NEXT: v_readlane_b32 s23, v47, 30
+; SI-NEXT: v_readlane_b32 s29, v47, 36
+; SI-NEXT: v_readlane_b32 s37, v47, 40
+; SI-NEXT: v_readlane_b32 s35, v47, 41
+; SI-NEXT: v_readlane_b32 s66, v47, 42
; SI-NEXT: s_cbranch_scc1 .LBB93_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: v_readlane_b32 s4, v47, 28
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: v_readlane_b32 s5, v47, 27
-; SI-NEXT: v_readlane_b32 s6, v47, 17
+; SI-NEXT: v_readlane_b32 s5, v47, 23
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_add_i32 s6, s80, 3
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 15
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 24
+; SI-NEXT: s_lshl_b32 s5, s71, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 2
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: v_readlane_b32 s6, v56, 62
-; SI-NEXT: v_readlane_b32 s7, v56, 55
+; SI-NEXT: s_add_i32 s5, s16, 3
+; SI-NEXT: v_readlane_b32 s6, v56, 60
+; SI-NEXT: v_readlane_b32 s7, v56, 53
; SI-NEXT: s_and_b32 s5, s5, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 8
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 52
+; SI-NEXT: v_readlane_b32 s6, v56, 50
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
@@ -174112,15 +174322,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v47, 41
+; SI-NEXT: v_readlane_b32 s6, v47, 37
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 35
-; SI-NEXT: v_readlane_b32 s8, v47, 30
+; SI-NEXT: v_readlane_b32 s7, v47, 31
+; SI-NEXT: v_readlane_b32 s8, v47, 26
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s8, s8, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 29
+; SI-NEXT: v_readlane_b32 s7, v47, 25
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_lshl_b32 s8, s8, 16
@@ -174128,14 +174338,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s7, s7, s8
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 6
+; SI-NEXT: s_add_i32 s7, s90, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 4
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s21, 8
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 1
+; SI-NEXT: v_readlane_b32 s8, v56, 63
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
@@ -174143,15 +174352,13 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 43
-; SI-NEXT: v_readlane_b32 s10, v47, 40
+; SI-NEXT: s_add_i32 s8, s37, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 39
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
-; SI-NEXT: s_add_i32 s10, s10, 3
+; SI-NEXT: s_add_i32 s10, s29, 3
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 37
+; SI-NEXT: v_readlane_b32 s9, v47, 33
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 24
; SI-NEXT: s_lshl_b32 s10, s10, 16
@@ -174159,31 +174366,27 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_and_b32 s8, s8, 0xffff
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 25
+; SI-NEXT: v_readlane_b32 s9, v47, 21
; SI-NEXT: s_add_i32 s9, s9, 3
-; SI-NEXT: v_readlane_b32 s10, v47, 21
-; SI-NEXT: v_readlane_b32 s11, v47, 11
+; SI-NEXT: v_readlane_b32 s10, v47, 17
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 8
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s11, s88, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 10
; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_lshl_b32 s10, s19, 24
; SI-NEXT: s_lshl_b32 s11, s11, 16
; SI-NEXT: s_addk_i32 s9, 0x300
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 46
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: v_readlane_b32 s11, v47, 45
-; SI-NEXT: v_readlane_b32 s12, v47, 42
+; SI-NEXT: s_add_i32 s10, s66, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 38
; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 8
+; SI-NEXT: s_lshl_b32 s11, s35, 8
; SI-NEXT: s_add_i32 s12, s12, 3
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 39
+; SI-NEXT: v_readlane_b32 s11, v47, 35
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s12, s12, 16
@@ -174191,15 +174394,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s11, s11, s12
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 31
-; SI-NEXT: s_add_i32 s11, s11, 3
-; SI-NEXT: v_readlane_b32 s12, v47, 28
-; SI-NEXT: v_readlane_b32 s13, v47, 22
+; SI-NEXT: s_add_i32 s11, vcc_lo, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 24
+; SI-NEXT: v_readlane_b32 s13, v47, 18
; SI-NEXT: s_and_b32 s11, s11, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_add_i32 s13, s13, 3
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 16
+; SI-NEXT: v_readlane_b32 s12, v47, 13
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 24
; SI-NEXT: s_lshl_b32 s13, s13, 16
@@ -174207,47 +174409,44 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 38
+; SI-NEXT: v_readlane_b32 s12, v47, 34
; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: v_readlane_b32 s13, v47, 36
-; SI-NEXT: v_readlane_b32 s14, v47, 33
+; SI-NEXT: v_readlane_b32 s13, v47, 32
+; SI-NEXT: v_readlane_b32 s14, v47, 29
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s13, s13, 8
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 34
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_lshl_b32 s13, s23, 24
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_addk_i32 s12, 0x300
; SI-NEXT: s_or_b32 s13, s13, s14
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 26
+; SI-NEXT: v_readlane_b32 s13, v47, 22
; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s14, v47, 24
-; SI-NEXT: v_readlane_b32 s15, v47, 19
+; SI-NEXT: v_readlane_b32 s14, v47, 20
+; SI-NEXT: v_readlane_b32 s15, v47, 15
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 8
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 20
; SI-NEXT: s_and_b32 s15, s15, 0xff
-; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_lshl_b32 s14, s94, 24
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_addk_i32 s13, 0x300
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 23
+; SI-NEXT: v_readlane_b32 s14, v47, 19
; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 18
-; SI-NEXT: v_readlane_b32 s16, v47, 13
+; SI-NEXT: v_readlane_b32 s15, v47, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s78, 3
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 14
+; SI-NEXT: v_readlane_b32 s15, v47, 12
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 24
; SI-NEXT: s_lshl_b32 s16, s16, 16
@@ -174255,15 +174454,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s15, s15, s16
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 9
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 8
-; SI-NEXT: v_readlane_b32 s17, v47, 4
+; SI-NEXT: s_add_i32 s15, s18, 3
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: v_readlane_b32 s17, v47, 2
; SI-NEXT: s_and_b32 s15, s15, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 8
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: v_readlane_b32 s16, v47, 3
; SI-NEXT: s_and_b32 s17, s17, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
@@ -174271,15 +174469,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: v_readlane_b32 s16, v47, 5
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v47, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 63
+; SI-NEXT: v_readlane_b32 s17, v47, 1
+; SI-NEXT: v_readlane_b32 s18, v56, 61
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v47, 0
+; SI-NEXT: v_readlane_b32 s17, v56, 62
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174288,15 +174486,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s40, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 60
-; SI-NEXT: v_readlane_b32 s18, v56, 57
+; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s18, v56, 55
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s17, v56, 56
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174305,15 +174503,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s41, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 59
+; SI-NEXT: v_readlane_b32 s16, v56, 57
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 56
-; SI-NEXT: v_readlane_b32 s18, v56, 53
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s18, v56, 51
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s17, v56, 52
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174322,15 +174520,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 51
+; SI-NEXT: v_readlane_b32 s16, v56, 49
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 50
-; SI-NEXT: v_readlane_b32 s18, v56, 46
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: v_readlane_b32 s18, v56, 44
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: v_readlane_b32 s17, v56, 45
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174339,15 +174537,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s43, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 49
+; SI-NEXT: v_readlane_b32 s16, v56, 47
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 48
-; SI-NEXT: v_readlane_b32 s18, v56, 44
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: v_readlane_b32 s18, v56, 42
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: v_readlane_b32 s17, v56, 43
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174356,15 +174554,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s44, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 43
+; SI-NEXT: v_readlane_b32 s16, v56, 41
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 42
-; SI-NEXT: v_readlane_b32 s18, v56, 38
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s18, v56, 36
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: v_readlane_b32 s17, v56, 37
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174373,14 +174571,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s45, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 41
+; SI-NEXT: v_readlane_b32 s16, v56, 39
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s17, v56, 38
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s87, 3
+; SI-NEXT: s_add_i32 s18, s85, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: v_readlane_b32 s17, v56, 35
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174392,7 +174590,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_add_i32 s16, s28, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s25, 8
-; SI-NEXT: s_add_i32 s18, s85, 3
+; SI-NEXT: s_add_i32 s18, s30, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
@@ -174468,7 +174666,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_add_i32 s59, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s76, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 29
+; SI-NEXT: v_readlane_b32 s17, v56, 28
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s97, 3
@@ -174481,8 +174679,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s60, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 28
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s86, 3
; SI-NEXT: v_readlane_b32 s17, v56, 27
; SI-NEXT: v_readlane_b32 s18, v56, 26
; SI-NEXT: s_and_b32 s16, s16, 0xff
@@ -174499,7 +174696,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s61, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s77, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: v_readlane_b32 s17, v56, 32
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s99, 3
@@ -174512,15 +174709,14 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s62, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 33
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 32
-; SI-NEXT: v_readlane_b32 s18, v56, 31
+; SI-NEXT: s_add_i32 s16, s87, 3
+; SI-NEXT: v_readlane_b32 s17, v56, 31
+; SI-NEXT: v_readlane_b32 s18, v56, 30
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 30
+; SI-NEXT: v_readlane_b32 s17, v56, 29
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174548,12 +174744,12 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: v_readlane_b32 s16, v56, 1
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s17, v56, 0
-; SI-NEXT: v_readlane_b32 s18, v56, 36
+; SI-NEXT: v_readlane_b32 s18, v56, 34
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: v_readlane_b32 s17, v56, 33
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -174593,6 +174789,25 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s18, s18, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s75, s16, 0x3000000
+; SI-NEXT: s_lshr_b64 s[16:17], s[72:73], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 43
+; SI-NEXT: v_writelane_b32 v47, s17, 44
+; SI-NEXT: s_lshr_b64 s[16:17], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 45
+; SI-NEXT: v_writelane_b32 v47, s17, 46
+; SI-NEXT: s_lshr_b64 s[16:17], s[60:61], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 47
+; SI-NEXT: v_writelane_b32 v47, s17, 48
+; SI-NEXT: s_lshr_b64 s[16:17], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 49
+; SI-NEXT: v_writelane_b32 v47, s17, 50
+; SI-NEXT: s_lshr_b64 s[16:17], s[56:57], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 51
+; SI-NEXT: v_writelane_b32 v47, s17, 52
+; SI-NEXT: s_lshr_b64 s[16:17], s[46:47], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 53
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -174605,11 +174820,9 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_add_i32 s13, s13, 0x3000000
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x3000000
-; SI-NEXT: s_lshr_b64 s[30:31], s[72:73], 16
-; SI-NEXT: s_lshr_b64 s[94:95], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 54
+; SI-NEXT: s_lshr_b64 s[16:17], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 55
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
@@ -174617,18 +174830,15 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[82:83], s[74:75], 16
-; SI-NEXT: s_lshr_b64 vcc, s[60:61], 16
-; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[80:81], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[84:85], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[68:69], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 56
+; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b32 s49, s75, 16
; SI-NEXT: s_lshr_b32 s51, s73, 16
; SI-NEXT: s_lshr_b32 s53, s63, 16
; SI-NEXT: s_lshr_b32 s55, s61, 16
; SI-NEXT: s_lshr_b32 s65, s59, 16
-; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s69, s57, 16
; SI-NEXT: s_lshr_b32 s79, s47, 16
; SI-NEXT: s_lshr_b32 s89, s45, 16
; SI-NEXT: s_lshr_b32 s91, s43, 16
@@ -174637,7 +174847,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: s_lshr_b32 s17, s11, 16
; SI-NEXT: s_lshr_b32 s92, s9, 16
-; SI-NEXT: s_lshr_b32 s78, s7, 16
+; SI-NEXT: s_lshr_b32 s68, s7, 16
; SI-NEXT: s_lshr_b32 s39, s5, 16
; SI-NEXT: .LBB93_5: ; %end
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -174650,45 +174860,57 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s16, s16, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s49, 16
+; SI-NEXT: v_readlane_b32 s22, v47, 43
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s20, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s30, 16
+; SI-NEXT: s_lshl_b32 s21, s22, 16
+; SI-NEXT: v_readlane_b32 s23, v47, 44
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_and_b32 s21, s73, 0xffff
; SI-NEXT: s_lshl_b32 s22, s51, 16
+; SI-NEXT: v_readlane_b32 s24, v47, 45
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s22, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s94, 16
+; SI-NEXT: s_lshl_b32 s23, s24, 16
+; SI-NEXT: v_readlane_b32 s25, v47, 46
; SI-NEXT: s_or_b32 s22, s22, s23
; SI-NEXT: s_and_b32 s23, s63, 0xffff
; SI-NEXT: s_lshl_b32 s24, s53, 16
+; SI-NEXT: v_readlane_b32 s26, v47, 47
; SI-NEXT: s_or_b32 s23, s23, s24
; SI-NEXT: s_and_b32 s24, s60, 0xffff
-; SI-NEXT: s_lshl_b32 s25, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s25, s26, 16
+; SI-NEXT: v_readlane_b32 s27, v47, 48
; SI-NEXT: s_or_b32 s24, s24, s25
; SI-NEXT: s_and_b32 s25, s61, 0xffff
; SI-NEXT: s_lshl_b32 s26, s55, 16
+; SI-NEXT: v_readlane_b32 s28, v47, 49
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s58, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s70, 16
+; SI-NEXT: s_lshl_b32 s27, s28, 16
+; SI-NEXT: v_readlane_b32 s29, v47, 50
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s59, 0xffff
; SI-NEXT: s_lshl_b32 s28, s65, 16
+; SI-NEXT: v_readlane_b32 s58, v47, 51
; SI-NEXT: s_or_b32 s27, s27, s28
; SI-NEXT: s_and_b32 s28, s56, 0xffff
-; SI-NEXT: s_lshl_b32 s29, s80, 16
+; SI-NEXT: s_lshl_b32 s29, s58, 16
; SI-NEXT: s_or_b32 s28, s28, s29
; SI-NEXT: s_and_b32 s29, s57, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_lshl_b32 s56, s69, 16
; SI-NEXT: s_or_b32 s29, s29, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 53
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s84, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: v_readlane_b32 s57, v47, 54
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s47, s47, 0xffff
; SI-NEXT: s_lshl_b32 s56, s79, 16
; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 55
; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_or_b32 s44, s44, s56
; SI-NEXT: s_and_b32 s45, s45, 0xffff
; SI-NEXT: s_lshl_b32 s56, s89, 16
@@ -174726,7 +174948,7 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_lshl_b32 s56, s48, 16
; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s78, 16
+; SI-NEXT: s_lshl_b32 s17, s68, 16
; SI-NEXT: s_or_b32 s12, s12, s56
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s56, s31, 16
@@ -174742,6 +174964,8 @@ define inreg <64 x half> @bitcast_v128i8_to_v64f16_scalar(<128 x i8> inreg %a, i
; SI-NEXT: s_or_b32 s10, s10, s56
; SI-NEXT: s_or_b32 s5, s5, s17
; SI-NEXT: v_readlane_b32 s30, v45, 34
+; SI-NEXT: v_readlane_b32 s59, v47, 52
+; SI-NEXT: v_readlane_b32 s57, v47, 56
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s19
; SI-NEXT: v_mov_b32_e32 v2, s20
@@ -194023,13 +194247,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s4, v21
; SI-NEXT: v_writelane_b32 v56, s4, 21
; SI-NEXT: v_readfirstlane_b32 s4, v20
+; SI-NEXT: v_writelane_b32 v56, s4, 22
+; SI-NEXT: v_readfirstlane_b32 s4, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:332
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:328
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:328
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:324
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:320
-; SI-NEXT: v_writelane_b32 v56, s4, 22
-; SI-NEXT: v_readfirstlane_b32 s4, v19
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:316
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:316
; SI-NEXT: v_writelane_b32 v56, s4, 23
; SI-NEXT: v_readfirstlane_b32 s4, v18
; SI-NEXT: v_writelane_b32 v56, s4, 24
@@ -194039,28 +194263,26 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_writelane_b32 v56, s4, 26
; SI-NEXT: v_readfirstlane_b32 s4, v15
; SI-NEXT: v_writelane_b32 v56, s4, 27
-; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v11
-; SI-NEXT: v_writelane_b32 v56, s4, 29
+; SI-NEXT: v_writelane_b32 v56, s4, 28
; SI-NEXT: v_readfirstlane_b32 s4, v9
-; SI-NEXT: v_writelane_b32 v56, s4, 30
+; SI-NEXT: v_writelane_b32 v56, s4, 29
; SI-NEXT: v_readfirstlane_b32 s4, v8
-; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_writelane_b32 v56, s4, 30
; SI-NEXT: v_readfirstlane_b32 s4, v7
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:288
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:284
; SI-NEXT: buffer_load_dword v11, off, s[0:3], s32 offset:264
-; SI-NEXT: v_writelane_b32 v56, s4, 32
-; SI-NEXT: v_readfirstlane_b32 s4, v6
+; SI-NEXT: v_writelane_b32 v56, s4, 31
+; SI-NEXT: v_readfirstlane_b32 s87, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:280
; SI-NEXT: v_readfirstlane_b32 s98, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:276
; SI-NEXT: v_readfirstlane_b32 s99, v4
; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:272
-; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:268
+; SI-NEXT: v_readfirstlane_b32 s86, v14
; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:312
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:308
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:292
@@ -194070,34 +194292,35 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:300
; SI-NEXT: v_readfirstlane_b32 s76, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:296
-; SI-NEXT: v_writelane_b32 v56, s4, 34
+; SI-NEXT: v_writelane_b32 v56, s4, 32
; SI-NEXT: v_readfirstlane_b32 s77, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:260
; SI-NEXT: ; implicit-def: $vgpr47 : SGPR spill to VGPR lane
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v22
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s81, v21
+; SI-NEXT: v_readfirstlane_b32 s83, v21
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:228
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:224
; SI-NEXT: v_readfirstlane_b32 s4, v1
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:256
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:252
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:248
-; SI-NEXT: v_writelane_b32 v56, s4, 35
+; SI-NEXT: v_writelane_b32 v56, s4, 33
; SI-NEXT: v_readfirstlane_b32 s4, v0
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:244
; SI-NEXT: buffer_load_dword v0, off, s[0:3], s32 offset:240
-; SI-NEXT: v_readfirstlane_b32 s82, v23
+; SI-NEXT: v_readfirstlane_b32 s90, v23
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:220
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:216
-; SI-NEXT: v_readfirstlane_b32 s80, v20
+; SI-NEXT: v_readfirstlane_b32 s82, v19
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:236
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:232
-; SI-NEXT: v_readfirstlane_b32 s83, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:232
+; SI-NEXT: v_readfirstlane_b32 s94, v20
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:212
-; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:208
-; SI-NEXT: v_writelane_b32 v56, s4, 36
+; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:208
+; SI-NEXT: v_writelane_b32 v47, s94, 0
+; SI-NEXT: v_writelane_b32 v56, s4, 34
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s21, v8
; SI-NEXT: buffer_load_dword v8, off, s[0:3], s32 offset:156
@@ -194108,56 +194331,56 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s46, v7
; SI-NEXT: buffer_load_dword v7, off, s[0:3], s32 offset:148
; SI-NEXT: buffer_load_dword v34, off, s[0:3], s32 offset:144
-; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_readfirstlane_b32 s63, v6
; SI-NEXT: buffer_load_dword v6, off, s[0:3], s32 offset:140
; SI-NEXT: buffer_load_dword v35, off, s[0:3], s32 offset:136
-; SI-NEXT: v_readfirstlane_b32 s73, v5
-; SI-NEXT: v_readfirstlane_b32 s40, v4
-; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
-; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v5
; SI-NEXT: buffer_load_dword v5, off, s[0:3], s32 offset:132
; SI-NEXT: buffer_load_dword v36, off, s[0:3], s32 offset:128
+; SI-NEXT: v_readfirstlane_b32 s40, v4
; SI-NEXT: v_readfirstlane_b32 s69, v3
; SI-NEXT: buffer_load_dword v3, off, s[0:3], s32 offset:116
; SI-NEXT: buffer_load_dword v38, off, s[0:3], s32 offset:112
; SI-NEXT: v_readfirstlane_b32 s47, v15
+; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
+; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
; SI-NEXT: v_readfirstlane_b32 s55, v9
; SI-NEXT: buffer_load_dword v9, off, s[0:3], s32 offset:164
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:160
-; SI-NEXT: buffer_load_dword v15, off, s[0:3], s32 offset:204
-; SI-NEXT: buffer_load_dword v26, off, s[0:3], s32 offset:200
-; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s56, v14
; SI-NEXT: buffer_load_dword v14, off, s[0:3], s32 offset:196
; SI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:192
; SI-NEXT: v_readfirstlane_b32 s16, v12
+; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
+; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
; SI-NEXT: v_readfirstlane_b32 s54, v10
; SI-NEXT: buffer_load_dword v10, off, s[0:3], s32 offset:172
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:168
-; SI-NEXT: buffer_load_dword v12, off, s[0:3], s32 offset:188
-; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 offset:184
+; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: buffer_load_dword v2, off, s[0:3], s32 offset:100
; SI-NEXT: buffer_load_dword v48, off, s[0:3], s32 offset:96
-; SI-NEXT: v_readfirstlane_b32 s18, v13
; SI-NEXT: buffer_load_dword v13, off, s[0:3], s32 offset:180
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 offset:176
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v21
+; SI-NEXT: buffer_load_dword v4, off, s[0:3], s32 offset:124
+; SI-NEXT: buffer_load_dword v37, off, s[0:3], s32 offset:120
+; SI-NEXT: v_readfirstlane_b32 s59, v24
+; SI-NEXT: v_readfirstlane_b32 s62, v21
; SI-NEXT: buffer_load_dword v21, off, s[0:3], s32 offset:28
; SI-NEXT: s_waitcnt expcnt(3)
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:24
-; SI-NEXT: v_readfirstlane_b32 s84, v17
+; SI-NEXT: v_readfirstlane_b32 s74, v17
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v51, off, s[0:3], s32 offset:72
-; SI-NEXT: v_readfirstlane_b32 s90, v18
+; SI-NEXT: v_readfirstlane_b32 s29, v18
; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:68
; SI-NEXT: buffer_load_dword v52, off, s[0:3], s32 offset:64
-; SI-NEXT: v_readfirstlane_b32 s23, v25
+; SI-NEXT: v_readfirstlane_b32 s72, v25
; SI-NEXT: buffer_load_dword v25, off, s[0:3], s32 offset:20
; SI-NEXT: s_waitcnt expcnt(2)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:16
-; SI-NEXT: v_readfirstlane_b32 s65, v23
+; SI-NEXT: v_readfirstlane_b32 s64, v23
; SI-NEXT: buffer_load_dword v23, off, s[0:3], s32 offset:12
; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: buffer_load_dword v43, off, s[0:3], s32 offset:8
@@ -194167,190 +194390,187 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s37, v22
; SI-NEXT: buffer_load_dword v22, off, s[0:3], s32 offset:52
; SI-NEXT: buffer_load_dword v54, off, s[0:3], s32 offset:48
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: buffer_load_dword v20, off, s[0:3], s32 offset:44
+; SI-NEXT: v_readfirstlane_b32 s71, v19
+; SI-NEXT: buffer_load_dword v19, off, s[0:3], s32 offset:44
; SI-NEXT: buffer_load_dword v55, off, s[0:3], s32 offset:40
-; SI-NEXT: v_readfirstlane_b32 s17, v16
+; SI-NEXT: v_readfirstlane_b32 s58, v16
; SI-NEXT: buffer_load_dword v16, off, s[0:3], s32 offset:92
; SI-NEXT: buffer_load_dword v49, off, s[0:3], s32 offset:88
-; SI-NEXT: v_readfirstlane_b32 s62, v1
+; SI-NEXT: v_readfirstlane_b32 s60, v1
; SI-NEXT: buffer_load_dword v1, off, s[0:3], s32 offset:84
; SI-NEXT: buffer_load_dword v50, off, s[0:3], s32 offset:80
; SI-NEXT: buffer_load_dword v24, off, s[0:3], s32 offset:36
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:32
-; SI-NEXT: v_readfirstlane_b32 s94, v29
+; SI-NEXT: v_readfirstlane_b32 s65, v29
; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32
-; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s13, v11
-; SI-NEXT: v_readfirstlane_b32 s66, v33
+; SI-NEXT: v_readfirstlane_b32 s88, v8
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_readfirstlane_b32 s66, v33
; SI-NEXT: v_readfirstlane_b32 s14, v34
; SI-NEXT: v_readfirstlane_b32 s15, v6
; SI-NEXT: v_readfirstlane_b32 s93, v39
-; SI-NEXT: v_readfirstlane_b32 s39, v35
-; SI-NEXT: v_readfirstlane_b32 s49, v4
-; SI-NEXT: v_readfirstlane_b32 s51, v37
-; SI-NEXT: v_writelane_b32 v47, s51, 0
-; SI-NEXT: v_writelane_b32 v47, s54, 1
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_readfirstlane_b32 s12, v38
-; SI-NEXT: v_writelane_b32 v47, s83, 2
+; SI-NEXT: v_writelane_b32 v47, s12, 1
+; SI-NEXT: v_readfirstlane_b32 s57, v26
; SI-NEXT: v_readfirstlane_b32 s52, v9
-; SI-NEXT: v_writelane_b32 v47, s12, 3
-; SI-NEXT: v_writelane_b32 v47, s52, 4
+; SI-NEXT: v_writelane_b32 v47, s52, 2
; SI-NEXT: v_readfirstlane_b32 s67, v32
-; SI-NEXT: v_readfirstlane_b32 s58, v14
-; SI-NEXT: v_readfirstlane_b32 s59, v26
-; SI-NEXT: v_readfirstlane_b32 s61, v27
+; SI-NEXT: v_readfirstlane_b32 s80, v14
+; SI-NEXT: v_readfirstlane_b32 s17, v27
+; SI-NEXT: v_readfirstlane_b32 s61, v12
+; SI-NEXT: v_readfirstlane_b32 s73, v28
; SI-NEXT: v_readfirstlane_b32 s53, v31
-; SI-NEXT: v_writelane_b32 v47, s53, 5
-; SI-NEXT: v_writelane_b32 v47, s55, 6
-; SI-NEXT: v_writelane_b32 v47, s13, 7
-; SI-NEXT: v_writelane_b32 v47, s67, 8
-; SI-NEXT: v_writelane_b32 v47, s88, 9
-; SI-NEXT: v_writelane_b32 v47, s44, 10
-; SI-NEXT: v_writelane_b32 v47, s45, 11
-; SI-NEXT: v_writelane_b32 v47, s46, 12
-; SI-NEXT: v_writelane_b32 v47, s78, 13
-; SI-NEXT: v_writelane_b32 v47, s66, 14
-; SI-NEXT: v_writelane_b32 v47, s47, 15
-; SI-NEXT: v_writelane_b32 v47, s56, 16
-; SI-NEXT: v_writelane_b32 v47, s57, 17
-; SI-NEXT: v_writelane_b32 v47, s14, 18
-; SI-NEXT: v_writelane_b32 v47, s58, 19
-; SI-NEXT: v_writelane_b32 v47, s59, 20
-; SI-NEXT: v_writelane_b32 v47, s17, 21
-; SI-NEXT: v_writelane_b32 v47, s60, 22
-; SI-NEXT: v_writelane_b32 v47, s15, 23
-; SI-NEXT: v_writelane_b32 v47, s61, 24
-; SI-NEXT: v_readfirstlane_b32 s63, v12
-; SI-NEXT: v_writelane_b32 v47, s62, 25
-; SI-NEXT: v_writelane_b32 v47, s63, 26
-; SI-NEXT: v_writelane_b32 v47, s16, 27
+; SI-NEXT: v_writelane_b32 v47, s53, 3
+; SI-NEXT: v_writelane_b32 v47, s55, 4
+; SI-NEXT: v_writelane_b32 v47, s13, 5
+; SI-NEXT: v_writelane_b32 v47, s67, 6
+; SI-NEXT: v_writelane_b32 v47, s88, 7
+; SI-NEXT: v_writelane_b32 v47, s44, 8
+; SI-NEXT: v_writelane_b32 v47, s45, 9
+; SI-NEXT: v_writelane_b32 v47, s46, 10
+; SI-NEXT: v_writelane_b32 v47, s78, 11
+; SI-NEXT: v_writelane_b32 v47, s66, 12
+; SI-NEXT: v_writelane_b32 v47, s71, 13
+; SI-NEXT: v_writelane_b32 v47, s14, 14
+; SI-NEXT: v_writelane_b32 v47, s80, 15
+; SI-NEXT: v_writelane_b32 v47, s57, 16
+; SI-NEXT: v_writelane_b32 v47, s58, 17
+; SI-NEXT: v_writelane_b32 v47, s59, 18
+; SI-NEXT: v_writelane_b32 v47, s15, 19
+; SI-NEXT: v_writelane_b32 v47, s17, 20
+; SI-NEXT: v_writelane_b32 v47, s60, 21
; SI-NEXT: v_readfirstlane_b32 s92, v2
-; SI-NEXT: v_writelane_b32 v47, vcc_lo, 28
-; SI-NEXT: v_writelane_b32 v47, s72, 29
-; SI-NEXT: v_writelane_b32 v47, s73, 30
-; SI-NEXT: v_writelane_b32 v47, s23, 31
-; SI-NEXT: v_readfirstlane_b32 s74, v13
+; SI-NEXT: v_writelane_b32 v47, s61, 22
+; SI-NEXT: v_writelane_b32 v47, s16, 23
+; SI-NEXT: v_writelane_b32 v47, s62, 24
+; SI-NEXT: v_writelane_b32 v47, s63, 25
; SI-NEXT: v_readfirstlane_b32 s95, v48
-; SI-NEXT: v_writelane_b32 v47, s18, 32
-; SI-NEXT: v_readfirstlane_b32 s64, v28
-; SI-NEXT: v_writelane_b32 v47, s74, 33
-; SI-NEXT: v_writelane_b32 v47, s64, 34
+; SI-NEXT: v_writelane_b32 v47, vcc_lo, 26
+; SI-NEXT: v_writelane_b32 v47, s72, 27
+; SI-NEXT: v_readfirstlane_b32 s23, v13
+; SI-NEXT: v_writelane_b32 v47, s18, 28
+; SI-NEXT: v_writelane_b32 v47, s23, 29
+; SI-NEXT: v_writelane_b32 v47, s73, 30
; SI-NEXT: v_readfirstlane_b32 s41, v30
+; SI-NEXT: v_writelane_b32 v47, s40, 31
+; SI-NEXT: v_writelane_b32 v47, s41, 32
+; SI-NEXT: v_readfirstlane_b32 s68, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v5
+; SI-NEXT: v_writelane_b32 v47, s74, 33
+; SI-NEXT: v_readfirstlane_b32 s39, v35
+; SI-NEXT: v_writelane_b32 v47, s68, 34
+; SI-NEXT: v_writelane_b32 v47, s64, 35
; SI-NEXT: v_readfirstlane_b32 s19, v41
-; SI-NEXT: v_writelane_b32 v56, s19, 37
+; SI-NEXT: v_writelane_b32 v56, s19, 35
; SI-NEXT: v_readfirstlane_b32 s43, v51
; SI-NEXT: v_readfirstlane_b32 s42, v18
-; SI-NEXT: v_writelane_b32 v56, s42, 38
-; SI-NEXT: v_writelane_b32 v56, s43, 39
+; SI-NEXT: v_writelane_b32 v56, s42, 36
+; SI-NEXT: v_writelane_b32 v56, s43, 37
; SI-NEXT: v_readfirstlane_b32 s6, v42
; SI-NEXT: v_readfirstlane_b32 s7, v23
-; SI-NEXT: v_writelane_b32 v56, s6, 40
+; SI-NEXT: v_writelane_b32 v56, s6, 38
; SI-NEXT: v_readfirstlane_b32 s75, v52
-; SI-NEXT: v_writelane_b32 v56, s7, 41
+; SI-NEXT: v_writelane_b32 v56, s7, 39
; SI-NEXT: s_waitcnt vmcnt(13)
; SI-NEXT: v_readfirstlane_b32 s79, v0
-; SI-NEXT: v_writelane_b32 v56, s75, 42
+; SI-NEXT: v_writelane_b32 v56, s75, 40
; SI-NEXT: s_waitcnt vmcnt(11)
; SI-NEXT: v_readfirstlane_b32 s89, v22
-; SI-NEXT: v_writelane_b32 v56, s79, 43
+; SI-NEXT: v_writelane_b32 v56, s79, 41
; SI-NEXT: v_readfirstlane_b32 s91, v53
-; SI-NEXT: v_writelane_b32 v56, s89, 44
-; SI-NEXT: v_writelane_b32 v56, s91, 45
-; SI-NEXT: v_writelane_b32 v56, s92, 46
+; SI-NEXT: v_writelane_b32 v56, s89, 42
+; SI-NEXT: v_writelane_b32 v56, s91, 43
+; SI-NEXT: v_writelane_b32 v56, s92, 44
; SI-NEXT: s_waitcnt vmcnt(10)
; SI-NEXT: v_readfirstlane_b32 s8, v54
-; SI-NEXT: v_writelane_b32 v56, s93, 47
+; SI-NEXT: v_writelane_b32 v56, s93, 45
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s9, v20
-; SI-NEXT: v_writelane_b32 v56, s8, 48
-; SI-NEXT: v_writelane_b32 v56, s9, 49
+; SI-NEXT: v_readfirstlane_b32 s9, v19
+; SI-NEXT: v_writelane_b32 v56, s8, 46
+; SI-NEXT: v_writelane_b32 v56, s9, 47
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s31, v16
-; SI-NEXT: v_writelane_b32 v56, s95, 50
-; SI-NEXT: v_writelane_b32 v56, s31, 51
+; SI-NEXT: v_writelane_b32 v56, s95, 48
+; SI-NEXT: v_writelane_b32 v56, s31, 49
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_writelane_b32 v56, s80, 52
-; SI-NEXT: v_writelane_b32 v47, s40, 35
+; SI-NEXT: v_writelane_b32 v56, s82, 50
; SI-NEXT: v_readfirstlane_b32 s36, v49
-; SI-NEXT: v_writelane_b32 v56, s34, 53
-; SI-NEXT: v_writelane_b32 v47, s41, 36
-; SI-NEXT: v_readfirstlane_b32 s68, v10
-; SI-NEXT: v_writelane_b32 v56, s36, 54
-; SI-NEXT: v_writelane_b32 v47, s84, 37
+; SI-NEXT: v_writelane_b32 v56, s34, 51
+; SI-NEXT: v_writelane_b32 v56, s36, 52
; SI-NEXT: s_waitcnt vmcnt(4)
; SI-NEXT: v_readfirstlane_b32 s10, v50
-; SI-NEXT: v_writelane_b32 v56, s81, 55
-; SI-NEXT: v_writelane_b32 v47, s68, 38
-; SI-NEXT: v_readfirstlane_b32 s38, v5
-; SI-NEXT: v_writelane_b32 v56, s10, 56
-; SI-NEXT: v_writelane_b32 v47, s65, 39
-; SI-NEXT: v_writelane_b32 v56, s38, 57
-; SI-NEXT: v_writelane_b32 v47, s90, 40
+; SI-NEXT: v_writelane_b32 v56, s83, 53
+; SI-NEXT: v_writelane_b32 v56, s10, 54
+; SI-NEXT: v_writelane_b32 v56, s38, 55
; SI-NEXT: v_readfirstlane_b32 s11, v17
-; SI-NEXT: v_writelane_b32 v56, s39, 58
-; SI-NEXT: v_writelane_b32 v47, s69, 41
+; SI-NEXT: v_writelane_b32 v56, s39, 56
; SI-NEXT: v_readfirstlane_b32 s48, v36
-; SI-NEXT: v_writelane_b32 v56, s11, 59
-; SI-NEXT: v_writelane_b32 v47, s94, 42
-; SI-NEXT: v_writelane_b32 v56, s48, 60
-; SI-NEXT: v_writelane_b32 v47, s70, 43
-; SI-NEXT: v_readfirstlane_b32 s35, v19
-; SI-NEXT: v_writelane_b32 v56, s49, 61
-; SI-NEXT: v_writelane_b32 v47, s37, 44
-; SI-NEXT: v_readfirstlane_b32 s71, v15
+; SI-NEXT: v_writelane_b32 v56, s11, 57
+; SI-NEXT: v_writelane_b32 v47, s29, 36
+; SI-NEXT: v_readfirstlane_b32 s49, v4
+; SI-NEXT: v_writelane_b32 v56, s48, 58
+; SI-NEXT: v_writelane_b32 v47, s69, 37
+; SI-NEXT: v_writelane_b32 v56, s49, 59
+; SI-NEXT: v_writelane_b32 v47, s65, 38
; SI-NEXT: v_readfirstlane_b32 s50, v3
+; SI-NEXT: v_writelane_b32 v56, s90, 60
+; SI-NEXT: v_writelane_b32 v47, s70, 39
+; SI-NEXT: v_readfirstlane_b32 s35, v20
+; SI-NEXT: v_readfirstlane_b32 s51, v37
+; SI-NEXT: v_writelane_b32 v56, s50, 61
+; SI-NEXT: v_writelane_b32 v47, s37, 40
+; SI-NEXT: v_readfirstlane_b32 s81, v15
; SI-NEXT: v_readfirstlane_b32 s22, v55
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s85, v24
+; SI-NEXT: v_readfirstlane_b32 s30, v24
; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s25, v40
; SI-NEXT: v_readfirstlane_b32 s28, v21
-; SI-NEXT: v_readfirstlane_b32 s87, v25
+; SI-NEXT: v_readfirstlane_b32 s85, v25
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v43
; SI-NEXT: s_waitcnt vmcnt(1)
; SI-NEXT: v_readfirstlane_b32 s26, v29
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s27, v44
-; SI-NEXT: v_writelane_b32 v56, s82, 62
-; SI-NEXT: v_writelane_b32 v47, s35, 45
-; SI-NEXT: v_writelane_b32 v56, s50, 63
-; SI-NEXT: v_writelane_b32 v47, s71, 46
+; SI-NEXT: v_writelane_b32 v56, s51, 62
+; SI-NEXT: v_writelane_b32 v47, s35, 41
+; SI-NEXT: v_writelane_b32 v56, s54, 63
+; SI-NEXT: v_writelane_b32 v47, s81, 42
; SI-NEXT: s_cbranch_scc0 .LBB97_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: v_readlane_b32 s4, v56, 11
; SI-NEXT: v_readlane_b32 s5, v56, 10
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: v_writelane_b32 v47, s21, 47
+; SI-NEXT: v_writelane_b32 v47, s21, 57
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 4
-; SI-NEXT: v_writelane_b32 v47, s4, 48
+; SI-NEXT: v_writelane_b32 v47, s4, 43
; SI-NEXT: s_and_b32 s4, s24, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 34
-; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_readlane_b32 s5, v56, 32
+; SI-NEXT: v_writelane_b32 v47, s4, 45
; SI-NEXT: s_and_b32 s4, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_readlane_b32 s5, v56, 29
-; SI-NEXT: v_writelane_b32 v47, s4, 50
+; SI-NEXT: v_readlane_b32 s5, v56, 28
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s4, s76, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s4, 49
; SI-NEXT: v_readlane_b32 s4, v56, 24
; SI-NEXT: s_and_b32 s5, s4, 0xff
; SI-NEXT: v_readlane_b32 s4, v56, 23
; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_writelane_b32 v47, s4, 52
+; SI-NEXT: v_writelane_b32 v47, s4, 51
; SI-NEXT: v_readlane_b32 s4, v56, 16
; SI-NEXT: v_readlane_b32 s5, v56, 15
; SI-NEXT: s_and_b32 s4, s4, 0xff
@@ -194360,41 +194580,45 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s7, 0xff
; SI-NEXT: s_lshl_b32 s5, s6, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 54
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_and_b32 s4, s9, 0xff
; SI-NEXT: s_lshl_b32 s5, s8, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s4, 58
; SI-NEXT: s_and_b32 s4, s11, 0xff
; SI-NEXT: s_lshl_b32 s5, s10, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 56
+; SI-NEXT: v_writelane_b32 v47, s4, 59
; SI-NEXT: s_and_b32 s4, s13, 0xff
; SI-NEXT: s_lshl_b32 s5, s12, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 57
+; SI-NEXT: v_writelane_b32 v47, s4, 60
; SI-NEXT: s_and_b32 s4, s15, 0xff
; SI-NEXT: s_lshl_b32 s5, s14, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 58
+; SI-NEXT: v_writelane_b32 v47, s4, 61
; SI-NEXT: s_and_b32 s4, s68, 0xff
; SI-NEXT: s_lshl_b32 s5, s41, 8
; SI-NEXT: s_or_b32 s4, s4, s5
-; SI-NEXT: v_writelane_b32 v47, s4, 59
-; SI-NEXT: s_and_b32 s4, s71, 0xff
+; SI-NEXT: v_writelane_b32 v47, s4, 62
+; SI-NEXT: s_and_b32 s4, s81, 0xff
; SI-NEXT: s_lshl_b32 s5, s35, 8
-; SI-NEXT: s_or_b32 s71, s4, s5
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v47, s4, 63
; SI-NEXT: s_and_b32 s4, s37, 0xff
; SI-NEXT: s_lshl_b32 s5, s70, 8
-; SI-NEXT: s_or_b32 s70, s4, s5
-; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_lshl_b32 s5, s40, 8
-; SI-NEXT: s_or_b32 s69, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 0
+; SI-NEXT: s_and_b32 s4, s69, 0xff
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_writelane_b32 v46, s4, 1
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s5, s16, 8
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: v_readlane_b32 s5, v56, 7
-; SI-NEXT: v_writelane_b32 v47, s4, 60
+; SI-NEXT: v_writelane_b32 v46, s4, 2
; SI-NEXT: s_and_b32 s4, s20, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_or_b32 s5, s4, s5
@@ -194403,57 +194627,52 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s6, v56, 5
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s7, s6, s4
+; SI-NEXT: s_or_b32 s81, s6, s4
; SI-NEXT: v_readlane_b32 s4, v56, 9
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: v_readlane_b32 s6, v56, 8
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s7, 61
; SI-NEXT: s_or_b32 s4, s6, s4
-; SI-NEXT: s_or_b32 s5, s5, s7
; SI-NEXT: v_readlane_b32 s6, v56, 1
; SI-NEXT: v_readlane_b32 s7, v56, 0
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_or_b32 s7, s6, s7
-; SI-NEXT: v_readlane_b32 s6, v56, 36
+; SI-NEXT: v_readlane_b32 s6, v56, 34
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: v_readlane_b32 s8, v56, 35
+; SI-NEXT: v_readlane_b32 s8, v56, 33
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_or_b32 s9, s8, s6
; SI-NEXT: v_readlane_b32 s6, v56, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: v_readlane_b32 s8, v56, 2
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s9, 3
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s9, 62
-; SI-NEXT: s_or_b32 s6, s8, s6
; SI-NEXT: s_or_b32 s7, s7, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 33
-; SI-NEXT: v_readlane_b32 s9, v56, 32
-; SI-NEXT: s_and_b32 s8, s8, 0xff
+; SI-NEXT: v_readlane_b32 s9, v56, 31
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_and_b32 s8, s87, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
; SI-NEXT: s_or_b32 s9, s8, s9
-; SI-NEXT: v_readlane_b32 s8, v56, 31
+; SI-NEXT: v_readlane_b32 s8, v56, 30
; SI-NEXT: s_and_b32 s8, s8, 0xff
-; SI-NEXT: v_readlane_b32 s10, v56, 30
+; SI-NEXT: v_readlane_b32 s10, v56, 29
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s10, 24
; SI-NEXT: s_or_b32 s11, s10, s8
; SI-NEXT: s_and_b32 s8, s99, 0xff
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: v_writelane_b32 v46, s11, 4
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_lshl_b32 s10, s98, 24
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: v_writelane_b32 v47, s11, 63
-; SI-NEXT: s_or_b32 s8, s10, s8
; SI-NEXT: s_or_b32 s9, s9, s11
-; SI-NEXT: v_readlane_b32 s10, v56, 28
; SI-NEXT: v_readlane_b32 s11, v56, 27
-; SI-NEXT: s_and_b32 s10, s10, 0xff
+; SI-NEXT: s_or_b32 s8, s10, s8
+; SI-NEXT: s_and_b32 s10, s86, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 8
; SI-NEXT: s_or_b32 s11, s10, s11
; SI-NEXT: v_readlane_b32 s10, v56, 26
@@ -194461,16 +194680,12 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s12, v56, 25
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s13, s12, s10
+; SI-NEXT: s_or_b32 s84, s12, s10
; SI-NEXT: s_and_b32 s10, s97, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s12, s96, 24
-; SI-NEXT: ; implicit-def: $vgpr46 : SGPR spill to VGPR lane
; SI-NEXT: s_or_b32 s10, s12, s10
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: v_readlane_b32 s12, v56, 20
-; SI-NEXT: v_writelane_b32 v46, s13, 0
-; SI-NEXT: s_or_b32 s11, s11, s13
; SI-NEXT: s_and_b32 s13, s12, 0xff
; SI-NEXT: v_readlane_b32 s12, v56, 19
; SI-NEXT: s_lshl_b32 s12, s12, 8
@@ -194480,7 +194695,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s14, s12, 16
; SI-NEXT: v_readlane_b32 s12, v56, 17
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s86, s12, s14
+; SI-NEXT: s_or_b32 s69, s12, s14
; SI-NEXT: v_readlane_b32 s12, v56, 22
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s14, s12, 16
@@ -194494,7 +194709,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s14, s26, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_lshl_b32 s16, vcc_hi, 24
-; SI-NEXT: s_or_b32 s29, s16, s14
+; SI-NEXT: s_or_b32 s70, s16, s14
; SI-NEXT: v_readlane_b32 s14, v56, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s16, s14, 16
@@ -194504,16 +194719,16 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s16, s28, 0xff
; SI-NEXT: s_lshl_b32 s18, s25, 8
; SI-NEXT: s_or_b32 s18, s16, s18
-; SI-NEXT: s_and_b32 s16, s85, 0xff
-; SI-NEXT: v_writelane_b32 v46, s20, 1
+; SI-NEXT: s_and_b32 s16, s30, 0xff
+; SI-NEXT: v_writelane_b32 v46, s20, 5
; SI-NEXT: s_lshl_b32 s16, s16, 16
; SI-NEXT: s_lshl_b32 s20, s22, 24
-; SI-NEXT: v_writelane_b32 v46, vcc_hi, 2
-; SI-NEXT: v_writelane_b32 v46, s28, 3
+; SI-NEXT: v_writelane_b32 v46, vcc_hi, 6
+; SI-NEXT: v_writelane_b32 v46, s28, 7
; SI-NEXT: s_or_b32 s28, s20, s16
-; SI-NEXT: s_and_b32 s20, s87, 0xff
+; SI-NEXT: s_and_b32 s20, s85, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: v_writelane_b32 v46, s24, 4
+; SI-NEXT: v_writelane_b32 v46, s24, 8
; SI-NEXT: s_lshl_b32 s24, s19, 24
; SI-NEXT: s_and_b32 s18, s18, 0xffff
; SI-NEXT: s_or_b32 s40, s24, s20
@@ -194527,7 +194742,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s19, s24, s18
; SI-NEXT: s_and_b32 s24, s89, 0xff
; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: v_writelane_b32 v46, s25, 5
+; SI-NEXT: v_writelane_b32 v46, s25, 9
; SI-NEXT: s_lshl_b32 s25, s91, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s42, s25, s24
@@ -194552,7 +194767,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s25, s38, 0xff
; SI-NEXT: s_lshl_b32 s25, s25, 16
; SI-NEXT: s_lshl_b32 s26, s39, 24
-; SI-NEXT: v_writelane_b32 v46, s27, 6
+; SI-NEXT: v_writelane_b32 v46, s27, 10
; SI-NEXT: s_or_b32 s27, s26, s25
; SI-NEXT: s_and_b32 s26, s50, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
@@ -194573,44 +194788,44 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s38, s75, s26
; SI-NEXT: s_or_b32 s39, s20, s18
-; SI-NEXT: s_and_b32 s20, s63, 0xff
-; SI-NEXT: s_lshl_b32 s26, s61, 8
+; SI-NEXT: s_and_b32 s20, s61, 0xff
+; SI-NEXT: s_lshl_b32 s26, s17, 8
; SI-NEXT: s_or_b32 s20, s20, s26
-; SI-NEXT: s_and_b32 s26, s58, 0xff
+; SI-NEXT: s_and_b32 s26, s80, 0xff
; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_lshl_b32 s75, s59, 24
+; SI-NEXT: s_lshl_b32 s75, s57, 24
; SI-NEXT: s_or_b32 s26, s75, s26
-; SI-NEXT: s_and_b32 s75, s74, 0xff
+; SI-NEXT: s_and_b32 s75, s23, 0xff
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s78, s64, 24
+; SI-NEXT: s_lshl_b32 s78, s73, 24
; SI-NEXT: s_and_b32 s20, s20, 0xffff
; SI-NEXT: s_or_b32 s48, s78, s75
; SI-NEXT: s_or_b32 s49, s20, s26
-; SI-NEXT: s_and_b32 s20, s23, 0xff
-; SI-NEXT: s_lshl_b32 s75, vcc_lo, 8
+; SI-NEXT: s_and_b32 s20, s72, 0xff
+; SI-NEXT: s_lshl_b32 s75, s62, 8
; SI-NEXT: s_or_b32 s75, s20, s75
-; SI-NEXT: s_and_b32 s20, s60, 0xff
+; SI-NEXT: s_and_b32 s20, s59, 0xff
; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_lshl_b32 s78, s56, 24
+; SI-NEXT: s_lshl_b32 s78, s71, 24
; SI-NEXT: s_or_b32 s20, s78, s20
-; SI-NEXT: s_and_b32 s78, s94, 0xff
+; SI-NEXT: s_and_b32 s78, s65, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s79, s65, 24
+; SI-NEXT: s_lshl_b32 s79, s64, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_or_b32 s50, s79, s78
; SI-NEXT: s_or_b32 s51, s75, s20
-; SI-NEXT: s_and_b32 s75, s62, 0xff
-; SI-NEXT: s_lshl_b32 s78, s17, 8
+; SI-NEXT: s_and_b32 s75, s60, 0xff
+; SI-NEXT: s_lshl_b32 s78, s58, 8
; SI-NEXT: s_or_b32 s75, s75, s78
; SI-NEXT: s_and_b32 s78, s45, 0xff
; SI-NEXT: s_lshl_b32 s78, s78, 16
; SI-NEXT: s_lshl_b32 s79, s44, 24
; SI-NEXT: s_or_b32 s78, s79, s78
-; SI-NEXT: s_and_b32 s79, s90, 0xff
+; SI-NEXT: s_and_b32 s79, s29, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s88, s84, 24
+; SI-NEXT: s_lshl_b32 s88, s74, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 47
+; SI-NEXT: v_readlane_b32 s16, v47, 57
; SI-NEXT: s_or_b32 s52, s88, s79
; SI-NEXT: s_or_b32 s53, s75, s78
; SI-NEXT: s_and_b32 s75, s46, 0xff
@@ -194620,167 +194835,188 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_lshl_b32 s88, s54, 24
; SI-NEXT: s_or_b32 s88, s88, s79
-; SI-NEXT: s_and_b32 s79, s73, 0xff
+; SI-NEXT: s_and_b32 s79, vcc_lo, 0xff
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s72, 24
+; SI-NEXT: s_lshl_b32 s89, s63, 24
; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 48
+; SI-NEXT: v_readlane_b32 s16, v47, 43
; SI-NEXT: s_or_b32 s54, s89, s79
; SI-NEXT: s_or_b32 s55, s75, s88
-; SI-NEXT: s_and_b32 s75, s83, 0xff
-; SI-NEXT: s_lshl_b32 s79, s82, 8
+; SI-NEXT: s_and_b32 s75, s94, 0xff
+; SI-NEXT: s_lshl_b32 s79, s90, 8
; SI-NEXT: s_and_b32 s44, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 49
+; SI-NEXT: v_readlane_b32 s16, v47, 45
; SI-NEXT: s_or_b32 s75, s75, s79
-; SI-NEXT: s_and_b32 s79, s81, 0xff
+; SI-NEXT: s_and_b32 s79, s83, 0xff
; SI-NEXT: s_and_b32 s45, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 50
+; SI-NEXT: v_readlane_b32 s16, v47, 47
; SI-NEXT: s_lshl_b32 s79, s79, 16
-; SI-NEXT: s_lshl_b32 s89, s80, 24
+; SI-NEXT: s_lshl_b32 s89, s82, 24
; SI-NEXT: s_and_b32 s46, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: v_readlane_b32 s16, v47, 49
; SI-NEXT: s_or_b32 s90, s89, s79
+; SI-NEXT: s_mov_b32 s71, s47
; SI-NEXT: s_lshl_b32 s89, s47, 24
; SI-NEXT: s_and_b32 s47, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 52
-; SI-NEXT: s_and_b32 s79, s57, 0xff
+; SI-NEXT: v_readlane_b32 s16, v47, 51
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: s_and_b32 s79, s56, 0xff
; SI-NEXT: s_and_b32 s56, s16, 0xffff
; SI-NEXT: v_readlane_b32 s16, v47, 53
; SI-NEXT: s_lshl_b32 s79, s79, 16
; SI-NEXT: s_and_b32 s57, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 54
+; SI-NEXT: v_readlane_b32 s16, v47, 55
; SI-NEXT: s_or_b32 s64, s89, s79
; SI-NEXT: s_and_b32 s79, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 55
+; SI-NEXT: v_readlane_b32 s16, v47, 58
; SI-NEXT: s_and_b32 s89, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 56
+; SI-NEXT: v_readlane_b32 s16, v47, 59
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_and_b32 s17, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 57
+; SI-NEXT: v_readlane_b32 s16, v47, 60
+; SI-NEXT: s_or_b32 s5, s5, s81
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
; SI-NEXT: s_and_b32 s91, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 58
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: s_or_b32 s65, s75, s90
; SI-NEXT: s_and_b32 s92, s16, 0xffff
-; SI-NEXT: v_readlane_b32 s16, v47, 59
-; SI-NEXT: s_or_b32 s13, s13, s86
+; SI-NEXT: v_readlane_b32 s16, v47, 62
+; SI-NEXT: s_or_b32 s74, s44, s4
+; SI-NEXT: s_mov_b32 s75, s5
+; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[4:5], s[6:7], 16
; SI-NEXT: s_and_b32 s93, s16, 0xffff
-; SI-NEXT: s_mov_b32 s16, s85
-; SI-NEXT: s_lshr_b64 s[84:85], s[40:41], 16
-; SI-NEXT: s_and_b32 s94, s71, 0xffff
-; SI-NEXT: s_and_b32 s95, s70, 0xffff
-; SI-NEXT: s_or_b32 s58, s56, s12
-; SI-NEXT: s_mov_b32 s59, s13
-; SI-NEXT: s_lshr_b64 s[70:71], s[12:13], 16
-; SI-NEXT: s_or_b32 s12, s93, s48
-; SI-NEXT: s_mov_b32 s13, s49
-; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s85, s16
-; SI-NEXT: v_readlane_b32 s16, v47, 61
+; SI-NEXT: v_readlane_b32 s16, v47, 63
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: s_lshr_b64 s[4:5], s[8:9], 16
+; SI-NEXT: s_or_b32 s11, s11, s84
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 16
+; SI-NEXT: s_or_b32 s13, s13, s69
+; SI-NEXT: v_writelane_b32 v47, s4, 47
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_mov_b32 s25, s22
-; SI-NEXT: s_and_b32 s75, s75, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v47, 60
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: s_lshr_b64 s[4:5], s[12:13], 16
+; SI-NEXT: s_or_b32 s15, s15, s70
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: s_lshr_b64 s[4:5], s[14:15], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: s_lshr_b64 s[4:5], s[40:41], 16
+; SI-NEXT: s_mov_b32 s23, s21
+; SI-NEXT: v_readlane_b32 s21, v46, 1
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: s_and_b32 s94, s16, 0xffff
+; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_and_b32 vcc_lo, s21, 0xffff
+; SI-NEXT: v_readlane_b32 s21, v46, 2
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: s_lshr_b64 s[4:5], s[42:43], 16
+; SI-NEXT: s_and_b32 s95, s16, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s21, 0xffff
; SI-NEXT: s_or_b32 s60, s47, s10
; SI-NEXT: s_mov_b32 s61, s11
-; SI-NEXT: s_lshr_b64 vcc, s[10:11], 16
+; SI-NEXT: v_writelane_b32 v47, s4, 55
; SI-NEXT: s_or_b32 s10, s94, s50
; SI-NEXT: s_mov_b32 s11, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[50:51], 16
-; SI-NEXT: s_lshr_b32 s49, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 62
-; SI-NEXT: s_or_b32 s15, s15, s29
-; SI-NEXT: s_mov_b32 s23, s21
-; SI-NEXT: s_or_b32 s65, s75, s90
-; SI-NEXT: s_and_b32 s21, s69, 0xffff
-; SI-NEXT: s_and_b32 s22, s22, 0xffff
-; SI-NEXT: s_or_b32 s62, s46, s8
-; SI-NEXT: s_mov_b32 s63, s9
-; SI-NEXT: s_lshr_b64 s[66:67], s[8:9], 16
-; SI-NEXT: s_or_b32 s8, s95, s52
-; SI-NEXT: s_mov_b32 s9, s53
-; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_lshr_b32 s51, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v47, 63
-; SI-NEXT: s_or_b32 s74, s44, s4
-; SI-NEXT: s_mov_b32 s75, s5
-; SI-NEXT: s_lshr_b64 s[82:83], s[4:5], 16
+; SI-NEXT: v_readlane_b32 s16, v46, 3
; SI-NEXT: s_or_b32 s72, s45, s6
; SI-NEXT: s_mov_b32 s73, s7
-; SI-NEXT: s_lshr_b64 s[30:31], s[6:7], 16
+; SI-NEXT: s_or_b32 s62, s46, s8
+; SI-NEXT: s_mov_b32 s63, s9
+; SI-NEXT: s_or_b32 s58, s56, s12
+; SI-NEXT: s_mov_b32 s59, s13
; SI-NEXT: s_or_b32 s56, s57, s14
; SI-NEXT: s_mov_b32 s57, s15
-; SI-NEXT: s_lshr_b64 s[80:81], s[14:15], 16
-; SI-NEXT: s_or_b32 s44, s89, s42
-; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_lshr_b64 s[68:69], s[42:43], 16
-; SI-NEXT: s_or_b32 s42, s17, s34
-; SI-NEXT: s_mov_b32 s43, s35
-; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
+; SI-NEXT: v_writelane_b32 v47, s5, 56
; SI-NEXT: s_or_b32 s14, s92, s38
; SI-NEXT: s_mov_b32 s15, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[38:39], 16
-; SI-NEXT: s_or_b32 s6, s21, s54
+; SI-NEXT: s_or_b32 s12, s93, s48
+; SI-NEXT: s_mov_b32 s13, s49
+; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
+; SI-NEXT: s_or_b32 s8, s95, s52
+; SI-NEXT: s_mov_b32 s9, s53
+; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
+; SI-NEXT: s_or_b32 s6, vcc_lo, s54
; SI-NEXT: s_mov_b32 s7, s55
; SI-NEXT: s_lshr_b64 s[54:55], s[54:55], 16
-; SI-NEXT: s_or_b32 s4, s22, s64
+; SI-NEXT: s_or_b32 s4, vcc_hi, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_lshr_b32 s53, s16, 16
-; SI-NEXT: v_readlane_b32 s16, v46, 0
+; SI-NEXT: s_lshr_b32 s51, s16, 16
+; SI-NEXT: v_readlane_b32 s16, v46, 4
; SI-NEXT: s_or_b32 s46, s79, s40
; SI-NEXT: s_mov_b32 s47, s41
+; SI-NEXT: s_or_b32 s44, s89, s42
+; SI-NEXT: s_mov_b32 s45, s43
+; SI-NEXT: s_or_b32 s42, s17, s34
+; SI-NEXT: s_mov_b32 s43, s35
+; SI-NEXT: s_lshr_b64 s[34:35], s[34:35], 16
; SI-NEXT: s_or_b32 s40, s91, s36
; SI-NEXT: s_mov_b32 s41, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[36:37], 16
-; SI-NEXT: s_lshr_b32 s55, s16, 16
-; SI-NEXT: s_lshr_b32 s65, s86, 16
-; SI-NEXT: s_lshr_b32 s35, s29, 16
-; SI-NEXT: s_mov_b32 s94, s66
+; SI-NEXT: s_lshr_b32 s49, s81, 16
+; SI-NEXT: s_lshr_b32 s53, s16, 16
+; SI-NEXT: s_lshr_b32 s55, s84, 16
+; SI-NEXT: s_lshr_b32 s65, s69, 16
+; SI-NEXT: s_lshr_b32 s69, s70, 16
; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_readlane_b32 s28, v46, 3
-; SI-NEXT: v_readlane_b32 vcc_hi, v46, 2
+; SI-NEXT: v_readlane_b32 s28, v46, 7
+; SI-NEXT: v_readlane_b32 vcc_hi, v46, 6
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s91, s24, 16
-; SI-NEXT: v_readlane_b32 s24, v46, 4
+; SI-NEXT: v_readlane_b32 s24, v46, 8
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: v_readlane_b32 s27, v46, 6
+; SI-NEXT: v_readlane_b32 s27, v46, 10
; SI-NEXT: s_lshr_b32 s95, s18, 16
; SI-NEXT: s_lshr_b32 s31, s26, 16
; SI-NEXT: s_lshr_b32 s17, s20, 16
-; SI-NEXT: v_readlane_b32 s20, v46, 1
+; SI-NEXT: v_readlane_b32 s20, v46, 5
; SI-NEXT: s_lshr_b32 s92, s78, 16
-; SI-NEXT: s_lshr_b32 s78, s88, 16
-; SI-NEXT: v_readlane_b32 s21, v47, 47
+; SI-NEXT: s_lshr_b32 s68, s88, 16
+; SI-NEXT: v_readlane_b32 s21, v47, 57
; SI-NEXT: s_mov_b32 s26, s23
-; SI-NEXT: s_mov_b32 s22, s25
-; SI-NEXT: v_readlane_b32 s25, v46, 5
+; SI-NEXT: v_readlane_b32 s25, v46, 9
; SI-NEXT: s_lshr_b32 s39, s90, 16
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB97_3
; SI-NEXT: .LBB97_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s80, s56
+; SI-NEXT: v_writelane_b32 v47, s4, 43
+; SI-NEXT: v_writelane_b32 v47, s5, 44
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s71, s47
+; SI-NEXT: v_writelane_b32 v47, s4, 45
+; SI-NEXT: v_writelane_b32 v47, s5, 46
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v47, s4, 47
+; SI-NEXT: v_writelane_b32 v47, s5, 48
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr49
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr70
; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr69
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
@@ -194802,41 +195038,62 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: ; implicit-def: $sgpr92
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr54
-; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr68
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: v_writelane_b32 v47, s4, 49
+; SI-NEXT: v_writelane_b32 v47, s5, 50
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 51
+; SI-NEXT: v_writelane_b32 v47, s5, 52
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 53
+; SI-NEXT: v_writelane_b32 v47, s5, 54
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v47, s4, 55
+; SI-NEXT: v_writelane_b32 v47, s5, 56
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB97_3: ; %Flow
; SI-NEXT: s_and_b64 s[18:19], s[66:67], exec
; SI-NEXT: s_cselect_b32 s16, 1, 0
; SI-NEXT: s_cmp_lg_u32 s16, 1
+; SI-NEXT: v_readlane_b32 s16, v47, 0
+; SI-NEXT: v_readlane_b32 s18, v47, 7
+; SI-NEXT: v_readlane_b32 s19, v47, 8
+; SI-NEXT: v_readlane_b32 s88, v47, 9
+; SI-NEXT: v_readlane_b32 s90, v47, 10
+; SI-NEXT: v_readlane_b32 s78, v47, 11
+; SI-NEXT: v_readlane_b32 s94, v47, 16
+; SI-NEXT: v_readlane_b32 vcc_lo, v47, 27
+; SI-NEXT: v_readlane_b32 s23, v47, 30
+; SI-NEXT: v_readlane_b32 s29, v47, 36
+; SI-NEXT: v_readlane_b32 s37, v47, 40
+; SI-NEXT: v_readlane_b32 s35, v47, 41
+; SI-NEXT: v_readlane_b32 s66, v47, 42
; SI-NEXT: s_cbranch_scc1 .LBB97_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_readlane_b32 s4, v47, 32
+; SI-NEXT: v_readlane_b32 s4, v47, 28
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: v_readlane_b32 s5, v47, 27
-; SI-NEXT: v_readlane_b32 s6, v47, 17
+; SI-NEXT: v_readlane_b32 s5, v47, 23
; SI-NEXT: s_and_b32 s4, s4, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
-; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_add_i32 s6, s80, 3
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 15
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 24
+; SI-NEXT: s_lshl_b32 s5, s71, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_or_b32 s5, s5, s6
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v47, 2
-; SI-NEXT: s_add_i32 s5, s5, 3
-; SI-NEXT: v_readlane_b32 s6, v56, 62
-; SI-NEXT: v_readlane_b32 s7, v56, 55
+; SI-NEXT: s_add_i32 s5, s16, 3
+; SI-NEXT: v_readlane_b32 s6, v56, 60
+; SI-NEXT: v_readlane_b32 s7, v56, 53
; SI-NEXT: s_and_b32 s5, s5, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 8
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v56, 52
+; SI-NEXT: v_readlane_b32 s6, v56, 50
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 24
; SI-NEXT: s_lshl_b32 s7, s7, 16
@@ -194844,15 +195101,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: v_readlane_b32 s6, v47, 41
+; SI-NEXT: v_readlane_b32 s6, v47, 37
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: v_readlane_b32 s7, v47, 35
-; SI-NEXT: v_readlane_b32 s8, v47, 30
+; SI-NEXT: v_readlane_b32 s7, v47, 31
+; SI-NEXT: v_readlane_b32 s8, v47, 26
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s8, s8, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 29
+; SI-NEXT: v_readlane_b32 s7, v47, 25
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_lshl_b32 s8, s8, 16
@@ -194860,14 +195117,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s7, s7, s8
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: v_readlane_b32 s7, v47, 12
-; SI-NEXT: s_add_i32 s7, s7, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 6
+; SI-NEXT: s_add_i32 s7, s90, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 4
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s21, 8
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 1
+; SI-NEXT: v_readlane_b32 s8, v56, 63
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
@@ -194875,15 +195131,13 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: v_readlane_b32 s8, v47, 44
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: v_readlane_b32 s9, v47, 43
-; SI-NEXT: v_readlane_b32 s10, v47, 40
+; SI-NEXT: s_add_i32 s8, s37, 3
+; SI-NEXT: v_readlane_b32 s9, v47, 39
; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 8
-; SI-NEXT: s_add_i32 s10, s10, 3
+; SI-NEXT: s_add_i32 s10, s29, 3
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 37
+; SI-NEXT: v_readlane_b32 s9, v47, 33
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s9, s9, 24
; SI-NEXT: s_lshl_b32 s10, s10, 16
@@ -194891,31 +195145,27 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_and_b32 s8, s8, 0xffff
; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: v_readlane_b32 s9, v47, 25
+; SI-NEXT: v_readlane_b32 s9, v47, 21
; SI-NEXT: s_add_i32 s9, s9, 3
-; SI-NEXT: v_readlane_b32 s10, v47, 21
-; SI-NEXT: v_readlane_b32 s11, v47, 11
+; SI-NEXT: v_readlane_b32 s10, v47, 17
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s10, 8
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s11, s88, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 10
; SI-NEXT: s_and_b32 s11, s11, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_lshl_b32 s10, s19, 24
; SI-NEXT: s_lshl_b32 s11, s11, 16
; SI-NEXT: s_addk_i32 s9, 0x300
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: v_readlane_b32 s10, v47, 46
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: v_readlane_b32 s11, v47, 45
-; SI-NEXT: v_readlane_b32 s12, v47, 42
+; SI-NEXT: s_add_i32 s10, s66, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 38
; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 8
+; SI-NEXT: s_lshl_b32 s11, s35, 8
; SI-NEXT: s_add_i32 s12, s12, 3
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 39
+; SI-NEXT: v_readlane_b32 s11, v47, 35
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s12, s12, 16
@@ -194923,15 +195173,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s11, s11, s12
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: v_readlane_b32 s11, v47, 31
-; SI-NEXT: s_add_i32 s11, s11, 3
-; SI-NEXT: v_readlane_b32 s12, v47, 28
-; SI-NEXT: v_readlane_b32 s13, v47, 22
+; SI-NEXT: s_add_i32 s11, vcc_lo, 3
+; SI-NEXT: v_readlane_b32 s12, v47, 24
+; SI-NEXT: v_readlane_b32 s13, v47, 18
; SI-NEXT: s_and_b32 s11, s11, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_add_i32 s13, s13, 3
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 16
+; SI-NEXT: v_readlane_b32 s12, v47, 13
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s12, s12, 24
; SI-NEXT: s_lshl_b32 s13, s13, 16
@@ -194939,47 +195188,44 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: v_readlane_b32 s12, v47, 38
+; SI-NEXT: v_readlane_b32 s12, v47, 34
; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: v_readlane_b32 s13, v47, 36
-; SI-NEXT: v_readlane_b32 s14, v47, 33
+; SI-NEXT: v_readlane_b32 s13, v47, 32
+; SI-NEXT: v_readlane_b32 s14, v47, 29
; SI-NEXT: s_and_b32 s12, s12, 0xff
; SI-NEXT: s_lshl_b32 s13, s13, 8
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 34
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s13, s13, 24
+; SI-NEXT: s_lshl_b32 s13, s23, 24
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_addk_i32 s12, 0x300
; SI-NEXT: s_or_b32 s13, s13, s14
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: v_readlane_b32 s13, v47, 26
+; SI-NEXT: v_readlane_b32 s13, v47, 22
; SI-NEXT: s_add_i32 s13, s13, 3
-; SI-NEXT: v_readlane_b32 s14, v47, 24
-; SI-NEXT: v_readlane_b32 s15, v47, 19
+; SI-NEXT: v_readlane_b32 s14, v47, 20
+; SI-NEXT: v_readlane_b32 s15, v47, 15
; SI-NEXT: s_and_b32 s13, s13, 0xff
; SI-NEXT: s_lshl_b32 s14, s14, 8
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 20
; SI-NEXT: s_and_b32 s15, s15, 0xff
-; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_lshl_b32 s14, s94, 24
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_addk_i32 s13, 0x300
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: v_readlane_b32 s14, v47, 23
+; SI-NEXT: v_readlane_b32 s14, v47, 19
; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: v_readlane_b32 s15, v47, 18
-; SI-NEXT: v_readlane_b32 s16, v47, 13
+; SI-NEXT: v_readlane_b32 s15, v47, 14
; SI-NEXT: s_and_b32 s14, s14, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s78, 3
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 14
+; SI-NEXT: v_readlane_b32 s15, v47, 12
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s15, s15, 24
; SI-NEXT: s_lshl_b32 s16, s16, 16
@@ -194987,15 +195233,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s15, s15, s16
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: v_readlane_b32 s15, v47, 9
-; SI-NEXT: s_add_i32 s15, s15, 3
-; SI-NEXT: v_readlane_b32 s16, v47, 8
-; SI-NEXT: v_readlane_b32 s17, v47, 4
+; SI-NEXT: s_add_i32 s15, s18, 3
+; SI-NEXT: v_readlane_b32 s16, v47, 6
+; SI-NEXT: v_readlane_b32 s17, v47, 2
; SI-NEXT: s_and_b32 s15, s15, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 8
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 5
+; SI-NEXT: v_readlane_b32 s16, v47, 3
; SI-NEXT: s_and_b32 s17, s17, 0xff
; SI-NEXT: s_lshl_b32 s16, s16, 24
; SI-NEXT: s_lshl_b32 s17, s17, 16
@@ -195003,15 +195248,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: v_readlane_b32 s16, v47, 7
+; SI-NEXT: v_readlane_b32 s16, v47, 5
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v47, 3
-; SI-NEXT: v_readlane_b32 s18, v56, 63
+; SI-NEXT: v_readlane_b32 s17, v47, 1
+; SI-NEXT: v_readlane_b32 s18, v56, 61
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v47, 0
+; SI-NEXT: v_readlane_b32 s17, v56, 62
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195020,15 +195265,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s40, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 61
+; SI-NEXT: v_readlane_b32 s16, v56, 59
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 60
-; SI-NEXT: v_readlane_b32 s18, v56, 57
+; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s18, v56, 55
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 58
+; SI-NEXT: v_readlane_b32 s17, v56, 56
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195037,15 +195282,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s41, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 59
+; SI-NEXT: v_readlane_b32 s16, v56, 57
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 56
-; SI-NEXT: v_readlane_b32 s18, v56, 53
+; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s18, v56, 51
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 54
+; SI-NEXT: v_readlane_b32 s17, v56, 52
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195054,15 +195299,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 51
+; SI-NEXT: v_readlane_b32 s16, v56, 49
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 50
-; SI-NEXT: v_readlane_b32 s18, v56, 46
+; SI-NEXT: v_readlane_b32 s17, v56, 48
+; SI-NEXT: v_readlane_b32 s18, v56, 44
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 47
+; SI-NEXT: v_readlane_b32 s17, v56, 45
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195071,15 +195316,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s43, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 49
+; SI-NEXT: v_readlane_b32 s16, v56, 47
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 48
-; SI-NEXT: v_readlane_b32 s18, v56, 44
+; SI-NEXT: v_readlane_b32 s17, v56, 46
+; SI-NEXT: v_readlane_b32 s18, v56, 42
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 45
+; SI-NEXT: v_readlane_b32 s17, v56, 43
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195088,15 +195333,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s44, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 43
+; SI-NEXT: v_readlane_b32 s16, v56, 41
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 42
-; SI-NEXT: v_readlane_b32 s18, v56, 38
+; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s18, v56, 36
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 39
+; SI-NEXT: v_readlane_b32 s17, v56, 37
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195105,14 +195350,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s45, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 41
+; SI-NEXT: v_readlane_b32 s16, v56, 39
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 40
+; SI-NEXT: v_readlane_b32 s17, v56, 38
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s87, 3
+; SI-NEXT: s_add_i32 s18, s85, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 37
+; SI-NEXT: v_readlane_b32 s17, v56, 35
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195124,7 +195369,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s16, s28, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s25, 8
-; SI-NEXT: s_add_i32 s18, s85, 3
+; SI-NEXT: s_add_i32 s18, s30, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
@@ -195200,7 +195445,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_add_i32 s59, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s76, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 29
+; SI-NEXT: v_readlane_b32 s17, v56, 28
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s97, 3
@@ -195213,8 +195458,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s60, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 28
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s16, s86, 3
; SI-NEXT: v_readlane_b32 s17, v56, 27
; SI-NEXT: v_readlane_b32 s18, v56, 26
; SI-NEXT: s_and_b32 s16, s16, 0xff
@@ -195231,7 +195475,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s61, s16, 0x3000000
; SI-NEXT: s_add_i32 s16, s77, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 34
+; SI-NEXT: v_readlane_b32 s17, v56, 32
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s99, 3
@@ -195244,15 +195488,14 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s62, s16, 0x3000000
-; SI-NEXT: v_readlane_b32 s16, v56, 33
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: v_readlane_b32 s17, v56, 32
-; SI-NEXT: v_readlane_b32 s18, v56, 31
+; SI-NEXT: s_add_i32 s16, s87, 3
+; SI-NEXT: v_readlane_b32 s17, v56, 31
+; SI-NEXT: v_readlane_b32 s18, v56, 30
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 30
+; SI-NEXT: v_readlane_b32 s17, v56, 29
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195280,12 +195523,12 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: v_readlane_b32 s16, v56, 1
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s17, v56, 0
-; SI-NEXT: v_readlane_b32 s18, v56, 36
+; SI-NEXT: v_readlane_b32 s18, v56, 34
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: v_readlane_b32 s17, v56, 35
+; SI-NEXT: v_readlane_b32 s17, v56, 33
; SI-NEXT: s_and_b32 s18, s18, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s17, 24
@@ -195325,6 +195568,25 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s18, s18, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
+; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s75, s16, 0x3000000
+; SI-NEXT: s_lshr_b64 s[16:17], s[72:73], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 43
+; SI-NEXT: v_writelane_b32 v47, s17, 44
+; SI-NEXT: s_lshr_b64 s[16:17], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 45
+; SI-NEXT: v_writelane_b32 v47, s17, 46
+; SI-NEXT: s_lshr_b64 s[16:17], s[60:61], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 47
+; SI-NEXT: v_writelane_b32 v47, s17, 48
+; SI-NEXT: s_lshr_b64 s[16:17], s[58:59], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 49
+; SI-NEXT: v_writelane_b32 v47, s17, 50
+; SI-NEXT: s_lshr_b64 s[16:17], s[56:57], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 51
+; SI-NEXT: v_writelane_b32 v47, s17, 52
+; SI-NEXT: s_lshr_b64 s[16:17], s[46:47], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 53
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -195337,11 +195599,9 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_add_i32 s13, s13, 0x3000000
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x3000000
-; SI-NEXT: s_lshr_b64 s[30:31], s[72:73], 16
-; SI-NEXT: s_lshr_b64 s[94:95], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 54
+; SI-NEXT: s_lshr_b64 s[16:17], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s16, 55
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
@@ -195349,18 +195609,15 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[82:83], s[74:75], 16
-; SI-NEXT: s_lshr_b64 vcc, s[60:61], 16
-; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 16
-; SI-NEXT: s_lshr_b64 s[80:81], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[84:85], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[68:69], s[44:45], 16
+; SI-NEXT: v_writelane_b32 v47, s17, 56
+; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b32 s49, s75, 16
; SI-NEXT: s_lshr_b32 s51, s73, 16
; SI-NEXT: s_lshr_b32 s53, s63, 16
; SI-NEXT: s_lshr_b32 s55, s61, 16
; SI-NEXT: s_lshr_b32 s65, s59, 16
-; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s69, s57, 16
; SI-NEXT: s_lshr_b32 s79, s47, 16
; SI-NEXT: s_lshr_b32 s89, s45, 16
; SI-NEXT: s_lshr_b32 s91, s43, 16
@@ -195369,7 +195626,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshr_b32 s31, s13, 16
; SI-NEXT: s_lshr_b32 s17, s11, 16
; SI-NEXT: s_lshr_b32 s92, s9, 16
-; SI-NEXT: s_lshr_b32 s78, s7, 16
+; SI-NEXT: s_lshr_b32 s68, s7, 16
; SI-NEXT: s_lshr_b32 s39, s5, 16
; SI-NEXT: .LBB97_5: ; %end
; SI-NEXT: buffer_load_dword v44, off, s[0:3], s32 offset:336 ; 4-byte Folded Reload
@@ -195382,45 +195639,57 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s16, s16, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s49, 16
+; SI-NEXT: v_readlane_b32 s22, v47, 43
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s20, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s21, s30, 16
+; SI-NEXT: s_lshl_b32 s21, s22, 16
+; SI-NEXT: v_readlane_b32 s23, v47, 44
; SI-NEXT: s_or_b32 s20, s20, s21
; SI-NEXT: s_and_b32 s21, s73, 0xffff
; SI-NEXT: s_lshl_b32 s22, s51, 16
+; SI-NEXT: v_readlane_b32 s24, v47, 45
; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s22, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s23, s94, 16
+; SI-NEXT: s_lshl_b32 s23, s24, 16
+; SI-NEXT: v_readlane_b32 s25, v47, 46
; SI-NEXT: s_or_b32 s22, s22, s23
; SI-NEXT: s_and_b32 s23, s63, 0xffff
; SI-NEXT: s_lshl_b32 s24, s53, 16
+; SI-NEXT: v_readlane_b32 s26, v47, 47
; SI-NEXT: s_or_b32 s23, s23, s24
; SI-NEXT: s_and_b32 s24, s60, 0xffff
-; SI-NEXT: s_lshl_b32 s25, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s25, s26, 16
+; SI-NEXT: v_readlane_b32 s27, v47, 48
; SI-NEXT: s_or_b32 s24, s24, s25
; SI-NEXT: s_and_b32 s25, s61, 0xffff
; SI-NEXT: s_lshl_b32 s26, s55, 16
+; SI-NEXT: v_readlane_b32 s28, v47, 49
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s58, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s70, 16
+; SI-NEXT: s_lshl_b32 s27, s28, 16
+; SI-NEXT: v_readlane_b32 s29, v47, 50
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s59, 0xffff
; SI-NEXT: s_lshl_b32 s28, s65, 16
+; SI-NEXT: v_readlane_b32 s58, v47, 51
; SI-NEXT: s_or_b32 s27, s27, s28
; SI-NEXT: s_and_b32 s28, s56, 0xffff
-; SI-NEXT: s_lshl_b32 s29, s80, 16
+; SI-NEXT: s_lshl_b32 s29, s58, 16
; SI-NEXT: s_or_b32 s28, s28, s29
; SI-NEXT: s_and_b32 s29, s57, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_lshl_b32 s56, s69, 16
; SI-NEXT: s_or_b32 s29, s29, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 53
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s84, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: v_readlane_b32 s57, v47, 54
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s47, s47, 0xffff
; SI-NEXT: s_lshl_b32 s56, s79, 16
; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: v_readlane_b32 s56, v47, 55
; SI-NEXT: s_and_b32 s44, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_or_b32 s44, s44, s56
; SI-NEXT: s_and_b32 s45, s45, 0xffff
; SI-NEXT: s_lshl_b32 s56, s89, 16
@@ -195458,7 +195727,7 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_lshl_b32 s56, s48, 16
; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s78, 16
+; SI-NEXT: s_lshl_b32 s17, s68, 16
; SI-NEXT: s_or_b32 s12, s12, s56
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s56, s31, 16
@@ -195474,6 +195743,8 @@ define inreg <64 x i16> @bitcast_v128i8_to_v64i16_scalar(<128 x i8> inreg %a, i3
; SI-NEXT: s_or_b32 s10, s10, s56
; SI-NEXT: s_or_b32 s5, s5, s17
; SI-NEXT: v_readlane_b32 s30, v45, 34
+; SI-NEXT: v_readlane_b32 s59, v47, 52
+; SI-NEXT: v_readlane_b32 s57, v47, 56
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s19
; SI-NEXT: v_mov_b32_e32 v2, s20
@@ -203545,7 +203816,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_or_b32 s8, s4, s5
; SI-NEXT: s_and_b32 s4, s94, 0xffff
; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: v_writelane_b32 v21, s18, 42
+; SI-NEXT: v_writelane_b32 v21, s18, 41
; SI-NEXT: s_lshr_b32 s18, s93, 8
; SI-NEXT: s_or_b32 s9, s4, s5
; SI-NEXT: s_and_b32 s4, s38, 0xffff
@@ -203583,7 +203854,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s96, s5, 8
; SI-NEXT: v_writelane_b32 v21, s96, 62
; SI-NEXT: s_bfe_u32 s18, s53, 0x80008
-; SI-NEXT: v_writelane_b32 v21, s18, 41
+; SI-NEXT: v_writelane_b32 v21, s18, 42
; SI-NEXT: s_bfe_u32 s18, s51, 0x80008
; SI-NEXT: v_writelane_b32 v21, s18, 43
; SI-NEXT: s_bfe_u32 s18, s49, 0x80008
@@ -203949,9 +204220,11 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB99_3: ; %Flow
; SI-NEXT: s_and_b64 s[96:97], s[96:97], exec
-; SI-NEXT: s_cselect_b32 s96, 1, 0
; SI-NEXT: v_writelane_b32 v21, s54, 37
+; SI-NEXT: s_cselect_b32 s96, 1, 0
+; SI-NEXT: v_writelane_b32 v21, s55, 38
; SI-NEXT: s_cmp_lg_u32 s96, 1
+; SI-NEXT: s_mov_b32 s54, s18
; SI-NEXT: s_mov_b32 s96, s72
; SI-NEXT: s_mov_b32 s72, s76
; SI-NEXT: s_mov_b32 s76, s78
@@ -203961,13 +204234,12 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_mov_b32 s68, s20
; SI-NEXT: s_mov_b32 s20, s70
; SI-NEXT: v_readlane_b32 s70, v22, 47
-; SI-NEXT: v_writelane_b32 v21, s55, 38
+; SI-NEXT: v_readlane_b32 s18, v22, 29
; SI-NEXT: v_readlane_b32 s71, v22, 48
-; SI-NEXT: s_mov_b32 s54, s18
+; SI-NEXT: v_readlane_b32 s19, v22, 30
; SI-NEXT: v_readlane_b32 s71, v21, 39
; SI-NEXT: v_readlane_b32 s97, v21, 40
-; SI-NEXT: v_readlane_b32 s18, v22, 29
-; SI-NEXT: v_readlane_b32 s19, v22, 30
+; SI-NEXT: v_readlane_b32 s19, v21, 41
; SI-NEXT: s_cbranch_scc1 .LBB99_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s50, s50, 3
@@ -204263,8 +204535,6 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v21, s16, 37
; SI-NEXT: v_writelane_b32 v21, s17, 38
; SI-NEXT: s_lshr_b32 s16, s65, 24
-; SI-NEXT: v_writelane_b32 v21, s16, 41
-; SI-NEXT: s_lshr_b32 s16, s65, 8
; SI-NEXT: v_writelane_b32 v21, s16, 42
; SI-NEXT: s_lshr_b32 s16, s93, 24
; SI-NEXT: v_writelane_b32 v21, s16, 43
@@ -204309,6 +204579,7 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s8, s8, 0x30000
; SI-NEXT: v_writelane_b32 v22, s16, 0
; SI-NEXT: s_lshr_b32 s16, s7, 8
+; SI-NEXT: s_lshr_b64 s[18:19], s[64:65], 24
; SI-NEXT: s_lshr_b64 s[70:71], s[58:59], 24
; SI-NEXT: s_lshr_b64 s[20:21], s[56:57], 8
; SI-NEXT: s_lshr_b64 vcc, s[8:9], 24
@@ -204321,8 +204592,8 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b64 s[96:97], s[4:5], 8
; SI-NEXT: v_writelane_b32 v21, s16, 60
; SI-NEXT: s_lshr_b32 s16, s5, 24
-; SI-NEXT: s_lshr_b64 s[18:19], s[64:65], 24
; SI-NEXT: s_lshr_b32 s53, s65, 16
+; SI-NEXT: s_lshr_b32 s19, s65, 8
; SI-NEXT: s_lshr_b32 s51, s93, 16
; SI-NEXT: s_lshr_b32 s49, s63, 16
; SI-NEXT: s_lshr_b32 s39, s59, 16
@@ -204362,13 +204633,12 @@ define inreg <128 x i8> @bitcast_v64i16_to_v128i8_scalar(<64 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s18, s17
; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: v_readlane_b32 s17, v21, 42
; SI-NEXT: v_mov_b32_e32 v1, s16
; SI-NEXT: s_and_b32 s16, s65, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_lshl_b32 s17, s19, 8
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s53, 0xff
-; SI-NEXT: v_readlane_b32 s18, v21, 41
+; SI-NEXT: v_readlane_b32 s18, v21, 42
; SI-NEXT: s_lshl_b32 s17, s17, 16
; SI-NEXT: s_lshl_b32 s18, s18, 24
; SI-NEXT: s_or_b32 s17, s18, s17
@@ -217509,22 +217779,22 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_writelane_b32 v63, s99, 33
; SI-NEXT: v_writelane_b32 v63, s30, 34
; SI-NEXT: v_writelane_b32 v63, s31, 35
-; SI-NEXT: s_lshr_b32 s6, s16, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
; SI-NEXT: ; implicit-def: $vgpr62 : SGPR spill to VGPR lane
-; SI-NEXT: s_lshr_b32 s7, s17, 16
-; SI-NEXT: v_writelane_b32 v62, s6, 0
-; SI-NEXT: s_lshr_b32 s8, s18, 16
-; SI-NEXT: v_writelane_b32 v62, s7, 1
-; SI-NEXT: s_lshr_b32 s9, s19, 16
-; SI-NEXT: v_writelane_b32 v62, s8, 2
-; SI-NEXT: s_lshr_b32 s10, s20, 16
-; SI-NEXT: v_writelane_b32 v62, s9, 3
-; SI-NEXT: s_lshr_b32 s11, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s10, 4
-; SI-NEXT: s_lshr_b32 s12, s22, 16
-; SI-NEXT: v_readfirstlane_b32 s49, v17
-; SI-NEXT: v_readfirstlane_b32 s37, v16
-; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s8, s17, 16
+; SI-NEXT: v_writelane_b32 v62, s7, 0
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: v_writelane_b32 v62, s8, 1
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: v_writelane_b32 v62, s9, 2
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: v_writelane_b32 v62, s10, 3
+; SI-NEXT: s_lshr_b32 s30, s21, 16
+; SI-NEXT: v_writelane_b32 v62, s11, 4
+; SI-NEXT: s_lshr_b32 s31, s22, 16
+; SI-NEXT: v_readfirstlane_b32 s4, v17
+; SI-NEXT: v_readfirstlane_b32 s12, v16
+; SI-NEXT: v_readfirstlane_b32 s70, v15
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v14
; SI-NEXT: v_readfirstlane_b32 s94, v13
; SI-NEXT: v_readfirstlane_b32 s92, v12
@@ -217540,17 +217810,17 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readfirstlane_b32 s56, v2
; SI-NEXT: v_readfirstlane_b32 s46, v1
; SI-NEXT: v_readfirstlane_b32 s44, v0
-; SI-NEXT: v_writelane_b32 v62, s11, 5
+; SI-NEXT: v_writelane_b32 v62, s30, 5
; SI-NEXT: s_lshr_b32 s43, s29, 16
; SI-NEXT: s_lshr_b32 s42, s28, 16
; SI-NEXT: s_lshr_b32 s41, s27, 16
; SI-NEXT: s_lshr_b32 s40, s26, 16
; SI-NEXT: s_lshr_b32 s15, s25, 16
; SI-NEXT: s_lshr_b32 s14, s24, 16
-; SI-NEXT: s_lshr_b32 s13, s23, 16
-; SI-NEXT: s_lshr_b32 s52, s49, 16
-; SI-NEXT: s_lshr_b32 s48, s37, 16
-; SI-NEXT: s_lshr_b32 s35, s4, 16
+; SI-NEXT: s_lshr_b32 s34, s23, 16
+; SI-NEXT: s_lshr_b32 s5, s4, 16
+; SI-NEXT: s_lshr_b32 s13, s12, 16
+; SI-NEXT: s_lshr_b32 s71, s70, 16
; SI-NEXT: s_lshr_b32 vcc_hi, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
; SI-NEXT: s_lshr_b32 s93, s92, 16
@@ -217566,161 +217836,171 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: s_lshr_b32 s57, s56, 16
; SI-NEXT: s_lshr_b32 s47, s46, 16
; SI-NEXT: s_lshr_b32 s45, s44, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_writelane_b32 v62, s12, 6
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_writelane_b32 v62, s13, 7
+; SI-NEXT: v_readfirstlane_b32 s6, v18
+; SI-NEXT: v_writelane_b32 v62, s31, 6
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v62, s34, 7
; SI-NEXT: s_cbranch_scc0 .LBB103_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshl_b32 s5, s16, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 33
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 32
-; SI-NEXT: s_lshl_b32 s5, s17, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 31
-; SI-NEXT: s_lshl_b32 s5, s7, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 30
-; SI-NEXT: s_lshl_b32 s5, s18, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 29
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 28
-; SI-NEXT: s_lshl_b32 s5, s19, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 27
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 26
-; SI-NEXT: s_lshl_b32 s5, s20, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 25
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 24
-; SI-NEXT: s_lshl_b32 s5, s21, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 23
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 22
-; SI-NEXT: s_lshl_b32 s5, s22, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 21
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 20
-; SI-NEXT: s_lshl_b32 s5, s23, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 19
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 18
-; SI-NEXT: s_lshl_b32 s5, s24, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 17
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 16
-; SI-NEXT: s_lshl_b32 s5, s25, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 15
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 14
-; SI-NEXT: s_lshl_b32 s5, s26, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 13
-; SI-NEXT: s_lshl_b32 s5, s40, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 12
-; SI-NEXT: s_lshl_b32 s5, s27, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 11
-; SI-NEXT: s_lshl_b32 s5, s41, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 10
-; SI-NEXT: s_lshl_b32 s5, s28, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 9
-; SI-NEXT: s_lshl_b32 s5, s42, 16
-; SI-NEXT: v_writelane_b32 v62, s5, 8
-; SI-NEXT: s_lshl_b32 s87, s29, 16
-; SI-NEXT: s_lshl_b32 s96, s43, 16
-; SI-NEXT: s_lshl_b32 s97, s44, 16
-; SI-NEXT: s_lshl_b32 s98, s45, 16
-; SI-NEXT: s_lshl_b32 s99, s46, 16
-; SI-NEXT: s_lshl_b32 s30, s47, 16
-; SI-NEXT: s_lshl_b32 s31, s56, 16
-; SI-NEXT: s_lshl_b32 s36, s57, 16
-; SI-NEXT: s_lshl_b32 s38, s58, 16
-; SI-NEXT: s_lshl_b32 s39, s59, 16
-; SI-NEXT: s_lshl_b32 s50, s60, 16
-; SI-NEXT: s_lshl_b32 s51, s61, 16
-; SI-NEXT: s_lshl_b32 s53, s62, 16
-; SI-NEXT: s_lshl_b32 s54, s63, 16
-; SI-NEXT: s_lshl_b32 s55, s72, 16
-; SI-NEXT: s_lshl_b32 s64, s73, 16
+; SI-NEXT: s_lshl_b32 s6, s16, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 33
+; SI-NEXT: s_lshl_b32 s6, s7, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 32
+; SI-NEXT: s_lshl_b32 s6, s17, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 31
+; SI-NEXT: s_lshl_b32 s6, s8, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 30
+; SI-NEXT: s_lshl_b32 s6, s18, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 29
+; SI-NEXT: s_lshl_b32 s6, s9, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 28
+; SI-NEXT: s_lshl_b32 s6, s19, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 27
+; SI-NEXT: s_lshl_b32 s6, s10, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 26
+; SI-NEXT: s_lshl_b32 s6, s20, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 25
+; SI-NEXT: s_lshl_b32 s6, s11, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 24
+; SI-NEXT: s_lshl_b32 s6, s21, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 23
+; SI-NEXT: s_lshl_b32 s6, s30, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 22
+; SI-NEXT: s_lshl_b32 s6, s22, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 21
+; SI-NEXT: s_lshl_b32 s6, s31, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 20
+; SI-NEXT: s_lshl_b32 s6, s23, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 19
+; SI-NEXT: s_lshl_b32 s6, s34, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 18
+; SI-NEXT: s_lshl_b32 s6, s24, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 17
+; SI-NEXT: s_lshl_b32 s6, s14, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 16
+; SI-NEXT: s_lshl_b32 s6, s25, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 15
+; SI-NEXT: s_lshl_b32 s6, s15, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 14
+; SI-NEXT: s_lshl_b32 s6, s26, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 13
+; SI-NEXT: s_lshl_b32 s6, s40, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 12
+; SI-NEXT: s_lshl_b32 s6, s27, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 11
+; SI-NEXT: s_lshl_b32 s6, s41, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 10
+; SI-NEXT: s_lshl_b32 s6, s28, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 9
+; SI-NEXT: s_lshl_b32 s6, s42, 16
+; SI-NEXT: v_writelane_b32 v62, s6, 8
+; SI-NEXT: s_lshl_b32 s82, s29, 16
+; SI-NEXT: s_lshl_b32 s83, s43, 16
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_lshl_b32 s85, s45, 16
+; SI-NEXT: s_lshl_b32 s86, s46, 16
+; SI-NEXT: s_lshl_b32 s87, s47, 16
+; SI-NEXT: s_lshl_b32 s96, s56, 16
+; SI-NEXT: s_lshl_b32 s98, s57, 16
+; SI-NEXT: s_lshl_b32 s99, s58, 16
+; SI-NEXT: s_lshl_b32 s31, s59, 16
+; SI-NEXT: s_lshl_b32 s34, s60, 16
+; SI-NEXT: s_lshl_b32 s36, s61, 16
+; SI-NEXT: s_lshl_b32 s37, s62, 16
+; SI-NEXT: s_lshl_b32 s39, s63, 16
+; SI-NEXT: s_lshl_b32 s48, s72, 16
+; SI-NEXT: s_lshl_b32 s50, s73, 16
; SI-NEXT: s_lshl_b32 s6, s74, 16
-; SI-NEXT: s_lshl_b32 s65, s75, 16
+; SI-NEXT: s_lshl_b32 s52, s75, 16
; SI-NEXT: s_lshl_b32 s7, s76, 16
-; SI-NEXT: s_lshl_b32 s66, s77, 16
-; SI-NEXT: s_lshl_b32 s67, s78, 16
-; SI-NEXT: s_lshl_b32 s68, s79, 16
+; SI-NEXT: s_lshl_b32 s53, s77, 16
+; SI-NEXT: s_lshl_b32 s54, s78, 16
+; SI-NEXT: s_lshl_b32 s55, s79, 16
; SI-NEXT: s_lshl_b32 s8, s88, 16
-; SI-NEXT: s_lshl_b32 s69, s89, 16
-; SI-NEXT: s_lshl_b32 s70, s90, 16
+; SI-NEXT: s_lshl_b32 s64, s89, 16
+; SI-NEXT: s_lshl_b32 s65, s90, 16
; SI-NEXT: s_lshl_b32 s9, s91, 16
-; SI-NEXT: s_lshl_b32 s71, s92, 16
-; SI-NEXT: s_lshl_b32 s80, s93, 16
+; SI-NEXT: s_lshl_b32 s66, s92, 16
+; SI-NEXT: s_lshl_b32 s67, s93, 16
; SI-NEXT: s_lshl_b32 s10, s94, 16
-; SI-NEXT: s_lshl_b32 s81, s95, 16
-; SI-NEXT: s_lshl_b32 s82, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s68, s95, 16
+; SI-NEXT: s_lshl_b32 s69, vcc_lo, 16
; SI-NEXT: s_lshl_b32 s11, vcc_hi, 16
-; SI-NEXT: s_mov_b32 s34, s4
-; SI-NEXT: s_lshl_b32 s83, s4, 16
-; SI-NEXT: s_lshl_b32 s84, s35, 16
-; SI-NEXT: s_lshl_b32 s12, s37, 16
-; SI-NEXT: s_lshl_b32 s85, s48, 16
-; SI-NEXT: s_lshl_b32 s13, s49, 16
-; SI-NEXT: s_lshl_b32 s86, s52, 16
+; SI-NEXT: s_mov_b32 s30, s70
+; SI-NEXT: s_lshl_b32 s70, s70, 16
+; SI-NEXT: s_mov_b32 s97, s71
+; SI-NEXT: s_lshl_b32 s71, s71, 16
+; SI-NEXT: s_mov_b32 s35, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_mov_b32 s38, s13
+; SI-NEXT: s_lshl_b32 s80, s13, 16
+; SI-NEXT: s_mov_b32 s49, s4
+; SI-NEXT: s_lshl_b32 s13, s4, 16
+; SI-NEXT: s_mov_b32 s51, s5
+; SI-NEXT: s_lshl_b32 s81, s5, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB103_3
; SI-NEXT: .LBB103_2:
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: s_mov_b32 s34, s4
+; SI-NEXT: s_mov_b32 s51, s5
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s49, s4
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s38, s13
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s35, s12
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s97, s71
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: s_mov_b32 s30, s70
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr82
+; SI-NEXT: ; implicit-def: $sgpr83
+; SI-NEXT: ; implicit-def: $sgpr84
+; SI-NEXT: ; implicit-def: $sgpr85
+; SI-NEXT: ; implicit-def: $sgpr86
; SI-NEXT: ; implicit-def: $sgpr87
; SI-NEXT: ; implicit-def: $sgpr96
-; SI-NEXT: ; implicit-def: $sgpr97
; SI-NEXT: ; implicit-def: $sgpr98
; SI-NEXT: ; implicit-def: $sgpr99
-; SI-NEXT: ; implicit-def: $sgpr30
; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr36
-; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr37
; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr54
; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr7
+; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr66
; SI-NEXT: ; implicit-def: $sgpr67
+; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr8
; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr11
; SI-NEXT: ; implicit-def: $sgpr70
-; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr81
-; SI-NEXT: ; implicit-def: $sgpr82
-; SI-NEXT: ; implicit-def: $sgpr11
-; SI-NEXT: ; implicit-def: $sgpr83
-; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr85
+; SI-NEXT: ; implicit-def: $sgpr80
; SI-NEXT: ; implicit-def: $sgpr13
-; SI-NEXT: ; implicit-def: $sgpr86
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
-; SI-NEXT: ; implicit-def: $sgpr6
-; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; kill: killed $sgpr6
; SI-NEXT: ; implicit-def: $sgpr6
@@ -217764,9 +218044,9 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB103_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s52
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s51
; SI-NEXT: v_cvt_f32_f16_e32 v1, s49
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s48
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s38
; SI-NEXT: v_readlane_b32 s4, v62, 7
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -217780,10 +218060,10 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s37
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s35
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s97
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -217792,7 +218072,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
; SI-NEXT: buffer_store_dword v1, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_cvt_f32_f16_e32 v1, vcc_hi
@@ -218124,7 +218404,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 25
; SI-NEXT: v_mov_b32_e32 v38, s4
; SI-NEXT: v_readlane_b32 s4, v62, 26
-; SI-NEXT: v_mov_b32_e32 v0, s86
+; SI-NEXT: v_mov_b32_e32 v0, s81
; SI-NEXT: v_mov_b32_e32 v48, s4
; SI-NEXT: v_readlane_b32 s4, v62, 27
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:60 ; 4-byte Folded Spill
@@ -218134,7 +218414,7 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 28
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:56 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s85
+; SI-NEXT: v_mov_b32_e32 v0, s80
; SI-NEXT: v_mov_b32_e32 v5, s4
; SI-NEXT: v_readlane_b32 s4, v62, 29
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:68 ; 4-byte Folded Spill
@@ -218144,12 +218424,12 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_readlane_b32 s4, v62, 30
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:64 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s84
+; SI-NEXT: v_mov_b32_e32 v0, s71
; SI-NEXT: v_mov_b32_e32 v3, s4
; SI-NEXT: v_readlane_b32 s4, v62, 31
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:76 ; 4-byte Folded Spill
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_mov_b32_e32 v0, s83
+; SI-NEXT: v_mov_b32_e32 v0, s70
; SI-NEXT: v_mov_b32_e32 v2, s4
; SI-NEXT: v_readlane_b32 s4, v62, 32
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:72 ; 4-byte Folded Spill
@@ -218158,37 +218438,37 @@ define inreg <64 x bfloat> @bitcast_v64f16_to_v64bf16_scalar(<64 x half> inreg %
; SI-NEXT: v_mov_b32_e32 v1, s4
; SI-NEXT: v_readlane_b32 s4, v62, 33
; SI-NEXT: buffer_store_dword v0, off, s[0:3], s32 offset:80 ; 4-byte Folded Spill
-; SI-NEXT: v_mov_b32_e32 v39, s82
-; SI-NEXT: v_mov_b32_e32 v51, s81
+; SI-NEXT: v_mov_b32_e32 v39, s69
+; SI-NEXT: v_mov_b32_e32 v51, s68
; SI-NEXT: v_mov_b32_e32 v49, s10
-; SI-NEXT: v_mov_b32_e32 v53, s80
-; SI-NEXT: v_mov_b32_e32 v52, s71
+; SI-NEXT: v_mov_b32_e32 v53, s67
+; SI-NEXT: v_mov_b32_e32 v52, s66
; SI-NEXT: v_mov_b32_e32 v55, s9
-; SI-NEXT: v_mov_b32_e32 v54, s70
-; SI-NEXT: v_mov_b32_e32 v41, s69
+; SI-NEXT: v_mov_b32_e32 v54, s65
+; SI-NEXT: v_mov_b32_e32 v41, s64
; SI-NEXT: v_mov_b32_e32 v40, s8
-; SI-NEXT: v_mov_b32_e32 v43, s68
-; SI-NEXT: v_mov_b32_e32 v42, s67
-; SI-NEXT: v_mov_b32_e32 v45, s66
+; SI-NEXT: v_mov_b32_e32 v43, s55
+; SI-NEXT: v_mov_b32_e32 v42, s54
+; SI-NEXT: v_mov_b32_e32 v45, s53
; SI-NEXT: v_mov_b32_e32 v44, s7
-; SI-NEXT: v_mov_b32_e32 v47, s65
+; SI-NEXT: v_mov_b32_e32 v47, s52
; SI-NEXT: v_mov_b32_e32 v46, s6
-; SI-NEXT: v_mov_b32_e32 v57, s64
-; SI-NEXT: v_mov_b32_e32 v56, s55
-; SI-NEXT: v_mov_b32_e32 v59, s54
-; SI-NEXT: v_mov_b32_e32 v58, s53
-; SI-NEXT: v_mov_b32_e32 v61, s51
-; SI-NEXT: v_mov_b32_e32 v60, s50
-; SI-NEXT: v_mov_b32_e32 v37, s39
-; SI-NEXT: v_mov_b32_e32 v36, s38
-; SI-NEXT: v_mov_b32_e32 v33, s36
-; SI-NEXT: v_mov_b32_e32 v32, s31
-; SI-NEXT: v_mov_b32_e32 v31, s30
-; SI-NEXT: v_mov_b32_e32 v30, s99
-; SI-NEXT: v_mov_b32_e32 v29, s98
-; SI-NEXT: v_mov_b32_e32 v28, s97
-; SI-NEXT: v_mov_b32_e32 v27, s96
-; SI-NEXT: v_mov_b32_e32 v26, s87
+; SI-NEXT: v_mov_b32_e32 v57, s50
+; SI-NEXT: v_mov_b32_e32 v56, s48
+; SI-NEXT: v_mov_b32_e32 v59, s39
+; SI-NEXT: v_mov_b32_e32 v58, s37
+; SI-NEXT: v_mov_b32_e32 v61, s36
+; SI-NEXT: v_mov_b32_e32 v60, s34
+; SI-NEXT: v_mov_b32_e32 v37, s31
+; SI-NEXT: v_mov_b32_e32 v36, s99
+; SI-NEXT: v_mov_b32_e32 v33, s98
+; SI-NEXT: v_mov_b32_e32 v32, s96
+; SI-NEXT: v_mov_b32_e32 v31, s87
+; SI-NEXT: v_mov_b32_e32 v30, s86
+; SI-NEXT: v_mov_b32_e32 v29, s85
+; SI-NEXT: v_mov_b32_e32 v28, s84
+; SI-NEXT: v_mov_b32_e32 v27, s83
+; SI-NEXT: v_mov_b32_e32 v26, s82
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_mov_b32_e32 v0, s4
; SI-NEXT: .LBB103_6: ; %end
@@ -227203,444 +227483,460 @@ define inreg <64 x bfloat> @bitcast_v64i16_to_v64bf16_scalar(<64 x i16> inreg %a
; SI-NEXT: v_writelane_b32 v33, s99, 33
; SI-NEXT: v_writelane_b32 v33, s30, 34
; SI-NEXT: v_writelane_b32 v33, s31, 35
-; SI-NEXT: s_lshr_b32 s4, s16, 16
+; SI-NEXT: s_lshr_b32 s78, s16, 16
; SI-NEXT: ; implicit-def: $vgpr34 : SGPR spill to VGPR lane
-; SI-NEXT: s_lshr_b32 s46, s17, 16
+; SI-NEXT: s_lshr_b32 s77, s17, 16
; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v34, s4, 0
-; SI-NEXT: s_lshr_b32 s47, s18, 16
-; SI-NEXT: v_writelane_b32 v34, s46, 1
-; SI-NEXT: s_lshr_b32 s57, s19, 16
-; SI-NEXT: v_writelane_b32 v34, s47, 2
-; SI-NEXT: s_lshr_b32 s59, s20, 16
-; SI-NEXT: v_writelane_b32 v34, s57, 3
-; SI-NEXT: s_lshr_b32 s61, s21, 16
-; SI-NEXT: v_writelane_b32 v34, s59, 4
-; SI-NEXT: s_lshr_b32 s63, s22, 16
-; SI-NEXT: v_writelane_b32 v34, s61, 5
-; SI-NEXT: s_lshr_b32 s73, s23, 16
-; SI-NEXT: v_writelane_b32 v34, s63, 6
-; SI-NEXT: s_lshr_b32 s75, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s73, 7
-; SI-NEXT: s_lshr_b32 s76, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s75, 8
-; SI-NEXT: s_lshr_b32 s77, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s76, 9
-; SI-NEXT: s_lshr_b32 s78, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s77, 10
-; SI-NEXT: s_lshr_b32 s79, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s78, 11
-; SI-NEXT: s_lshr_b32 s88, s29, 16
-; SI-NEXT: v_readfirstlane_b32 s54, v0
-; SI-NEXT: v_writelane_b32 v34, s79, 12
-; SI-NEXT: v_readfirstlane_b32 s64, v1
-; SI-NEXT: s_lshr_b32 s89, s54, 16
-; SI-NEXT: v_writelane_b32 v34, s88, 13
-; SI-NEXT: v_readfirstlane_b32 s66, v2
-; SI-NEXT: s_lshr_b32 s90, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s89, 14
-; SI-NEXT: v_readfirstlane_b32 s68, v3
-; SI-NEXT: s_lshr_b32 s91, s66, 16
-; SI-NEXT: v_writelane_b32 v34, s90, 15
-; SI-NEXT: v_readfirstlane_b32 s70, v4
-; SI-NEXT: s_lshr_b32 s92, s68, 16
-; SI-NEXT: v_writelane_b32 v34, s91, 16
-; SI-NEXT: v_readfirstlane_b32 s80, v5
-; SI-NEXT: s_lshr_b32 s93, s70, 16
-; SI-NEXT: v_writelane_b32 v34, s92, 17
-; SI-NEXT: v_readfirstlane_b32 s82, v6
-; SI-NEXT: s_lshr_b32 s94, s80, 16
-; SI-NEXT: v_writelane_b32 v34, s93, 18
-; SI-NEXT: v_readfirstlane_b32 s84, v7
-; SI-NEXT: s_lshr_b32 s95, s82, 16
-; SI-NEXT: v_writelane_b32 v34, s94, 19
-; SI-NEXT: v_readfirstlane_b32 s42, v17
-; SI-NEXT: v_readfirstlane_b32 s40, v16
-; SI-NEXT: v_readfirstlane_b32 s14, v15
-; SI-NEXT: v_readfirstlane_b32 s12, v14
-; SI-NEXT: v_readfirstlane_b32 s10, v13
-; SI-NEXT: v_readfirstlane_b32 s8, v12
-; SI-NEXT: v_readfirstlane_b32 s6, v11
-; SI-NEXT: v_readfirstlane_b32 s98, v10
-; SI-NEXT: v_readfirstlane_b32 s96, v9
-; SI-NEXT: v_readfirstlane_b32 s86, v8
-; SI-NEXT: s_lshr_b32 vcc_lo, s84, 16
-; SI-NEXT: v_writelane_b32 v34, s95, 20
-; SI-NEXT: s_lshr_b32 s41, s42, 16
-; SI-NEXT: s_lshr_b32 s15, s40, 16
-; SI-NEXT: s_lshr_b32 s13, s14, 16
-; SI-NEXT: s_lshr_b32 s11, s12, 16
-; SI-NEXT: s_lshr_b32 s9, s10, 16
-; SI-NEXT: s_lshr_b32 s7, s8, 16
-; SI-NEXT: s_lshr_b32 s99, s6, 16
-; SI-NEXT: s_lshr_b32 s97, s98, 16
-; SI-NEXT: s_lshr_b32 s30, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s86, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v18
-; SI-NEXT: v_writelane_b32 v34, vcc_lo, 21
-; SI-NEXT: s_cmp_lg_u32 s5, 0
-; SI-NEXT: v_writelane_b32 v34, vcc_hi, 22
-; SI-NEXT: v_writelane_b32 v34, s30, 23
+; SI-NEXT: v_writelane_b32 v34, s78, 0
+; SI-NEXT: s_lshr_b32 s76, s18, 16
+; SI-NEXT: v_writelane_b32 v34, s77, 1
+; SI-NEXT: s_lshr_b32 s74, s19, 16
+; SI-NEXT: v_writelane_b32 v34, s76, 2
+; SI-NEXT: s_lshr_b32 s73, s20, 16
+; SI-NEXT: v_writelane_b32 v34, s74, 3
+; SI-NEXT: s_lshr_b32 s63, s21, 16
+; SI-NEXT: v_writelane_b32 v34, s73, 4
+; SI-NEXT: s_lshr_b32 s7, s22, 16
+; SI-NEXT: v_writelane_b32 v34, s63, 5
+; SI-NEXT: s_lshr_b32 s75, s23, 16
+; SI-NEXT: v_writelane_b32 v34, s7, 6
+; SI-NEXT: s_lshr_b32 s62, s24, 16
+; SI-NEXT: v_writelane_b32 v34, s75, 7
+; SI-NEXT: s_lshr_b32 s61, s25, 16
+; SI-NEXT: v_writelane_b32 v34, s62, 8
+; SI-NEXT: s_lshr_b32 s72, s26, 16
+; SI-NEXT: v_writelane_b32 v34, s61, 9
+; SI-NEXT: s_lshr_b32 s53, s27, 16
+; SI-NEXT: v_writelane_b32 v34, s72, 10
+; SI-NEXT: s_lshr_b32 s49, s28, 16
+; SI-NEXT: v_writelane_b32 v34, s53, 11
+; SI-NEXT: s_lshr_b32 s56, s29, 16
+; SI-NEXT: v_readfirstlane_b32 s9, v0
+; SI-NEXT: v_writelane_b32 v34, s49, 12
+; SI-NEXT: s_lshr_b32 s96, s9, 16
+; SI-NEXT: v_writelane_b32 v34, s56, 13
+; SI-NEXT: v_readfirstlane_b32 s11, v1
+; SI-NEXT: v_writelane_b32 v34, s96, 14
+; SI-NEXT: s_lshr_b32 s87, s11, 16
+; SI-NEXT: v_writelane_b32 v34, s9, 15
+; SI-NEXT: v_readfirstlane_b32 s6, v2
+; SI-NEXT: v_writelane_b32 v34, s87, 16
+; SI-NEXT: s_lshr_b32 s84, s6, 16
+; SI-NEXT: v_writelane_b32 v34, s11, 17
+; SI-NEXT: v_readfirstlane_b32 s14, v3
+; SI-NEXT: v_writelane_b32 v34, s84, 18
+; SI-NEXT: s_lshr_b32 s83, s14, 16
+; SI-NEXT: v_writelane_b32 v34, s6, 19
+; SI-NEXT: v_writelane_b32 v34, s83, 20
+; SI-NEXT: v_readfirstlane_b32 s40, v4
+; SI-NEXT: v_writelane_b32 v34, s14, 21
+; SI-NEXT: v_readfirstlane_b32 s42, v5
+; SI-NEXT: v_writelane_b32 v34, s40, 22
+; SI-NEXT: v_readfirstlane_b32 s44, v6
+; SI-NEXT: v_writelane_b32 v34, s42, 23
+; SI-NEXT: v_readfirstlane_b32 s31, v7
+; SI-NEXT: v_writelane_b32 v34, s44, 24
+; SI-NEXT: v_readfirstlane_b32 s37, v8
+; SI-NEXT: v_writelane_b32 v34, s31, 25
+; SI-NEXT: v_readfirstlane_b32 s50, v9
+; SI-NEXT: v_writelane_b32 v34, s37, 26
+; SI-NEXT: v_readfirstlane_b32 s52, v11
+; SI-NEXT: v_readfirstlane_b32 s55, v10
+; SI-NEXT: v_writelane_b32 v34, s50, 27
+; SI-NEXT: v_readfirstlane_b32 s5, v17
+; SI-NEXT: v_readfirstlane_b32 s60, v16
+; SI-NEXT: v_readfirstlane_b32 s34, v15
+; SI-NEXT: v_readfirstlane_b32 s57, v14
+; SI-NEXT: v_readfirstlane_b32 s47, v13
+; SI-NEXT: v_readfirstlane_b32 s45, v12
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_writelane_b32 v34, s55, 28
+; SI-NEXT: s_lshr_b32 s4, s5, 16
+; SI-NEXT: s_lshr_b32 s95, s60, 16
+; SI-NEXT: s_lshr_b32 vcc_lo, s34, 16
+; SI-NEXT: s_lshr_b32 s35, s57, 16
+; SI-NEXT: s_lshr_b32 s38, s47, 16
+; SI-NEXT: s_lshr_b32 s39, s45, 16
+; SI-NEXT: s_lshr_b32 s64, s55, 16
+; SI-NEXT: s_lshr_b32 s67, s50, 16
+; SI-NEXT: s_lshr_b32 s68, s37, 16
+; SI-NEXT: s_lshr_b32 s70, s31, 16
+; SI-NEXT: s_lshr_b32 s71, s44, 16
+; SI-NEXT: s_lshr_b32 s80, s42, 16
+; SI-NEXT: s_lshr_b32 s81, s40, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v18
+; SI-NEXT: v_writelane_b32 v34, s51, 29
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v34, s52, 30
+; SI-NEXT: v_writelane_b32 v34, s45, 31
; SI-NEXT: s_cbranch_scc0 .LBB107_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_lshl_b32 s44, s4, 16
-; SI-NEXT: s_lshl_b32 s4, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 26
-; SI-NEXT: s_lshl_b32 s4, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 27
-; SI-NEXT: s_lshl_b32 s4, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 28
-; SI-NEXT: s_lshl_b32 s4, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 29
-; SI-NEXT: s_lshl_b32 s4, s28, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 25
-; SI-NEXT: s_lshl_b32 s4, s79, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 30
-; SI-NEXT: s_lshl_b32 s4, s29, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 24
-; SI-NEXT: s_lshl_b32 s4, s88, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 31
-; SI-NEXT: s_lshl_b32 s4, s89, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 32
-; SI-NEXT: s_lshl_b32 s4, s90, 16
-; SI-NEXT: s_lshl_b32 s43, s16, 16
-; SI-NEXT: s_lshl_b32 s45, s17, 16
-; SI-NEXT: s_lshl_b32 s46, s46, 16
-; SI-NEXT: s_lshl_b32 s56, s18, 16
-; SI-NEXT: s_lshl_b32 s47, s47, 16
-; SI-NEXT: s_lshl_b32 s58, s19, 16
-; SI-NEXT: s_lshl_b32 s57, s57, 16
-; SI-NEXT: s_lshl_b32 s60, s20, 16
-; SI-NEXT: s_lshl_b32 s59, s59, 16
-; SI-NEXT: s_lshl_b32 s62, s21, 16
-; SI-NEXT: s_lshl_b32 s61, s61, 16
-; SI-NEXT: s_lshl_b32 s72, s22, 16
-; SI-NEXT: s_lshl_b32 s63, s63, 16
-; SI-NEXT: s_lshl_b32 s74, s23, 16
-; SI-NEXT: s_lshl_b32 s73, s73, 16
+; SI-NEXT: s_lshl_b32 s8, s24, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 32
+; SI-NEXT: s_lshl_b32 s8, s25, 16
+; SI-NEXT: s_lshl_b32 s93, s16, 16
+; SI-NEXT: s_lshl_b32 s94, s78, 16
+; SI-NEXT: s_lshl_b32 s92, s17, 16
+; SI-NEXT: s_lshl_b32 s91, s77, 16
+; SI-NEXT: s_lshl_b32 s89, s18, 16
+; SI-NEXT: s_lshl_b32 s90, s76, 16
+; SI-NEXT: s_lshl_b32 s79, s19, 16
+; SI-NEXT: s_lshl_b32 s88, s74, 16
+; SI-NEXT: s_lshl_b32 s76, s20, 16
+; SI-NEXT: s_lshl_b32 s78, s73, 16
+; SI-NEXT: s_lshl_b32 s77, s21, 16
+; SI-NEXT: s_lshl_b32 s74, s63, 16
+; SI-NEXT: s_lshl_b32 s63, s22, 16
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_lshl_b32 s73, s23, 16
; SI-NEXT: s_lshl_b32 s75, s75, 16
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s77, 16
-; SI-NEXT: s_lshl_b32 s78, s78, 16
-; SI-NEXT: s_lshl_b32 s83, s54, 16
-; SI-NEXT: s_lshl_b32 s71, s64, 16
-; SI-NEXT: v_writelane_b32 v34, s4, 33
-; SI-NEXT: s_lshl_b32 s81, s66, 16
-; SI-NEXT: s_lshl_b32 s87, s91, 16
-; SI-NEXT: s_lshl_b32 s55, s68, 16
-; SI-NEXT: s_lshl_b32 s85, s92, 16
-; SI-NEXT: s_lshl_b32 s52, s70, 16
-; SI-NEXT: s_lshl_b32 s67, s93, 16
-; SI-NEXT: s_lshl_b32 s50, s80, 16
-; SI-NEXT: s_lshl_b32 s69, s94, 16
-; SI-NEXT: s_lshl_b32 s48, s82, 16
-; SI-NEXT: s_lshl_b32 s65, s95, 16
-; SI-NEXT: s_lshl_b32 s38, s84, 16
-; SI-NEXT: s_lshl_b32 s53, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 s36, s86, 16
-; SI-NEXT: s_lshl_b32 s51, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s34, s96, 16
-; SI-NEXT: s_lshl_b32 s49, s30, 16
-; SI-NEXT: s_lshl_b32 s30, s98, 16
-; SI-NEXT: s_lshl_b32 s39, s97, 16
-; SI-NEXT: s_lshl_b32 vcc_lo, s6, 16
-; SI-NEXT: s_lshl_b32 s37, s99, 16
-; SI-NEXT: s_lshl_b32 s94, s8, 16
-; SI-NEXT: s_lshl_b32 s35, s7, 16
-; SI-NEXT: s_lshl_b32 s92, s10, 16
-; SI-NEXT: s_lshl_b32 s31, s9, 16
-; SI-NEXT: s_lshl_b32 s95, s12, 16
-; SI-NEXT: s_lshl_b32 vcc_hi, s11, 16
-; SI-NEXT: s_lshl_b32 s89, s14, 16
-; SI-NEXT: s_lshl_b32 s93, s13, 16
-; SI-NEXT: s_lshl_b32 s90, s40, 16
-; SI-NEXT: s_lshl_b32 s91, s15, 16
-; SI-NEXT: s_lshl_b32 s79, s42, 16
-; SI-NEXT: s_lshl_b32 s88, s41, 16
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: v_writelane_b32 v34, s8, 33
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_lshl_b32 s46, s26, 16
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s66, s27, 16
+; SI-NEXT: s_lshl_b32 s99, s53, 16
+; SI-NEXT: s_lshl_b32 s54, s28, 16
+; SI-NEXT: s_lshl_b32 s69, s49, 16
+; SI-NEXT: s_lshl_b32 s36, s29, 16
+; SI-NEXT: s_lshl_b32 s65, s56, 16
+; SI-NEXT: s_lshl_b32 s43, s9, 16
+; SI-NEXT: s_lshl_b32 s48, s96, 16
+; SI-NEXT: s_lshl_b32 s15, s11, 16
+; SI-NEXT: s_lshl_b32 s30, s87, 16
+; SI-NEXT: s_lshl_b32 s13, s6, 16
+; SI-NEXT: s_lshl_b32 s41, s84, 16
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_lshl_b32 s6, s83, 16
+; SI-NEXT: s_lshl_b32 s11, s40, 16
+; SI-NEXT: s_mov_b32 s40, s81
+; SI-NEXT: s_lshl_b32 s12, s81, 16
+; SI-NEXT: s_lshl_b32 s96, s42, 16
+; SI-NEXT: s_mov_b32 s42, s80
+; SI-NEXT: s_lshl_b32 s10, s80, 16
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_mov_b32 s44, s71
+; SI-NEXT: s_lshl_b32 s9, s71, 16
+; SI-NEXT: s_lshl_b32 s83, s31, 16
+; SI-NEXT: s_mov_b32 s31, s70
+; SI-NEXT: s_lshl_b32 s8, s70, 16
+; SI-NEXT: s_lshl_b32 s80, s37, 16
+; SI-NEXT: s_mov_b32 s37, s68
+; SI-NEXT: s_lshl_b32 vcc_hi, s68, 16
+; SI-NEXT: s_lshl_b32 s71, s50, 16
+; SI-NEXT: s_mov_b32 s50, s67
+; SI-NEXT: s_lshl_b32 s87, s67, 16
+; SI-NEXT: s_lshl_b32 s68, s55, 16
+; SI-NEXT: s_mov_b32 s55, s64
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_lshl_b32 s67, s52, 16
+; SI-NEXT: s_lshl_b32 s81, s51, 16
+; SI-NEXT: s_lshl_b32 s64, s45, 16
+; SI-NEXT: s_mov_b32 s45, s39
+; SI-NEXT: s_lshl_b32 s70, s39, 16
+; SI-NEXT: s_mov_b32 s82, s47
+; SI-NEXT: s_lshl_b32 s52, s47, 16
+; SI-NEXT: s_mov_b32 s47, s38
+; SI-NEXT: s_lshl_b32 s53, s38, 16
+; SI-NEXT: s_mov_b32 s58, s57
+; SI-NEXT: s_lshl_b32 s49, s57, 16
+; SI-NEXT: s_mov_b32 s57, s35
+; SI-NEXT: s_lshl_b32 s51, s35, 16
+; SI-NEXT: s_mov_b32 s85, s34
+; SI-NEXT: s_lshl_b32 s38, s34, 16
+; SI-NEXT: s_mov_b32 s86, vcc_lo
+; SI-NEXT: s_lshl_b32 s39, vcc_lo, 16
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_lshl_b32 s34, s60, 16
+; SI-NEXT: s_mov_b32 s60, s95
+; SI-NEXT: s_lshl_b32 s35, s95, 16
+; SI-NEXT: s_mov_b32 s97, s5
+; SI-NEXT: s_lshl_b32 s95, s5, 16
+; SI-NEXT: s_mov_b32 s98, s4
+; SI-NEXT: s_lshl_b32 vcc_lo, s4, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB107_3
; SI-NEXT: .LBB107_2:
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: s_mov_b32 s97, s5
+; SI-NEXT: s_mov_b32 s98, s4
+; SI-NEXT: s_mov_b32 s59, s60
+; SI-NEXT: s_mov_b32 s60, s95
+; SI-NEXT: s_mov_b32 s85, s34
+; SI-NEXT: s_mov_b32 s86, vcc_lo
+; SI-NEXT: s_mov_b32 s58, s57
+; SI-NEXT: s_mov_b32 s57, s35
+; SI-NEXT: s_mov_b32 s82, s47
+; SI-NEXT: s_mov_b32 s47, s38
+; SI-NEXT: s_mov_b32 s45, s39
+; SI-NEXT: s_mov_b32 s55, s64
+; SI-NEXT: s_mov_b32 s50, s67
+; SI-NEXT: s_mov_b32 s37, s68
+; SI-NEXT: s_mov_b32 s31, s70
+; SI-NEXT: s_mov_b32 s44, s71
+; SI-NEXT: s_mov_b32 s42, s80
+; SI-NEXT: s_mov_b32 s40, s81
; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr93
+; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr92
+; SI-NEXT: ; implicit-def: $sgpr91
+; SI-NEXT: ; implicit-def: $sgpr89
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr75
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr77
; SI-NEXT: ; implicit-def: $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr47
-; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr57
-; SI-NEXT: ; implicit-def: $sgpr60
-; SI-NEXT: ; implicit-def: $sgpr59
-; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr61
-; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr77
; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr63
+; SI-NEXT: ; implicit-def: $sgpr7
; SI-NEXT: ; implicit-def: $sgpr73
-; SI-NEXT: ; kill: killed $sgpr75
; SI-NEXT: ; implicit-def: $sgpr75
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; kill: killed $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; kill: killed $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; kill: killed $sgpr78
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; kill: killed $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr61
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: ; implicit-def: $sgpr99
+; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr43
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr15
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr13
+; SI-NEXT: ; implicit-def: $sgpr41
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr96
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr84
+; SI-NEXT: ; implicit-def: $sgpr9
; SI-NEXT: ; implicit-def: $sgpr83
+; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr81
; SI-NEXT: ; implicit-def: $sgpr87
-; SI-NEXT: ; implicit-def: $sgpr55
-; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr56
; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr50
-; SI-NEXT: ; implicit-def: $sgpr69
-; SI-NEXT: ; implicit-def: $sgpr48
-; SI-NEXT: ; implicit-def: $sgpr65
-; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: ; implicit-def: $sgpr53
-; SI-NEXT: ; implicit-def: $sgpr36
-; SI-NEXT: ; implicit-def: $sgpr51
-; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr49
-; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr38
; SI-NEXT: ; implicit-def: $sgpr39
-; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr37
-; SI-NEXT: ; implicit-def: $sgpr94
+; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr35
-; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr31
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $vcc_hi
-; SI-NEXT: ; implicit-def: $sgpr89
-; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr90
-; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; kill: killed $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr79
+; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: .LBB107_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s4, s79
-; SI-NEXT: s_mov_b32 s5, s90
-; SI-NEXT: s_mov_b32 s79, s89
-; SI-NEXT: s_mov_b32 s89, s95
-; SI-NEXT: s_mov_b32 s90, s92
-; SI-NEXT: s_mov_b32 s92, s94
-; SI-NEXT: s_mov_b32 s94, vcc_lo
+; SI-NEXT: s_mov_b32 s4, s95
+; SI-NEXT: s_mov_b32 s5, vcc_lo
; SI-NEXT: s_mov_b32 s95, vcc_hi
-; SI-NEXT: s_mov_b32 vcc_lo, s30
-; SI-NEXT: s_mov_b32 vcc_hi, s31
-; SI-NEXT: s_mov_b32 s30, s34
-; SI-NEXT: s_mov_b32 s31, s35
-; SI-NEXT: s_mov_b32 s34, s36
-; SI-NEXT: s_mov_b32 s35, s37
-; SI-NEXT: s_mov_b32 s36, s38
-; SI-NEXT: s_mov_b32 s37, s39
-; SI-NEXT: s_mov_b32 s38, s48
-; SI-NEXT: s_mov_b32 s39, s49
-; SI-NEXT: s_mov_b32 s48, s50
-; SI-NEXT: s_mov_b32 s49, s51
-; SI-NEXT: s_mov_b32 s50, s52
-; SI-NEXT: s_mov_b32 s51, s53
-; SI-NEXT: s_mov_b32 s52, s55
-; SI-NEXT: s_mov_b32 s53, s65
-; SI-NEXT: s_mov_b32 s55, s81
-; SI-NEXT: s_mov_b32 s65, s69
-; SI-NEXT: s_mov_b32 s69, s85
-; SI-NEXT: v_readlane_b32 s81, v34, 24
-; SI-NEXT: v_readlane_b32 s85, v34, 25
+; SI-NEXT: s_mov_b32 vcc_lo, s8
+; SI-NEXT: s_mov_b32 s8, s9
+; SI-NEXT: s_mov_b32 s9, s11
+; SI-NEXT: s_mov_b32 vcc_hi, s46
; SI-NEXT: s_cbranch_scc1 .LBB107_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s8, s8, 3
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
+; SI-NEXT: s_add_i32 s4, s97, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s98, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s5, s59, 3
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s7, s60, 16
+; SI-NEXT: s_or_b32 s5, s7, s5
+; SI-NEXT: s_add_i32 s7, s85, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s86, 16
+; SI-NEXT: s_or_b32 s59, s56, s7
+; SI-NEXT: s_add_i32 s7, s58, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s57, 16
+; SI-NEXT: s_or_b32 s57, s56, s7
+; SI-NEXT: s_add_i32 s7, s82, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s7, s47, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 31
+; SI-NEXT: s_add_i32 s47, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s7, s45, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 30
+; SI-NEXT: s_add_i32 s45, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 29
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s6, 16
+; SI-NEXT: s_or_b32 s7, s46, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 28
+; SI-NEXT: s_add_i32 s46, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s7, s56, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 27
+; SI-NEXT: s_add_i32 s58, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s7, s56, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 26
+; SI-NEXT: s_add_i32 s56, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s37, 16
+; SI-NEXT: s_or_b32 s7, s60, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 25
+; SI-NEXT: s_add_i32 s60, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s31, 16
+; SI-NEXT: s_or_b32 s7, s61, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 24
+; SI-NEXT: s_add_i32 s95, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s7, s44, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 23
+; SI-NEXT: s_add_i32 s44, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s7, s42, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 22
+; SI-NEXT: s_add_i32 s42, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s7, s40, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 21
+; SI-NEXT: s_add_i32 s40, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 20
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s14, s6, 16
+; SI-NEXT: s_or_b32 s7, s14, s7
+; SI-NEXT: v_readlane_b32 s6, v34, 19
+; SI-NEXT: s_add_i32 s14, s7, 0x30000
+; SI-NEXT: s_add_i32 s7, s6, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 18
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s13, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s6, v34, 17
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: v_readlane_b32 s7, v34, 16
+; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
+; SI-NEXT: s_add_i32 s11, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s6, v34, 15
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_add_i32 s10, s10, 3
-; SI-NEXT: s_or_b32 s7, s7, s8
+; SI-NEXT: v_readlane_b32 s7, v34, 14
; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s99, 16
-; SI-NEXT: s_add_i32 s12, s12, 3
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s98, s98, 3
-; SI-NEXT: s_and_b32 s12, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_add_i32 s8, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s98, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s97, 16
-; SI-NEXT: s_add_i32 s14, s14, 3
-; SI-NEXT: s_or_b32 s11, s11, s12
-; SI-NEXT: s_or_b32 s6, s10, s6
-; SI-NEXT: s_add_i32 s96, s96, 3
-; SI-NEXT: v_readlane_b32 s12, v34, 23
-; SI-NEXT: s_add_i32 s42, s42, 3
-; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s13, 16
-; SI-NEXT: s_add_i32 s10, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_and_b32 s4, s42, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s41, 16
-; SI-NEXT: s_add_i32 s40, s40, 3
-; SI-NEXT: s_or_b32 s13, s13, s14
-; SI-NEXT: s_or_b32 s6, s12, s6
-; SI-NEXT: s_add_i32 s86, s86, 3
-; SI-NEXT: v_readlane_b32 s14, v34, 22
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_add_i32 s12, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_or_b32 s5, s15, s5
-; SI-NEXT: s_or_b32 s6, s14, s6
-; SI-NEXT: s_add_i32 s84, s84, 3
-; SI-NEXT: v_readlane_b32 s15, v34, 21
-; SI-NEXT: s_add_i32 s14, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s84, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s15, 16
-; SI-NEXT: s_or_b32 s6, s15, s6
-; SI-NEXT: s_add_i32 s82, s82, 3
-; SI-NEXT: v_readlane_b32 s40, v34, 20
-; SI-NEXT: s_add_i32 s15, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s82, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_or_b32 s6, s40, s6
-; SI-NEXT: s_add_i32 s80, s80, 3
-; SI-NEXT: v_readlane_b32 s41, v34, 19
-; SI-NEXT: s_add_i32 s40, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s80, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s41, 16
-; SI-NEXT: s_or_b32 s6, s41, s6
-; SI-NEXT: s_add_i32 s70, s70, 3
-; SI-NEXT: v_readlane_b32 s42, v34, 18
-; SI-NEXT: s_add_i32 s41, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s70, 0xffff
-; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_or_b32 s6, s42, s6
-; SI-NEXT: s_add_i32 s68, s68, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 17
-; SI-NEXT: s_add_i32 s42, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s66, s66, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 16
-; SI-NEXT: s_add_i32 s67, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s66, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s64, s64, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 15
-; SI-NEXT: s_add_i32 s65, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s64, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: v_readlane_b32 s43, v34, 14
-; SI-NEXT: s_add_i32 s88, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s54, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_or_b32 s6, s43, s6
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s90, s6, 0x30000
+; SI-NEXT: v_readlane_b32 s7, v34, 13
+; SI-NEXT: s_add_i32 s9, s6, 0x30000
; SI-NEXT: s_and_b32 s6, s29, 0xffff
-; SI-NEXT: v_readlane_b32 s29, v34, 13
-; SI-NEXT: s_lshl_b32 s29, s29, 16
-; SI-NEXT: s_or_b32 s6, s29, s6
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s6, s7, s6
+; SI-NEXT: s_add_i32 s8, s6, 0x30000
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s29, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s28, 0xffff
-; SI-NEXT: v_readlane_b32 s28, v34, 12
-; SI-NEXT: s_lshl_b32 s28, s28, 16
-; SI-NEXT: s_or_b32 s6, s28, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 12
+; SI-NEXT: s_and_b32 s7, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s10, s6, 0x30000
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_add_i32 s28, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s27, 0xffff
-; SI-NEXT: v_readlane_b32 s27, v34, 11
-; SI-NEXT: s_lshl_b32 s27, s27, 16
-; SI-NEXT: s_or_b32 s6, s27, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 11
+; SI-NEXT: s_and_b32 s7, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s12, s6, 0x30000
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_add_i32 s27, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s26, 0xffff
-; SI-NEXT: v_readlane_b32 s26, v34, 10
-; SI-NEXT: s_lshl_b32 s26, s26, 16
-; SI-NEXT: s_or_b32 s6, s26, s6
+; SI-NEXT: v_readlane_b32 s6, v34, 10
+; SI-NEXT: s_and_b32 s7, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s15, s6, 0x30000
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_add_i32 s26, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s25, 0xffff
-; SI-NEXT: v_readlane_b32 s25, v34, 9
-; SI-NEXT: s_lshl_b32 s25, s25, 16
-; SI-NEXT: s_or_b32 s6, s25, s6
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 9
+; SI-NEXT: s_and_b32 s7, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s25, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v34, 8
-; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: s_or_b32 s6, s24, s6
-; SI-NEXT: s_add_i32 s23, s23, 3
+; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 8
+; SI-NEXT: s_and_b32 s7, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s24, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s23, 0xffff
-; SI-NEXT: v_readlane_b32 s23, v34, 7
-; SI-NEXT: s_lshl_b32 s23, s23, 16
-; SI-NEXT: s_or_b32 s6, s23, s6
-; SI-NEXT: s_add_i32 s22, s22, 3
+; SI-NEXT: s_add_i32 s23, s23, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 7
+; SI-NEXT: s_and_b32 s7, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s23, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s22, 0xffff
-; SI-NEXT: v_readlane_b32 s22, v34, 6
-; SI-NEXT: s_lshl_b32 s22, s22, 16
-; SI-NEXT: s_or_b32 s6, s22, s6
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s22, s22, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 6
+; SI-NEXT: s_and_b32 s7, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: s_add_i32 s22, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s21, 0xffff
-; SI-NEXT: v_readlane_b32 s21, v34, 5
-; SI-NEXT: s_lshl_b32 s21, s21, 16
-; SI-NEXT: s_or_b32 s6, s21, s6
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 5
+; SI-NEXT: s_and_b32 s7, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s7, s6, 0x30000
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s21, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s20, 0xffff
-; SI-NEXT: v_readlane_b32 s20, v34, 4
-; SI-NEXT: s_lshl_b32 s20, s20, 16
-; SI-NEXT: s_or_b32 s6, s20, s6
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 4
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_add_i32 s20, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s19, 0xffff
-; SI-NEXT: v_readlane_b32 s19, v34, 3
-; SI-NEXT: s_lshl_b32 s19, s19, 16
-; SI-NEXT: s_or_b32 s6, s19, s6
-; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 3
+; SI-NEXT: s_and_b32 s19, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s19
; SI-NEXT: s_add_i32 s19, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s18, 0xffff
-; SI-NEXT: v_readlane_b32 s18, v34, 2
-; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s6, s18, s6
-; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 2
+; SI-NEXT: s_and_b32 s18, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s18
; SI-NEXT: s_add_i32 s18, s6, 0x30000
-; SI-NEXT: s_and_b32 s6, s17, 0xffff
-; SI-NEXT: v_readlane_b32 s17, v34, 1
-; SI-NEXT: s_lshl_b32 s17, s17, 16
-; SI-NEXT: s_or_b32 s6, s17, s6
+; SI-NEXT: s_add_i32 s17, s17, 3
+; SI-NEXT: v_readlane_b32 s6, v34, 1
+; SI-NEXT: s_and_b32 s17, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s6, s17
; SI-NEXT: s_add_i32 s17, s6, 0x30000
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: v_readlane_b32 s6, v34, 0
@@ -227648,218 +227944,204 @@ define inreg <64 x bfloat> @bitcast_v64i16_to_v64bf16_scalar(<64 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_or_b32 s6, s6, s16
; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_and_b32 s44, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s43, s6, 16
+; SI-NEXT: s_and_b32 s94, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s6, 16
; SI-NEXT: s_lshl_b32 s6, s24, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 26
-; SI-NEXT: s_lshl_b32 s6, s25, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 27
-; SI-NEXT: s_lshl_b32 s6, s26, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 28
-; SI-NEXT: s_lshl_b32 s6, s27, 16
-; SI-NEXT: v_writelane_b32 v34, s6, 29
-; SI-NEXT: s_and_b32 s6, s28, 0xffff0000
-; SI-NEXT: v_writelane_b32 v34, s6, 30
-; SI-NEXT: s_and_b32 s6, s29, 0xffff0000
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: v_writelane_b32 v34, s6, 31
-; SI-NEXT: s_and_b32 s6, s90, 0xffff0000
-; SI-NEXT: s_and_b32 s46, s17, 0xffff0000
-; SI-NEXT: s_lshl_b32 s45, s17, 16
-; SI-NEXT: s_and_b32 s47, s18, 0xffff0000
-; SI-NEXT: s_lshl_b32 s56, s18, 16
-; SI-NEXT: s_and_b32 s57, s19, 0xffff0000
-; SI-NEXT: s_lshl_b32 s58, s19, 16
-; SI-NEXT: s_and_b32 s59, s20, 0xffff0000
-; SI-NEXT: s_lshl_b32 s60, s20, 16
-; SI-NEXT: s_and_b32 s61, s21, 0xffff0000
-; SI-NEXT: s_lshl_b32 s62, s21, 16
-; SI-NEXT: s_and_b32 s63, s22, 0xffff0000
-; SI-NEXT: s_lshl_b32 s72, s22, 16
-; SI-NEXT: s_and_b32 s73, s23, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s23, 16
-; SI-NEXT: s_and_b32 s75, s24, 0xffff0000
-; SI-NEXT: s_and_b32 s76, s25, 0xffff0000
-; SI-NEXT: s_and_b32 s77, s26, 0xffff0000
-; SI-NEXT: s_and_b32 s78, s27, 0xffff0000
-; SI-NEXT: s_lshl_b32 s85, s28, 16
-; SI-NEXT: s_lshl_b32 s81, s29, 16
+; SI-NEXT: s_add_i32 s59, s59, 0x30000
+; SI-NEXT: s_add_i32 s57, s57, 0x30000
; SI-NEXT: v_writelane_b32 v34, s6, 32
-; SI-NEXT: s_lshl_b32 s83, s90, 16
-; SI-NEXT: s_and_b32 s6, s88, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s88, 16
-; SI-NEXT: s_and_b32 s87, s65, 0xffff0000
-; SI-NEXT: s_lshl_b32 s55, s65, 16
-; SI-NEXT: s_and_b32 s69, s67, 0xffff0000
-; SI-NEXT: s_lshl_b32 s52, s67, 16
-; SI-NEXT: s_and_b32 s67, s42, 0xffff0000
-; SI-NEXT: s_lshl_b32 s50, s42, 16
-; SI-NEXT: s_and_b32 s65, s41, 0xffff0000
-; SI-NEXT: s_lshl_b32 s48, s41, 16
-; SI-NEXT: s_and_b32 s53, s40, 0xffff0000
-; SI-NEXT: s_lshl_b32 s38, s40, 16
-; SI-NEXT: s_and_b32 s51, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s36, s15, 16
-; SI-NEXT: s_and_b32 s49, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s34, s14, 16
-; SI-NEXT: s_and_b32 s39, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s30, s12, 16
-; SI-NEXT: s_and_b32 s37, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s10, 16
-; SI-NEXT: s_and_b32 s35, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s94, s8, 16
-; SI-NEXT: s_and_b32 s31, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s92, s7, 16
-; SI-NEXT: s_and_b32 vcc_hi, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s9, 16
-; SI-NEXT: s_and_b32 s95, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s89, s11, 16
-; SI-NEXT: s_and_b32 s93, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s79, s13, 16
-; SI-NEXT: s_and_b32 s91, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_and_b32 s88, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_lshl_b32 s6, s25, 16
+; SI-NEXT: s_and_b32 s91, s17, 0xffff0000
+; SI-NEXT: s_lshl_b32 s92, s17, 16
+; SI-NEXT: s_and_b32 s90, s18, 0xffff0000
+; SI-NEXT: s_lshl_b32 s89, s18, 16
+; SI-NEXT: s_and_b32 s88, s19, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s19, 16
+; SI-NEXT: s_and_b32 s78, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s76, s20, 16
+; SI-NEXT: s_and_b32 s74, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s77, s7, 16
+; SI-NEXT: s_and_b32 s7, s22, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s22, 16
+; SI-NEXT: s_and_b32 s75, s23, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s23, 16
+; SI-NEXT: s_and_b32 s62, s24, 0xffff0000
+; SI-NEXT: s_and_b32 s61, s25, 0xffff0000
; SI-NEXT: v_writelane_b32 v34, s6, 33
+; SI-NEXT: s_and_b32 s72, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_hi, s15, 16
+; SI-NEXT: s_and_b32 s99, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s66, s12, 16
+; SI-NEXT: s_and_b32 s69, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s54, s10, 16
+; SI-NEXT: s_and_b32 s65, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s8, 16
+; SI-NEXT: s_and_b32 s48, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s43, s9, 16
+; SI-NEXT: s_and_b32 s30, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s11, 16
+; SI-NEXT: s_and_b32 s41, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_and_b32 s6, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_and_b32 s12, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s9, s40, 16
+; SI-NEXT: s_and_b32 s10, s42, 0xffff0000
+; SI-NEXT: s_lshl_b32 s96, s42, 16
+; SI-NEXT: s_and_b32 s8, s44, 0xffff0000
+; SI-NEXT: s_lshl_b32 s84, s44, 16
+; SI-NEXT: s_and_b32 vcc_lo, s95, 0xffff0000
+; SI-NEXT: s_lshl_b32 s83, s95, 16
+; SI-NEXT: s_and_b32 s95, s60, 0xffff0000
+; SI-NEXT: s_lshl_b32 s80, s60, 16
+; SI-NEXT: s_and_b32 s87, s56, 0xffff0000
+; SI-NEXT: s_lshl_b32 s71, s56, 16
+; SI-NEXT: s_and_b32 s56, s58, 0xffff0000
+; SI-NEXT: s_lshl_b32 s68, s58, 16
+; SI-NEXT: s_and_b32 s81, s46, 0xffff0000
+; SI-NEXT: s_lshl_b32 s67, s46, 16
+; SI-NEXT: s_and_b32 s70, s45, 0xffff0000
+; SI-NEXT: s_lshl_b32 s64, s45, 16
+; SI-NEXT: s_and_b32 s53, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s52, s47, 16
+; SI-NEXT: s_and_b32 s51, s57, 0xffff0000
+; SI-NEXT: s_lshl_b32 s49, s57, 16
+; SI-NEXT: s_and_b32 s39, s59, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s59, 16
+; SI-NEXT: s_and_b32 s35, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s5, 16
+; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: .LBB107_5: ; %end
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s44
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s43
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s93
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s46
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s91
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s92
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s90
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s56
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s89
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s57
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s58
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s79
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s59
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
-; SI-NEXT: v_mul_f32_e64 v4, 1.0, s60
+; SI-NEXT: v_mul_f32_e64 v4, 1.0, s76
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s61
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s74
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s62
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s77
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s63
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s7
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
-; SI-NEXT: v_mul_f32_e64 v6, 1.0, s72
+; SI-NEXT: v_mul_f32_e64 v6, 1.0, s63
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s75
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s73
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s75
-; SI-NEXT: v_readlane_b32 s6, v34, 26
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s62
+; SI-NEXT: v_readlane_b32 s7, v34, 32
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s7
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s76
-; SI-NEXT: v_readlane_b32 s6, v34, 27
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s61
+; SI-NEXT: v_readlane_b32 s7, v34, 33
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s7
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s77
-; SI-NEXT: v_readlane_b32 s6, v34, 28
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s72
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, vcc_hi
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s78
-; SI-NEXT: v_readlane_b32 s6, v34, 29
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s99
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s66
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 30
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s69
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s54
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 31
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s65
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s36
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 32
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s48
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s43
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_readlane_b32 s6, v34, 33
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s6
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s30
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s15
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s87
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v17, 16, v16
-; SI-NEXT: v_mul_f32_e64 v16, 1.0, s55
+; SI-NEXT: v_mul_f32_e64 v16, 1.0, s13
; SI-NEXT: v_lshr_b64 v[16:17], v[16:17], 16
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s6
; SI-NEXT: v_lshrrev_b32_e32 v18, 16, v17
-; SI-NEXT: v_mul_f32_e64 v17, 1.0, s52
+; SI-NEXT: v_mul_f32_e64 v17, 1.0, s14
; SI-NEXT: v_lshr_b64 v[17:18], v[17:18], 16
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v19, 16, v18
-; SI-NEXT: v_mul_f32_e64 v18, 1.0, s50
+; SI-NEXT: v_mul_f32_e64 v18, 1.0, s9
; SI-NEXT: v_lshr_b64 v[18:19], v[18:19], 16
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s65
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s10
; SI-NEXT: v_lshrrev_b32_e32 v20, 16, v19
-; SI-NEXT: v_mul_f32_e64 v19, 1.0, s48
+; SI-NEXT: v_mul_f32_e64 v19, 1.0, s96
; SI-NEXT: v_lshr_b64 v[19:20], v[19:20], 16
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s53
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s8
; SI-NEXT: v_lshrrev_b32_e32 v21, 16, v20
-; SI-NEXT: v_mul_f32_e64 v20, 1.0, s38
+; SI-NEXT: v_mul_f32_e64 v20, 1.0, s84
; SI-NEXT: v_lshr_b64 v[20:21], v[20:21], 16
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s51
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, vcc_lo
; SI-NEXT: v_lshrrev_b32_e32 v22, 16, v21
-; SI-NEXT: v_mul_f32_e64 v21, 1.0, s36
+; SI-NEXT: v_mul_f32_e64 v21, 1.0, s83
; SI-NEXT: v_lshr_b64 v[21:22], v[21:22], 16
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s49
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s95
; SI-NEXT: v_lshrrev_b32_e32 v23, 16, v22
-; SI-NEXT: v_mul_f32_e64 v22, 1.0, s34
+; SI-NEXT: v_mul_f32_e64 v22, 1.0, s80
; SI-NEXT: v_lshr_b64 v[22:23], v[22:23], 16
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s39
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s87
; SI-NEXT: v_lshrrev_b32_e32 v24, 16, v23
-; SI-NEXT: v_mul_f32_e64 v23, 1.0, s30
+; SI-NEXT: v_mul_f32_e64 v23, 1.0, s71
; SI-NEXT: v_lshr_b64 v[23:24], v[23:24], 16
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, s37
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s56
; SI-NEXT: v_lshrrev_b32_e32 v25, 16, v24
-; SI-NEXT: v_mul_f32_e64 v24, 1.0, vcc_lo
+; SI-NEXT: v_mul_f32_e64 v24, 1.0, s68
; SI-NEXT: v_lshr_b64 v[24:25], v[24:25], 16
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s35
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s81
; SI-NEXT: v_lshrrev_b32_e32 v26, 16, v25
-; SI-NEXT: v_mul_f32_e64 v25, 1.0, s94
+; SI-NEXT: v_mul_f32_e64 v25, 1.0, s67
; SI-NEXT: v_lshr_b64 v[25:26], v[25:26], 16
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s31
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s70
; SI-NEXT: v_lshrrev_b32_e32 v27, 16, v26
-; SI-NEXT: v_mul_f32_e64 v26, 1.0, s92
+; SI-NEXT: v_mul_f32_e64 v26, 1.0, s64
; SI-NEXT: v_lshr_b64 v[26:27], v[26:27], 16
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, vcc_hi
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s53
; SI-NEXT: v_lshrrev_b32_e32 v28, 16, v27
-; SI-NEXT: v_mul_f32_e64 v27, 1.0, s90
+; SI-NEXT: v_mul_f32_e64 v27, 1.0, s52
; SI-NEXT: v_lshr_b64 v[27:28], v[27:28], 16
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s95
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s51
; SI-NEXT: v_lshrrev_b32_e32 v29, 16, v28
-; SI-NEXT: v_mul_f32_e64 v28, 1.0, s89
+; SI-NEXT: v_mul_f32_e64 v28, 1.0, s49
; SI-NEXT: v_lshr_b64 v[28:29], v[28:29], 16
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s93
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v30, 16, v29
-; SI-NEXT: v_mul_f32_e64 v29, 1.0, s79
+; SI-NEXT: v_mul_f32_e64 v29, 1.0, s38
; SI-NEXT: v_lshr_b64 v[29:30], v[29:30], 16
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s91
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v31, 16, v30
-; SI-NEXT: v_mul_f32_e64 v30, 1.0, s5
+; SI-NEXT: v_mul_f32_e64 v30, 1.0, s34
; SI-NEXT: v_lshr_b64 v[30:31], v[30:31], 16
-; SI-NEXT: v_mul_f32_e64 v31, 1.0, s88
+; SI-NEXT: v_mul_f32_e64 v31, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v32, 16, v31
; SI-NEXT: v_mul_f32_e64 v31, 1.0, s4
; SI-NEXT: v_readlane_b32 s30, v33, 34
@@ -231583,24 +231865,24 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s99, 33
; SI-NEXT: v_writelane_b32 v32, s30, 34
; SI-NEXT: v_writelane_b32 v32, s31, 35
-; SI-NEXT: v_readfirstlane_b32 s68, v0
-; SI-NEXT: v_readfirstlane_b32 s69, v2
-; SI-NEXT: s_lshr_b32 s14, s68, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v0
+; SI-NEXT: v_readfirstlane_b32 s34, v2
+; SI-NEXT: s_lshr_b32 s14, s30, 16
; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s71, v4
-; SI-NEXT: s_lshr_b32 s12, s69, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v4
+; SI-NEXT: s_lshr_b32 s12, s34, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v33, s14, 0
-; SI-NEXT: v_readfirstlane_b32 s80, v6
-; SI-NEXT: s_lshr_b32 s10, s71, 16
+; SI-NEXT: v_readfirstlane_b32 s38, v6
+; SI-NEXT: s_lshr_b32 s10, s36, 16
; SI-NEXT: v_writelane_b32 v33, s12, 1
-; SI-NEXT: v_readfirstlane_b32 s82, v8
-; SI-NEXT: s_lshr_b32 s8, s80, 16
+; SI-NEXT: v_readfirstlane_b32 s48, v8
+; SI-NEXT: s_lshr_b32 s8, s38, 16
; SI-NEXT: v_writelane_b32 v33, s10, 2
-; SI-NEXT: v_readfirstlane_b32 s84, v10
-; SI-NEXT: s_lshr_b32 s6, s82, 16
+; SI-NEXT: v_readfirstlane_b32 s50, v10
+; SI-NEXT: s_lshr_b32 s6, s48, 16
; SI-NEXT: v_writelane_b32 v33, s8, 3
-; SI-NEXT: s_lshr_b32 s4, s84, 16
+; SI-NEXT: s_lshr_b32 s4, s50, 16
; SI-NEXT: v_writelane_b32 v33, s6, 4
; SI-NEXT: v_readfirstlane_b32 s15, v1
; SI-NEXT: v_writelane_b32 v33, s4, 5
@@ -231615,36 +231897,36 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: v_readfirstlane_b32 s5, v11
; SI-NEXT: v_writelane_b32 v33, s7, 10
; SI-NEXT: v_readfirstlane_b32 s75, v13
-; SI-NEXT: v_readfirstlane_b32 s86, v12
+; SI-NEXT: v_readfirstlane_b32 s87, v12
; SI-NEXT: v_writelane_b32 v33, s5, 11
-; SI-NEXT: s_lshr_b32 s74, s86, 16
+; SI-NEXT: s_lshr_b32 s74, s87, 16
; SI-NEXT: v_writelane_b32 v33, s75, 12
-; SI-NEXT: v_readfirstlane_b32 s72, v15
-; SI-NEXT: v_readfirstlane_b32 s96, v14
+; SI-NEXT: v_readfirstlane_b32 s73, v15
; SI-NEXT: v_writelane_b32 v33, s74, 13
-; SI-NEXT: s_lshr_b32 s73, s96, 16
-; SI-NEXT: v_writelane_b32 v33, s72, 14
+; SI-NEXT: v_readfirstlane_b32 s67, v14
+; SI-NEXT: v_writelane_b32 v33, s73, 14
+; SI-NEXT: s_lshr_b32 s72, s67, 16
+; SI-NEXT: v_writelane_b32 v33, s30, 15
; SI-NEXT: v_readfirstlane_b32 s62, v17
-; SI-NEXT: v_readfirstlane_b32 s98, v16
-; SI-NEXT: v_writelane_b32 v33, s73, 15
-; SI-NEXT: s_mov_b32 s76, s16
+; SI-NEXT: v_readfirstlane_b32 s66, v16
+; SI-NEXT: v_writelane_b32 v33, s72, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s99, s28, 16
+; SI-NEXT: s_lshr_b32 s82, s28, 16
; SI-NEXT: s_lshr_b32 s95, s27, 16
-; SI-NEXT: s_lshr_b32 s97, s26, 16
+; SI-NEXT: s_lshr_b32 s81, s26, 16
; SI-NEXT: s_lshr_b32 s93, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
+; SI-NEXT: s_lshr_b32 s80, s24, 16
; SI-NEXT: s_lshr_b32 s91, s23, 16
-; SI-NEXT: s_lshr_b32 s85, s22, 16
+; SI-NEXT: s_lshr_b32 s71, s22, 16
; SI-NEXT: s_lshr_b32 s89, s21, 16
-; SI-NEXT: s_lshr_b32 s83, s20, 16
+; SI-NEXT: s_lshr_b32 s70, s20, 16
; SI-NEXT: s_lshr_b32 s79, s19, 16
-; SI-NEXT: s_lshr_b32 s81, s18, 16
+; SI-NEXT: s_lshr_b32 s69, s18, 16
; SI-NEXT: s_lshr_b32 s77, s17, 16
-; SI-NEXT: s_lshr_b32 s70, s16, 16
+; SI-NEXT: s_lshr_b32 s68, s16, 16
; SI-NEXT: s_lshr_b32 s65, s62, 16
-; SI-NEXT: s_lshr_b32 s63, s98, 16
-; SI-NEXT: s_lshr_b32 s55, s72, 16
+; SI-NEXT: s_lshr_b32 s63, s66, 16
+; SI-NEXT: s_lshr_b32 s55, s73, 16
; SI-NEXT: s_lshr_b32 s53, s75, 16
; SI-NEXT: s_lshr_b32 s51, s5, 16
; SI-NEXT: s_lshr_b32 s49, s7, 16
@@ -231652,152 +231934,153 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s37, s11, 16
; SI-NEXT: s_lshr_b32 s35, s13, 16
; SI-NEXT: s_lshr_b32 s31, s15, 16
-; SI-NEXT: v_readfirstlane_b32 s16, v18
-; SI-NEXT: v_writelane_b32 v33, s62, 16
-; SI-NEXT: s_cmp_lg_u32 s16, 0
-; SI-NEXT: v_writelane_b32 v33, s17, 17
-; SI-NEXT: v_writelane_b32 v33, s76, 18
+; SI-NEXT: v_readfirstlane_b32 s40, v18
+; SI-NEXT: v_writelane_b32 v33, s62, 17
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s34, 18
+; SI-NEXT: v_writelane_b32 v33, s63, 19
; SI-NEXT: s_cbranch_scc0 .LBB111_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s16, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s77, 16
-; SI-NEXT: s_or_b32 s61, s16, s17
-; SI-NEXT: s_and_b32 s16, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s79, 16
-; SI-NEXT: s_or_b32 s59, s16, s17
-; SI-NEXT: s_and_b32 s16, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s89, 16
-; SI-NEXT: s_or_b32 s57, s16, s17
-; SI-NEXT: s_and_b32 s16, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s91, 16
-; SI-NEXT: s_or_b32 s47, s16, s17
-; SI-NEXT: s_and_b32 s16, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s93, 16
-; SI-NEXT: s_or_b32 s45, s16, s17
-; SI-NEXT: s_and_b32 s16, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s95, 16
-; SI-NEXT: s_or_b32 s43, s16, s17
-; SI-NEXT: s_and_b32 s16, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s17, vcc_hi, 16
-; SI-NEXT: s_or_b32 s41, s16, s17
-; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s31, 16
-; SI-NEXT: s_or_b32 s15, s15, s16
-; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s35, 16
-; SI-NEXT: s_or_b32 s13, s13, s16
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s37, 16
-; SI-NEXT: s_or_b32 s11, s11, s16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s39, 16
-; SI-NEXT: s_or_b32 s9, s9, s16
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s49, 16
-; SI-NEXT: s_or_b32 s7, s7, s16
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s51, 16
-; SI-NEXT: s_or_b32 s5, s5, s16
-; SI-NEXT: s_and_b32 s16, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s53, 16
-; SI-NEXT: s_mov_b32 s66, s53
-; SI-NEXT: s_or_b32 s53, s16, s17
-; SI-NEXT: s_and_b32 s16, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s55, 16
-; SI-NEXT: s_mov_b32 s67, s55
-; SI-NEXT: s_or_b32 s55, s16, s17
-; SI-NEXT: s_and_b32 s16, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s62, s65, 16
-; SI-NEXT: s_lshl_b32 s60, s70, 16
-; SI-NEXT: s_mov_b32 s17, s65
-; SI-NEXT: s_or_b32 s65, s16, s62
-; SI-NEXT: s_and_b32 s62, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s81, 16
+; SI-NEXT: s_and_b32 s41, s17, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s77, 16
+; SI-NEXT: s_or_b32 s61, s41, s43
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s79, 16
+; SI-NEXT: s_lshl_b32 s54, s72, 16
+; SI-NEXT: s_and_b32 s62, s62, 0xffff
+; SI-NEXT: s_lshl_b32 s72, s65, 16
+; SI-NEXT: s_lshl_b32 s60, s68, 16
+; SI-NEXT: s_or_b32 s59, s41, s43
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s89, 16
+; SI-NEXT: s_mov_b32 s97, s65
+; SI-NEXT: s_or_b32 s65, s62, s72
+; SI-NEXT: s_and_b32 s62, s16, 0xffff
+; SI-NEXT: s_or_b32 s57, s41, s43
+; SI-NEXT: s_and_b32 s41, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s91, 16
+; SI-NEXT: s_lshl_b32 s64, s63, 16
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s72, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[60:61], 16
-; SI-NEXT: s_and_b32 s60, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s83, 16
+; SI-NEXT: s_mov_b32 s63, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
+; SI-NEXT: s_or_b32 s47, s41, s43
+; SI-NEXT: s_and_b32 s41, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: v_writelane_b32 v33, s60, 20
+; SI-NEXT: s_lshl_b32 s58, s69, 16
+; SI-NEXT: s_or_b32 s45, s41, s43
+; SI-NEXT: s_and_b32 s41, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s95, 16
; SI-NEXT: s_lshl_b32 s52, s74, 16
+; SI-NEXT: s_and_b32 s73, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s74, s55, 16
+; SI-NEXT: v_writelane_b32 v33, s61, 21
+; SI-NEXT: s_and_b32 s60, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s70, 16
+; SI-NEXT: s_or_b32 s43, s41, s43
+; SI-NEXT: s_and_b32 s41, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s76, vcc_hi, 16
+; SI-NEXT: s_mov_b32 s96, s55
+; SI-NEXT: s_or_b32 s55, s73, s74
; SI-NEXT: s_or_b32 s74, s60, s58
-; SI-NEXT: s_mov_b32 s60, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[58:59], 16
+; SI-NEXT: s_lshr_b64 s[98:99], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s46, s85, 16
-; SI-NEXT: s_mov_b32 s77, s72
+; SI-NEXT: s_lshl_b32 s46, s71, 16
+; SI-NEXT: s_or_b32 s41, s41, s76
+; SI-NEXT: s_and_b32 s15, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s31, 16
; SI-NEXT: s_or_b32 s72, s58, s56
; SI-NEXT: s_mov_b32 s58, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[56:57], 16
; SI-NEXT: s_and_b32 s56, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s87, 16
-; SI-NEXT: s_mov_b32 s79, s60
+; SI-NEXT: s_lshl_b32 s44, s80, 16
+; SI-NEXT: s_or_b32 s15, s15, s76
+; SI-NEXT: s_and_b32 s13, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s35, 16
; SI-NEXT: s_or_b32 s60, s56, s46
; SI-NEXT: s_mov_b32 s56, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[46:47], 16
; SI-NEXT: s_and_b32 s46, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s42, s97, 16
+; SI-NEXT: s_lshl_b32 s42, s81, 16
+; SI-NEXT: s_or_b32 s13, s13, s76
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s37, 16
; SI-NEXT: s_mov_b32 s89, s58
; SI-NEXT: s_or_b32 s58, s46, s44
; SI-NEXT: s_mov_b32 s46, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[44:45], 16
; SI-NEXT: s_and_b32 s44, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s99, 16
+; SI-NEXT: s_lshl_b32 s40, s82, 16
+; SI-NEXT: s_or_b32 s11, s11, s76
+; SI-NEXT: s_and_b32 s9, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s39, 16
; SI-NEXT: s_mov_b32 s91, s56
; SI-NEXT: s_or_b32 s56, s44, s42
; SI-NEXT: s_mov_b32 s44, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[42:43], 16
; SI-NEXT: s_and_b32 s42, s28, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s9, s9, s76
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s49, 16
; SI-NEXT: s_mov_b32 s93, s46
; SI-NEXT: s_or_b32 s46, s42, s40
; SI-NEXT: s_mov_b32 s42, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s68, 0xffff
+; SI-NEXT: s_and_b32 s40, s30, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s7, s7, s76
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s51, 16
; SI-NEXT: s_mov_b32 s95, s44
; SI-NEXT: s_or_b32 s44, s40, s14
; SI-NEXT: s_mov_b32 s40, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s69, 0xffff
+; SI-NEXT: s_and_b32 s14, s34, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s5, s5, s76
+; SI-NEXT: s_and_b32 s75, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s76, s53, 16
; SI-NEXT: s_mov_b32 vcc_hi, s42
; SI-NEXT: s_or_b32 s42, s14, s12
; SI-NEXT: s_mov_b32 s14, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s71, 0xffff
+; SI-NEXT: s_and_b32 s12, s36, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_mov_b32 s86, s53
+; SI-NEXT: s_or_b32 s53, s75, s76
; SI-NEXT: s_mov_b32 s31, s40
+; SI-NEXT: s_mov_b32 s76, s36
; SI-NEXT: s_or_b32 s40, s12, s10
; SI-NEXT: s_mov_b32 s12, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[10:11], 16
-; SI-NEXT: s_and_b32 s10, s80, 0xffff
+; SI-NEXT: s_and_b32 s10, s38, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_mov_b32 s35, s14
+; SI-NEXT: s_mov_b32 s78, s38
; SI-NEXT: s_or_b32 s14, s10, s8
; SI-NEXT: s_mov_b32 s10, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[8:9], 16
-; SI-NEXT: s_and_b32 s8, s82, 0xffff
+; SI-NEXT: s_and_b32 s8, s48, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_mov_b32 s37, s12
+; SI-NEXT: s_mov_b32 s85, s48
; SI-NEXT: s_or_b32 s12, s8, s6
; SI-NEXT: s_mov_b32 s8, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[6:7], 16
-; SI-NEXT: s_and_b32 s6, s84, 0xffff
+; SI-NEXT: s_and_b32 s6, s50, 0xffff
; SI-NEXT: s_mov_b32 s39, s10
+; SI-NEXT: s_mov_b32 s84, s50
; SI-NEXT: s_or_b32 s10, s6, s4
; SI-NEXT: s_mov_b32 s6, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[4:5], 16
-; SI-NEXT: s_and_b32 s4, s86, 0xffff
-; SI-NEXT: s_lshl_b32 s54, s73, 16
+; SI-NEXT: s_and_b32 s4, s87, 0xffff
; SI-NEXT: s_mov_b32 s49, s8
; SI-NEXT: s_or_b32 s8, s4, s52
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s64, s63, 16
+; SI-NEXT: s_and_b32 s4, s67, 0xffff
; SI-NEXT: s_mov_b32 s51, s6
; SI-NEXT: s_or_b32 s6, s4, s54
-; SI-NEXT: s_and_b32 s4, s98, 0xffff
-; SI-NEXT: s_mov_b32 s16, s63
-; SI-NEXT: s_mov_b32 s63, s61
+; SI-NEXT: s_and_b32 s4, s66, 0xffff
; SI-NEXT: s_mov_b32 s75, s59
; SI-NEXT: s_mov_b32 s73, s57
; SI-NEXT: s_mov_b32 s61, s47
@@ -231817,18 +232100,28 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s4, s4, s64
; SI-NEXT: s_mov_b32 s5, s65
; SI-NEXT: s_lshr_b64 s[64:65], s[64:65], 16
-; SI-NEXT: s_mov_b32 s53, s66
-; SI-NEXT: s_mov_b32 s55, s67
-; SI-NEXT: s_mov_b32 s65, s17
+; SI-NEXT: s_mov_b32 s53, s86
+; SI-NEXT: s_mov_b32 s83, s67
+; SI-NEXT: s_mov_b32 s55, s96
+; SI-NEXT: s_mov_b32 s96, s98
+; SI-NEXT: s_mov_b32 s86, s66
+; SI-NEXT: s_mov_b32 s65, s97
; SI-NEXT: s_mov_b64 s[66:67], 0
; SI-NEXT: s_branch .LBB111_3
; SI-NEXT: .LBB111_2:
-; SI-NEXT: s_mov_b32 s16, s63
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s86, s66
+; SI-NEXT: s_mov_b32 s83, s67
+; SI-NEXT: s_mov_b32 s84, s50
+; SI-NEXT: s_mov_b32 s85, s48
+; SI-NEXT: s_mov_b32 s78, s38
+; SI-NEXT: s_mov_b32 s76, s36
; SI-NEXT: s_mov_b64 s[66:67], -1
+; SI-NEXT: v_writelane_b32 v33, s4, 20
; SI-NEXT: ; implicit-def: $sgpr62
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: v_writelane_b32 v33, s5, 21
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr96
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr60
@@ -231859,22 +232152,25 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: .LBB111_3: ; %Flow
; SI-NEXT: s_and_b64 s[66:67], s[66:67], exec
-; SI-NEXT: s_cselect_b32 s17, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s17, 1
+; SI-NEXT: s_cselect_b32 s66, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s66, 1
+; SI-NEXT: v_readlane_b32 s66, v33, 20
+; SI-NEXT: v_readlane_b32 s67, v33, 21
; SI-NEXT: s_cbranch_scc1 .LBB111_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s98, s98, 3
-; SI-NEXT: s_and_b32 s4, s98, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s16, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: v_readlane_b32 s5, v33, 16
+; SI-NEXT: s_add_i32 s4, s86, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v33, 19
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
+; SI-NEXT: v_readlane_b32 s5, v33, 17
; SI-NEXT: s_add_i32 s5, s5, 3
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s6, s65, 16
; SI-NEXT: s_or_b32 s5, s6, s5
-; SI-NEXT: s_add_i32 s96, s96, 3
-; SI-NEXT: v_readlane_b32 s6, v33, 15
-; SI-NEXT: s_and_b32 s7, s96, 0xffff
+; SI-NEXT: s_add_i32 s6, s83, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v33, 16
; SI-NEXT: s_lshl_b32 s6, s6, 16
; SI-NEXT: s_or_b32 s6, s6, s7
; SI-NEXT: v_readlane_b32 s7, v33, 14
@@ -231882,9 +232178,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s8, s55, 16
; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_add_i32 s86, s86, 3
+; SI-NEXT: s_add_i32 s8, s87, 3
+; SI-NEXT: s_and_b32 s9, s8, 0xffff
; SI-NEXT: v_readlane_b32 s8, v33, 13
-; SI-NEXT: s_and_b32 s9, s86, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
; SI-NEXT: s_or_b32 s8, s8, s9
; SI-NEXT: v_readlane_b32 s9, v33, 12
@@ -231892,9 +232188,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s10, s53, 16
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_add_i32 s84, s84, 3
+; SI-NEXT: s_add_i32 s10, s84, 3
+; SI-NEXT: s_and_b32 s11, s10, 0xffff
; SI-NEXT: v_readlane_b32 s10, v33, 5
-; SI-NEXT: s_and_b32 s11, s84, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_or_b32 s10, s10, s11
; SI-NEXT: v_readlane_b32 s11, v33, 11
@@ -231902,9 +232198,9 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s12, s51, 16
; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_add_i32 s82, s82, 3
+; SI-NEXT: s_add_i32 s12, s85, 3
+; SI-NEXT: s_and_b32 s13, s12, 0xffff
; SI-NEXT: v_readlane_b32 s12, v33, 4
-; SI-NEXT: s_and_b32 s13, s82, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_or_b32 s12, s12, s13
; SI-NEXT: v_readlane_b32 s13, v33, 10
@@ -231912,123 +232208,116 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s14, s49, 16
; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_add_i32 s80, s80, 3
+; SI-NEXT: s_add_i32 s14, s78, 3
+; SI-NEXT: s_and_b32 s15, s14, 0xffff
; SI-NEXT: v_readlane_b32 s14, v33, 3
-; SI-NEXT: s_and_b32 s15, s80, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_or_b32 s14, s14, s15
; SI-NEXT: v_readlane_b32 s15, v33, 9
; SI-NEXT: s_add_i32 s15, s15, 3
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s39, 16
-; SI-NEXT: s_or_b32 s15, s16, s15
-; SI-NEXT: s_add_i32 s71, s71, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 2
-; SI-NEXT: s_and_b32 s17, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s40, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 8
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s37, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s41, s16, 0x30000
-; SI-NEXT: s_add_i32 s69, s69, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 1
-; SI-NEXT: s_and_b32 s17, s69, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s42, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 7
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s35, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s43, s16, 0x30000
-; SI-NEXT: s_add_i32 s68, s68, 3
-; SI-NEXT: v_readlane_b32 s16, v33, 0
-; SI-NEXT: s_and_b32 s17, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s16, 16
-; SI-NEXT: s_or_b32 s16, s16, s17
-; SI-NEXT: s_add_i32 s44, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 6
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s31, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_lshl_b32 s40, s39, 16
+; SI-NEXT: s_or_b32 s15, s40, s15
+; SI-NEXT: s_add_i32 s40, s76, 3
+; SI-NEXT: s_and_b32 s41, s40, 0xffff
+; SI-NEXT: v_readlane_b32 s40, v33, 2
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: v_readlane_b32 s41, v33, 8
+; SI-NEXT: s_add_i32 s41, s41, 3
+; SI-NEXT: s_and_b32 s41, s41, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s37, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: v_readlane_b32 s42, v33, 18
+; SI-NEXT: s_add_i32 s42, s42, 3
+; SI-NEXT: s_and_b32 s43, s42, 0xffff
+; SI-NEXT: v_readlane_b32 s42, v33, 1
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: v_readlane_b32 s43, v33, 7
+; SI-NEXT: s_add_i32 s43, s43, 3
+; SI-NEXT: s_and_b32 s43, s43, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s35, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: v_readlane_b32 s44, v33, 15
+; SI-NEXT: s_add_i32 s44, s44, 3
+; SI-NEXT: s_and_b32 s45, s44, 0xffff
+; SI-NEXT: v_readlane_b32 s44, v33, 0
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: v_readlane_b32 s45, v33, 6
+; SI-NEXT: s_add_i32 s45, s45, 3
+; SI-NEXT: s_and_b32 s45, s45, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s31, 16
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s45, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s99, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s28, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s82, 16
+; SI-NEXT: s_or_b32 s28, s46, s28
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_add_i32 s46, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s17, vcc_hi, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s46, s28, 0x30000
+; SI-NEXT: s_and_b32 s28, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s29, vcc_hi, 16
+; SI-NEXT: s_or_b32 s28, s29, s28
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_add_i32 s47, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s97, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s47, s28, 0x30000
+; SI-NEXT: s_and_b32 s26, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s28, s81, 16
+; SI-NEXT: s_or_b32 s26, s28, s26
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_add_i32 s56, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s95, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s56, s26, 0x30000
+; SI-NEXT: s_and_b32 s26, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s27, s95, 16
+; SI-NEXT: s_or_b32 s26, s27, s26
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_add_i32 s57, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s87, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s57, s26, 0x30000
+; SI-NEXT: s_and_b32 s24, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s26, s80, 16
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_add_i32 s58, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s93, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s58, s24, 0x30000
+; SI-NEXT: s_and_b32 s24, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s25, s93, 16
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_add_i32 s59, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s85, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s59, s24, 0x30000
+; SI-NEXT: s_and_b32 s22, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s24, s71, 16
+; SI-NEXT: s_or_b32 s22, s24, s22
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_add_i32 s60, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s91, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s60, s22, 0x30000
+; SI-NEXT: s_and_b32 s22, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s23, s91, 16
+; SI-NEXT: s_or_b32 s22, s23, s22
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s61, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s83, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s61, s22, 0x30000
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s22, s70, 16
+; SI-NEXT: s_or_b32 s20, s22, s20
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_add_i32 s72, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s89, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s72, s20, 0x30000
+; SI-NEXT: s_and_b32 s20, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s21, s89, 16
+; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_add_i32 s73, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s81, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_add_i32 s73, s20, 0x30000
+; SI-NEXT: s_and_b32 s18, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_or_b32 s18, s20, s18
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_add_i32 s74, s16, 0x30000
-; SI-NEXT: s_and_b32 s16, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s79, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s75, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 18
+; SI-NEXT: s_add_i32 s74, s18, 0x30000
+; SI-NEXT: s_and_b32 s18, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s19, s79, 16
+; SI-NEXT: s_or_b32 s18, s19, s18
; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_add_i32 s75, s18, 0x30000
; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s70, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_lshl_b32 s18, s68, 16
+; SI-NEXT: s_or_b32 s16, s18, s16
+; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s62, s16, 0x30000
-; SI-NEXT: v_readlane_b32 s16, v33, 17
-; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_and_b32 s16, s16, 0xffff
+; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s77, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
@@ -232041,9 +232330,14 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s13, s13, 0x30000
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
+; SI-NEXT: s_add_i32 s40, s40, 0x30000
+; SI-NEXT: s_add_i32 s41, s41, 0x30000
+; SI-NEXT: s_add_i32 s42, s42, 0x30000
+; SI-NEXT: s_add_i32 s43, s43, 0x30000
+; SI-NEXT: s_add_i32 s44, s44, 0x30000
+; SI-NEXT: s_add_i32 s45, s45, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s63, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[76:77], s[62:63], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[74:75], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[72:73], 16
; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
@@ -232058,6 +232352,8 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[52:53], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[54:55], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[64:65], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[66:67], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[96:97], s[74:75], 16
; SI-NEXT: s_lshr_b32 s77, s63, 16
; SI-NEXT: s_lshr_b32 s79, s75, 16
; SI-NEXT: s_lshr_b32 s89, s73, 16
@@ -232076,13 +232372,13 @@ define inreg <64 x half> @bitcast_v64i16_to_v64f16_scalar(<64 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s65, s5, 16
; SI-NEXT: .LBB111_5: ; %end
; SI-NEXT: s_and_b32 s16, s62, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s76, 16
+; SI-NEXT: s_lshl_b32 s17, s66, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s63, 0xffff
; SI-NEXT: s_lshl_b32 s18, s77, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s78, 16
+; SI-NEXT: s_lshl_b32 s19, s96, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s75, 0xffff
; SI-NEXT: s_lshl_b32 s20, s79, 16
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
index f79e89b17eee4..4c88e35efc374 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
@@ -60648,18 +60648,17 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v17, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v18, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v17, s34, 0
; SI-NEXT: v_writelane_b32 v17, s35, 1
-; SI-NEXT: v_writelane_b32 v17, s36, 2
-; SI-NEXT: v_writelane_b32 v17, s37, 3
-; SI-NEXT: v_writelane_b32 v17, s30, 4
-; SI-NEXT: v_writelane_b32 v17, s31, 5
-; SI-NEXT: v_readfirstlane_b32 s36, v1
-; SI-NEXT: v_readfirstlane_b32 s34, v0
-; SI-NEXT: s_lshr_b32 s30, s29, 16
-; SI-NEXT: s_lshr_b32 s4, s28, 16
+; SI-NEXT: v_writelane_b32 v17, s30, 2
+; SI-NEXT: v_writelane_b32 v17, s31, 3
+; SI-NEXT: v_readfirstlane_b32 s10, v1
+; SI-NEXT: v_readfirstlane_b32 s8, v0
+; SI-NEXT: s_lshr_b32 s4, s29, 16
+; SI-NEXT: s_lshr_b32 s5, s28, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s27, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s26, 16
; SI-NEXT: s_lshr_b32 s95, s25, 16
@@ -60670,18 +60669,22 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshr_b32 s90, s20, 16
; SI-NEXT: s_lshr_b32 s89, s19, 16
; SI-NEXT: s_lshr_b32 s88, s18, 16
-; SI-NEXT: s_lshr_b32 s79, s17, 16
-; SI-NEXT: s_lshr_b32 s37, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s36, 16
-; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v2
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s6, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s10, 16
+; SI-NEXT: s_lshr_b32 s12, s8, 16
+; SI-NEXT: v_readfirstlane_b32 s9, v2
+; SI-NEXT: ; implicit-def: $vgpr18 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v18, s7, 0
+; SI-NEXT: v_writelane_b32 v18, s6, 1
; SI-NEXT: s_cbranch_scc0 .LBB93_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_lshl_b32 s74, s16, 16
-; SI-NEXT: s_lshl_b32 s78, s37, 16
+; SI-NEXT: s_lshl_b32 s78, s7, 16
; SI-NEXT: s_lshl_b32 s72, s17, 16
-; SI-NEXT: s_lshl_b32 s77, s79, 16
+; SI-NEXT: s_lshl_b32 s77, s6, 16
; SI-NEXT: s_lshl_b32 s62, s18, 16
; SI-NEXT: s_lshl_b32 s76, s88, 16
; SI-NEXT: s_lshl_b32 s60, s19, 16
@@ -60696,25 +60699,35 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s59, s93, 16
; SI-NEXT: s_lshl_b32 s42, s24, 16
; SI-NEXT: s_lshl_b32 s57, s94, 16
-; SI-NEXT: s_lshl_b32 s43, s25, 16
+; SI-NEXT: s_lshl_b32 s45, s25, 16
; SI-NEXT: s_lshl_b32 s47, s95, 16
-; SI-NEXT: s_lshl_b32 s14, s26, 16
-; SI-NEXT: s_lshl_b32 s45, vcc_lo, 16
-; SI-NEXT: s_lshl_b32 s12, s27, 16
-; SI-NEXT: s_lshl_b32 s40, vcc_hi, 16
-; SI-NEXT: s_lshl_b32 s15, s28, 16
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_lshl_b32 s41, s4, 16
-; SI-NEXT: s_lshl_b32 s9, s29, 16
-; SI-NEXT: s_lshl_b32 s13, s30, 16
-; SI-NEXT: s_lshl_b32 s10, s34, 16
-; SI-NEXT: s_lshl_b32 s11, s31, 16
-; SI-NEXT: s_lshl_b32 s7, s36, 16
-; SI-NEXT: s_lshl_b32 s8, s35, 16
+; SI-NEXT: s_lshl_b32 s15, s26, 16
+; SI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s13, s27, 16
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_lshl_b32 s11, s28, 16
+; SI-NEXT: s_mov_b32 s6, s5
+; SI-NEXT: s_lshl_b32 s40, s5, 16
+; SI-NEXT: s_lshl_b32 s79, s29, 16
+; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_lshl_b32 s14, s4, 16
+; SI-NEXT: s_mov_b32 s9, s8
+; SI-NEXT: s_lshl_b32 s34, s8, 16
+; SI-NEXT: s_mov_b32 s8, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_mov_b32 s31, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_mov_b32 s30, s35
+; SI-NEXT: s_lshl_b32 s35, s35, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB93_3
; SI-NEXT: .LBB93_2:
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_mov_b32 s31, s10
+; SI-NEXT: s_mov_b32 s30, s35
+; SI-NEXT: s_mov_b32 s9, s8
+; SI-NEXT: s_mov_b32 s8, s12
+; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s5
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr78
@@ -60734,116 +60747,117 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: ; implicit-def: $sgpr59
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr57
-; SI-NEXT: ; implicit-def: $sgpr43
-; SI-NEXT: ; implicit-def: $sgpr47
-; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr45
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr47
; SI-NEXT: ; implicit-def: $sgpr15
-; SI-NEXT: ; implicit-def: $sgpr41
-; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr43
; SI-NEXT: ; implicit-def: $sgpr13
-; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr41
; SI-NEXT: ; implicit-def: $sgpr11
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr40
+; SI-NEXT: ; implicit-def: $sgpr79
+; SI-NEXT: ; implicit-def: $sgpr14
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr12
+; SI-NEXT: ; implicit-def: $sgpr10
+; SI-NEXT: ; implicit-def: $sgpr35
; SI-NEXT: .LBB93_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s4, s7
-; SI-NEXT: s_mov_b32 s5, s10
-; SI-NEXT: s_mov_b32 s7, s9
-; SI-NEXT: s_mov_b32 s9, s15
-; SI-NEXT: s_mov_b32 s10, s12
-; SI-NEXT: s_mov_b32 s12, s14
-; SI-NEXT: s_mov_b32 s14, s43
-; SI-NEXT: s_mov_b32 s15, s41
+; SI-NEXT: s_mov_b32 s4, s10
+; SI-NEXT: s_mov_b32 s5, s34
+; SI-NEXT: s_mov_b32 s10, s11
+; SI-NEXT: s_mov_b32 s11, s13
+; SI-NEXT: s_mov_b32 s13, s15
+; SI-NEXT: s_mov_b32 s15, s45
+; SI-NEXT: s_mov_b32 s45, s47
; SI-NEXT: s_cbranch_scc1 .LBB93_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s36, s36, 3
-; SI-NEXT: s_and_b32 s4, s36, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_add_i32 s34, s34, 3
+; SI-NEXT: s_add_i32 s4, s31, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s30, 16
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s47, s4, 0x30000
+; SI-NEXT: s_add_i32 s4, s9, 3
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
; SI-NEXT: s_add_i32 s29, s29, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s7, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s30, 16
+; SI-NEXT: s_add_i32 s5, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s4, s7, s4
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s8, s28, 0xffff
+; SI-NEXT: s_add_i32 s7, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s28, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_add_i32 s27, s27, 3
-; SI-NEXT: s_or_b32 s6, s6, s8
-; SI-NEXT: s_and_b32 s8, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s9, vcc_hi, 16
+; SI-NEXT: s_add_i32 s6, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s8, vcc_hi, 16
+; SI-NEXT: s_or_b32 s4, s8, s4
; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: s_and_b32 s9, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_add_i32 s8, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s9, vcc_lo, 16
+; SI-NEXT: s_or_b32 s4, s9, s4
; SI-NEXT: s_add_i32 s25, s25, 3
-; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s10, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s11, s95, 16
+; SI-NEXT: s_add_i32 s9, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s4, s10, s4
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s10, s11, s10
-; SI-NEXT: s_and_b32 s11, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s12, s94, 16
+; SI-NEXT: s_add_i32 s10, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s11, s94, 16
+; SI-NEXT: s_or_b32 s4, s11, s4
; SI-NEXT: s_add_i32 s23, s23, 3
-; SI-NEXT: s_or_b32 s11, s12, s11
-; SI-NEXT: s_and_b32 s12, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s93, 16
+; SI-NEXT: s_add_i32 s11, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s93, 16
+; SI-NEXT: s_or_b32 s4, s12, s4
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_or_b32 s12, s13, s12
-; SI-NEXT: s_and_b32 s13, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s14, s92, 16
+; SI-NEXT: s_add_i32 s12, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s92, 16
+; SI-NEXT: s_or_b32 s4, s13, s4
; SI-NEXT: s_add_i32 s21, s21, 3
-; SI-NEXT: s_or_b32 s13, s14, s13
-; SI-NEXT: s_and_b32 s14, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s15, s91, 16
+; SI-NEXT: s_add_i32 s13, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s14, s91, 16
+; SI-NEXT: s_or_b32 s4, s14, s4
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_or_b32 s14, s15, s14
-; SI-NEXT: s_and_b32 s15, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s90, 16
+; SI-NEXT: s_add_i32 s14, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s15, s90, 16
+; SI-NEXT: s_or_b32 s4, s15, s4
; SI-NEXT: s_add_i32 s19, s19, 3
-; SI-NEXT: s_or_b32 s15, s20, s15
-; SI-NEXT: s_and_b32 s19, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s89, 16
+; SI-NEXT: s_add_i32 s15, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s19, s89, 16
+; SI-NEXT: s_or_b32 s4, s19, s4
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s19, s20, s19
-; SI-NEXT: s_and_b32 s18, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s88, 16
+; SI-NEXT: s_add_i32 s19, s4, 0x30000
+; SI-NEXT: s_and_b32 s4, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s18, s88, 16
+; SI-NEXT: s_or_b32 s4, s18, s4
+; SI-NEXT: s_add_i32 s18, s4, 0x30000
; SI-NEXT: s_add_i32 s17, s17, 3
-; SI-NEXT: s_or_b32 s18, s20, s18
+; SI-NEXT: v_readlane_b32 s4, v18, 1
; SI-NEXT: s_and_b32 s17, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s79, 16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s17
+; SI-NEXT: s_add_i32 s17, s4, 0x30000
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_or_b32 s17, s20, s17
+; SI-NEXT: v_readlane_b32 s4, v18, 0
; SI-NEXT: s_and_b32 s16, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s37, 16
-; SI-NEXT: s_or_b32 s16, s20, s16
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
-; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_add_i32 s8, s8, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
-; SI-NEXT: s_add_i32 s10, s10, 0x30000
-; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_add_i32 s12, s12, 0x30000
-; SI-NEXT: s_add_i32 s13, s13, 0x30000
-; SI-NEXT: s_add_i32 s14, s14, 0x30000
-; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s19, s19, 0x30000
-; SI-NEXT: s_add_i32 s18, s18, 0x30000
-; SI-NEXT: s_add_i32 s17, s17, 0x30000
-; SI-NEXT: s_add_i32 s16, s16, 0x30000
-; SI-NEXT: s_and_b32 s78, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s16, 16
+; SI-NEXT: s_and_b32 s78, s4, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s4, 16
; SI-NEXT: s_and_b32 s77, s17, 0xffff0000
; SI-NEXT: s_lshl_b32 s72, s17, 16
; SI-NEXT: s_and_b32 s76, s18, 0xffff0000
@@ -60860,20 +60874,20 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: s_lshl_b32 s44, s12, 16
; SI-NEXT: s_and_b32 s57, s11, 0xffff0000
; SI-NEXT: s_lshl_b32 s42, s11, 16
-; SI-NEXT: s_and_b32 s47, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s14, s10, 16
-; SI-NEXT: s_and_b32 s45, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s12, s9, 16
-; SI-NEXT: s_and_b32 s40, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s10, s8, 16
-; SI-NEXT: s_and_b32 s15, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s6, 16
-; SI-NEXT: s_and_b32 s13, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s11, s5, 0xffff0000
+; SI-NEXT: s_and_b32 s45, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s15, s10, 16
+; SI-NEXT: s_and_b32 s43, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s13, s9, 16
+; SI-NEXT: s_and_b32 s41, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s11, s8, 16
+; SI-NEXT: s_and_b32 s40, s6, 0xffff0000
+; SI-NEXT: s_lshl_b32 s10, s6, 16
+; SI-NEXT: s_and_b32 s14, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s79, s7, 16
+; SI-NEXT: s_and_b32 s12, s5, 0xffff0000
; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_and_b32 s8, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_and_b32 s35, s47, 0xffff0000
+; SI-NEXT: s_lshl_b32 s4, s47, 16
; SI-NEXT: .LBB93_5: ; %end
; SI-NEXT: v_mul_f32_e64 v0, 1.0, s78
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
@@ -60911,42 +60925,41 @@ define inreg <32 x bfloat> @bitcast_v32i16_to_v32bf16_scalar(<32 x i16> inreg %a
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
; SI-NEXT: v_mul_f32_e64 v8, 1.0, s42
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s47
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s45
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s14
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s15
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s45
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s43
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s12
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s13
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s40
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s41
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s11
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s15
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s40
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s10
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s13
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s14
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s79
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s11
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s12
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
; SI-NEXT: v_mul_f32_e64 v14, 1.0, s5
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
; SI-NEXT: v_mul_f32_e64 v15, 1.0, s4
-; SI-NEXT: v_readlane_b32 s30, v17, 4
+; SI-NEXT: v_readlane_b32 s30, v17, 2
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_readlane_b32 s31, v17, 5
-; SI-NEXT: v_readlane_b32 s37, v17, 3
-; SI-NEXT: v_readlane_b32 s36, v17, 2
+; SI-NEXT: v_readlane_b32 s31, v17, 3
; SI-NEXT: v_readlane_b32 s35, v17, 1
; SI-NEXT: v_readlane_b32 s34, v17, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v17, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v18, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -70612,13 +70625,8 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v43, s84, 26
; SI-NEXT: v_writelane_b32 v43, s85, 27
; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s97, 31
-; SI-NEXT: v_writelane_b32 v43, s98, 32
-; SI-NEXT: v_writelane_b32 v43, s99, 33
-; SI-NEXT: v_writelane_b32 v43, s30, 34
-; SI-NEXT: v_writelane_b32 v43, s31, 35
+; SI-NEXT: v_writelane_b32 v43, s30, 29
+; SI-NEXT: v_writelane_b32 v43, s31, 30
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -70662,8 +70670,9 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v44, s10, 7
; SI-NEXT: v_readfirstlane_b32 s46, v16
; SI-NEXT: v_writelane_b32 v44, s11, 8
-; SI-NEXT: v_readfirstlane_b32 s60, v2
+; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: v_writelane_b32 v44, s46, 9
+; SI-NEXT: v_writelane_b32 v44, s45, 10
; SI-NEXT: v_readfirstlane_b32 s63, v30
; SI-NEXT: v_readfirstlane_b32 s52, v29
; SI-NEXT: v_readfirstlane_b32 s51, v28
@@ -70683,8 +70692,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s59, v12
; SI-NEXT: v_readfirstlane_b32 s35, v11
; SI-NEXT: v_readfirstlane_b32 s36, v10
-; SI-NEXT: v_readfirstlane_b32 s45, v3
-; SI-NEXT: v_writelane_b32 v44, s60, 10
+; SI-NEXT: v_readfirstlane_b32 s68, v3
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
@@ -70693,73 +70701,70 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s40, v34
; SI-NEXT: v_readfirstlane_b32 s41, v35
; SI-NEXT: v_readfirstlane_b32 s13, v36
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_readfirstlane_b32 s5, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s76, v39
-; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s31, v48
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s73, v49
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s77, v39
+; SI-NEXT: v_writelane_b32 v44, s77, 11
+; SI-NEXT: v_writelane_b32 v44, s5, 12
+; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s44, v51
-; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: v_readfirstlane_b32 s31, v48
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s43, v53
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_readfirstlane_b32 s73, v49
+; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s76, v41
+; SI-NEXT: v_writelane_b32 v44, s76, 13
+; SI-NEXT: v_writelane_b32 v44, s44, 14
; SI-NEXT: v_readfirstlane_b32 s65, v54
-; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_readfirstlane_b32 s66, v55
-; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s55, v40
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s82, v41
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s64, v42
-; SI-NEXT: v_writelane_b32 v44, s76, 11
+; SI-NEXT: v_writelane_b32 v44, s43, 15
+; SI-NEXT: v_writelane_b32 v44, s42, 16
; SI-NEXT: s_cbranch_scc0 .LBB99_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_lshl_b32 s6, s17, 8
-; SI-NEXT: s_or_b32 vcc_hi, s4, s6
+; SI-NEXT: s_or_b32 s88, s4, s6
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s19, 24
; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s6, s24, 0xff
-; SI-NEXT: s_lshl_b32 s62, s25, 8
-; SI-NEXT: s_or_b32 s67, s6, s62
+; SI-NEXT: s_lshl_b32 s60, s25, 8
+; SI-NEXT: s_or_b32 s89, s6, s60
; SI-NEXT: s_and_b32 s6, s26, 0xff
; SI-NEXT: s_and_b32 s72, s72, 0xff
; SI-NEXT: s_lshl_b32 s58, s58, 8
; SI-NEXT: s_and_b32 s56, s56, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s62, s27, 24
+; SI-NEXT: s_lshl_b32 s60, s27, 24
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_or_b32 s58, s72, s58
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s9, s9, 24
-; SI-NEXT: s_or_b32 s6, s62, s6
-; SI-NEXT: s_and_b32 s60, s60, 0xff
-; SI-NEXT: s_lshl_b32 s62, s45, 8
+; SI-NEXT: s_or_b32 s6, s60, s6
+; SI-NEXT: s_and_b32 s45, s45, 0xff
+; SI-NEXT: s_lshl_b32 s60, s68, 8
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_and_b32 s58, s58, 0xffff
-; SI-NEXT: s_or_b32 s87, s9, s56
-; SI-NEXT: s_or_b32 s70, s60, s62
+; SI-NEXT: s_or_b32 s81, s9, s56
+; SI-NEXT: s_or_b32 s45, s45, s60
; SI-NEXT: s_or_b32 s8, s8, s10
; SI-NEXT: s_and_b32 s10, s36, 0xff
; SI-NEXT: s_lshl_b32 s60, s35, 8
-; SI-NEXT: s_or_b32 s9, s58, s87
+; SI-NEXT: s_or_b32 s9, s58, s81
; SI-NEXT: s_and_b32 s56, s57, 0xff
; SI-NEXT: s_lshl_b32 s58, s47, 8
; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_or_b32 s71, s10, s60
+; SI-NEXT: s_or_b32 s90, s10, s60
; SI-NEXT: s_and_b32 s10, s59, 0xff
; SI-NEXT: s_or_b32 s56, s56, s58
; SI-NEXT: s_lshl_b32 s46, s46, 16
@@ -70767,14 +70772,14 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s60, s30, 24
; SI-NEXT: s_and_b32 s56, s56, 0xffff
-; SI-NEXT: s_or_b32 s96, s11, s46
+; SI-NEXT: s_or_b32 s82, s11, s46
; SI-NEXT: s_or_b32 s10, s60, s10
; SI-NEXT: s_and_b32 s60, s50, 0xff
; SI-NEXT: s_lshl_b32 s62, s49, 8
-; SI-NEXT: s_or_b32 s11, s56, s96
+; SI-NEXT: s_or_b32 s11, s56, s82
; SI-NEXT: s_and_b32 s46, s38, 0xff
; SI-NEXT: s_lshl_b32 s56, s37, 8
-; SI-NEXT: s_or_b32 s80, s60, s62
+; SI-NEXT: s_or_b32 vcc_hi, s60, s62
; SI-NEXT: s_and_b32 s60, s39, 0xff
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s56, s61, 0xff
@@ -70784,102 +70789,95 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s58, s34, 24
; SI-NEXT: s_or_b32 s60, s62, s60
; SI-NEXT: s_and_b32 s62, s54, 0xff
-; SI-NEXT: s_lshl_b32 s77, s53, 8
+; SI-NEXT: s_lshl_b32 s78, s53, 8
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s97, s58, s56
-; SI-NEXT: s_or_b32 s81, s62, s77
+; SI-NEXT: s_or_b32 s83, s58, s56
+; SI-NEXT: s_or_b32 s67, s62, s78
; SI-NEXT: s_and_b32 s62, s51, 0xff
-; SI-NEXT: s_and_b32 s76, s76, 0xff
-; SI-NEXT: s_mov_b32 s79, s5
+; SI-NEXT: s_and_b32 s77, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_mov_b32 s72, s61
-; SI-NEXT: s_or_b32 s61, s46, s97
+; SI-NEXT: s_or_b32 s61, s46, s83
; SI-NEXT: s_and_b32 s46, s63, 0xff
; SI-NEXT: s_lshl_b32 s56, s64, 8
; SI-NEXT: s_lshl_b32 s62, s62, 16
-; SI-NEXT: s_lshl_b32 s77, s52, 24
-; SI-NEXT: s_or_b32 s84, s76, s5
+; SI-NEXT: s_lshl_b32 s78, s52, 24
+; SI-NEXT: s_or_b32 s70, s77, s5
; SI-NEXT: s_and_b32 s5, s20, 0xff
-; SI-NEXT: s_lshl_b32 s76, s21, 8
+; SI-NEXT: s_lshl_b32 s77, s21, 8
; SI-NEXT: s_or_b32 s46, s46, s56
-; SI-NEXT: s_and_b32 s56, s82, 0xff
-; SI-NEXT: s_or_b32 s62, s77, s62
-; SI-NEXT: s_and_b32 s77, s66, 0xff
-; SI-NEXT: s_lshl_b32 s78, s65, 8
-; SI-NEXT: s_or_b32 s5, s5, s76
-; SI-NEXT: s_and_b32 s76, s22, 0xff
+; SI-NEXT: s_and_b32 s56, s76, 0xff
+; SI-NEXT: s_or_b32 s62, s78, s62
+; SI-NEXT: s_and_b32 s78, s66, 0xff
+; SI-NEXT: s_lshl_b32 s79, s65, 8
+; SI-NEXT: s_or_b32 s5, s5, s77
+; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_and_b32 s74, s74, 0xff
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s58, s55, 24
; SI-NEXT: s_mov_b32 s91, s14
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_or_b32 s83, s77, s78
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s23, 24
+; SI-NEXT: s_or_b32 s69, s78, s79
+; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_lshl_b32 s74, s74, 16
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s98, s58, s56
+; SI-NEXT: s_or_b32 s84, s58, s56
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s85, s77, s76
-; SI-NEXT: s_and_b32 s76, s28, 0xff
-; SI-NEXT: s_lshl_b32 s77, s29, 8
-; SI-NEXT: s_or_b32 s86, s7, s74
+; SI-NEXT: s_or_b32 s71, s78, s77
+; SI-NEXT: s_and_b32 s77, s28, 0xff
+; SI-NEXT: s_lshl_b32 s78, s29, 8
+; SI-NEXT: s_or_b32 s80, s7, s74
; SI-NEXT: s_mov_b32 s74, s63
-; SI-NEXT: s_or_b32 s63, s46, s98
-; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_or_b32 s63, s46, s84
; SI-NEXT: s_and_b32 s44, s44, 0xff
; SI-NEXT: s_lshl_b32 s46, s75, 8
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_and_b32 s41, s41, 0xff
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_lshl_b32 s40, s40, 8
-; SI-NEXT: s_or_b32 s78, s12, s14
+; SI-NEXT: s_or_b32 s86, s12, s14
; SI-NEXT: s_and_b32 s12, s15, 0xff
-; SI-NEXT: s_or_b32 s76, s76, s77
+; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_or_b32 s44, s44, s46
; SI-NEXT: s_and_b32 s46, s73, 0xff
-; SI-NEXT: s_mov_b32 s89, s43
; SI-NEXT: s_and_b32 s43, s43, 0xff
; SI-NEXT: s_or_b32 s40, s41, s40
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_lshl_b32 s13, s13, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_and_b32 s76, s76, 0xffff
+; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_lshl_b32 s46, s46, 16
; SI-NEXT: s_lshl_b32 s56, s31, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_mov_b32 s90, s42
; SI-NEXT: s_lshl_b32 s42, s42, 24
-; SI-NEXT: s_or_b32 s68, s13, s12
+; SI-NEXT: s_or_b32 s78, s13, s12
; SI-NEXT: s_and_b32 s12, s40, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s85
-; SI-NEXT: s_or_b32 s7, s76, s86
-; SI-NEXT: s_or_b32 s99, s56, s46
+; SI-NEXT: s_or_b32 s5, s5, s71
+; SI-NEXT: s_or_b32 s7, s77, s80
+; SI-NEXT: s_or_b32 s85, s56, s46
; SI-NEXT: s_or_b32 s76, s42, s43
; SI-NEXT: s_and_b32 s42, s44, 0xffff
-; SI-NEXT: s_or_b32 s69, s12, s78
-; SI-NEXT: s_and_b32 s12, vcc_hi, 0xffff
-; SI-NEXT: s_or_b32 s77, s42, s99
+; SI-NEXT: s_or_b32 s79, s12, s86
+; SI-NEXT: s_and_b32 s12, s88, 0xffff
+; SI-NEXT: s_or_b32 s77, s42, s85
+; SI-NEXT: s_and_b32 s14, s45, 0xffff
; SI-NEXT: s_or_b32 s42, s12, s4
-; SI-NEXT: s_mov_b32 s12, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[4:5], 16
; SI-NEXT: s_mov_b32 s4, s47
; SI-NEXT: s_lshr_b64 s[46:47], s[6:7], 16
-; SI-NEXT: s_and_b32 s13, s67, 0xffff
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
-; SI-NEXT: s_and_b32 s58, s71, 0xffff
-; SI-NEXT: s_and_b32 vcc_hi, s80, 0xffff
-; SI-NEXT: s_and_b32 s67, s81, 0xffff
+; SI-NEXT: s_and_b32 s13, s89, 0xffff
+; SI-NEXT: s_and_b32 s58, s90, 0xffff
+; SI-NEXT: s_and_b32 s88, vcc_hi, 0xffff
+; SI-NEXT: s_and_b32 s89, s67, 0xffff
; SI-NEXT: s_mov_b32 s47, s4
; SI-NEXT: s_mov_b32 s4, s57
; SI-NEXT: s_lshr_b64 s[56:57], s[8:9], 16
; SI-NEXT: s_mov_b32 s95, s15
-; SI-NEXT: s_and_b32 s70, s83, 0xffff
-; SI-NEXT: s_mov_b32 s45, s12
; SI-NEXT: s_or_b32 s40, s13, s6
; SI-NEXT: s_or_b32 s14, s14, s8
; SI-NEXT: s_mov_b32 s15, s9
@@ -70888,62 +70886,59 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_mov_b32 s13, s11
; SI-NEXT: s_mov_b32 s4, s59
; SI-NEXT: s_lshr_b64 s[58:59], s[10:11], 16
-; SI-NEXT: s_or_b32 s10, vcc_hi, s60
+; SI-NEXT: s_or_b32 s10, s88, s60
; SI-NEXT: s_mov_b32 s11, s61
; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_or_b32 s8, s67, s62
+; SI-NEXT: s_or_b32 s8, s89, s62
; SI-NEXT: s_mov_b32 s9, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
-; SI-NEXT: s_and_b32 s71, s84, 0xffff
+; SI-NEXT: s_and_b32 s90, s69, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s70, 0xffff
; SI-NEXT: s_mov_b32 s59, s4
; SI-NEXT: s_mov_b32 s61, s72
; SI-NEXT: s_mov_b32 s63, s74
-; SI-NEXT: s_or_b32 s6, s70, s76
; SI-NEXT: s_mov_b32 s4, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[76:77], 16
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[68:69], 16
+; SI-NEXT: s_mov_b32 s45, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[78:79], 16
; SI-NEXT: s_mov_b32 s43, s5
; SI-NEXT: s_mov_b32 s41, s7
+; SI-NEXT: s_or_b32 s6, s90, s76
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_mov_b32 s73, s4
-; SI-NEXT: s_or_b32 s4, s71, s68
-; SI-NEXT: s_mov_b32 s5, s69
-; SI-NEXT: s_mov_b32 s75, s76
-; SI-NEXT: s_lshr_b32 s71, s85, 16
-; SI-NEXT: s_lshr_b32 s70, s86, 16
-; SI-NEXT: s_lshr_b32 s68, s87, 16
-; SI-NEXT: s_lshr_b32 s69, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s97, 16
-; SI-NEXT: s_lshr_b32 s67, s98, 16
-; SI-NEXT: s_lshr_b32 s80, s99, 16
-; SI-NEXT: s_lshr_b32 s81, s78, 16
+; SI-NEXT: s_or_b32 s4, vcc_hi, s78
+; SI-NEXT: s_mov_b32 s5, s79
+; SI-NEXT: s_mov_b32 s75, s45
+; SI-NEXT: s_lshr_b32 s89, s71, 16
+; SI-NEXT: s_lshr_b32 s88, s80, 16
+; SI-NEXT: s_lshr_b32 s79, s81, 16
+; SI-NEXT: s_lshr_b32 s78, s82, 16
+; SI-NEXT: s_lshr_b32 vcc_hi, s83, 16
+; SI-NEXT: s_lshr_b32 s67, s84, 16
+; SI-NEXT: s_lshr_b32 s90, s85, 16
+; SI-NEXT: s_lshr_b32 s45, s86, 16
; SI-NEXT: s_mov_b64 s[76:77], 0
; SI-NEXT: s_branch .LBB99_3
; SI-NEXT: .LBB99_2:
-; SI-NEXT: s_mov_b32 s79, s5
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_mov_b32 s95, s15
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_mov_b32 s91, s14
-; SI-NEXT: s_mov_b32 s90, s42
-; SI-NEXT: s_mov_b32 s89, s43
-; SI-NEXT: s_mov_b32 s88, s44
; SI-NEXT: s_mov_b64 s[76:77], -1
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr71
+; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $vcc_hi
@@ -70952,10 +70947,10 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr45
; SI-NEXT: .LBB99_3: ; %Flow
; SI-NEXT: s_and_b64 s[76:77], s[76:77], exec
; SI-NEXT: s_cselect_b32 s76, 1, 0
@@ -70964,10 +70959,11 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_readlane_b32 s4, v44, 11
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s79, 8
+; SI-NEXT: s_and_b32 s5, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v44, 12
+; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_add_i32 s6, s95, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_lshl_b32 s5, vcc_lo, 24
@@ -70990,16 +70986,19 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s6, s66, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s65, 8
-; SI-NEXT: s_add_i32 s8, s89, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s90, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s88, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_lshl_b32 s8, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: v_readlane_b32 s7, v44, 14
+; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s75, 8
; SI-NEXT: s_add_i32 s9, s73, 3
@@ -71026,15 +71025,16 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s9, s63, 3
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s64, 8
-; SI-NEXT: s_add_i32 s82, s82, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s11, s82, 0xff
; SI-NEXT: s_addk_i32 s9, 0x300
-; SI-NEXT: s_lshl_b32 s10, s55, 24
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_or_b32 s10, s10, s11
-; SI-NEXT: s_or_b32 s9, s10, s9
+; SI-NEXT: s_and_b32 s10, s9, 0xffff
+; SI-NEXT: v_readlane_b32 s9, v44, 13
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_lshl_b32 s11, s55, 24
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s9, s11, s9
+; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_add_i32 s10, s50, 3
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s11, s49, 8
@@ -71088,7 +71088,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readlane_b32 s14, v44, 10
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s15, s45, 8
+; SI-NEXT: s_lshl_b32 s15, s68, 8
; SI-NEXT: s_or_b32 s14, s15, s14
; SI-NEXT: s_addk_i32 s14, 0x300
; SI-NEXT: s_and_b32 s15, s14, 0xffff
@@ -71125,49 +71125,50 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_and_b32 s24, s24, 0xffff
; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_or_b32 s15, s41, s15
; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s15, s15, s40
; SI-NEXT: s_add_i32 s40, s24, 0x3000000
; SI-NEXT: s_and_b32 s24, s28, 0xff
; SI-NEXT: s_lshl_b32 s25, s29, 8
-; SI-NEXT: s_and_b32 s16, s16, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s25, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v44, 0
+; SI-NEXT: s_addk_i32 s24, 0x300
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_lshl_b32 s26, s24, 24
-; SI-NEXT: v_readlane_b32 s24, v44, 1
+; SI-NEXT: s_and_b32 s25, s24, 0xffff
+; SI-NEXT: v_readlane_b32 s24, v44, 0
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: s_lshl_b32 s26, s24, 24
+; SI-NEXT: v_readlane_b32 s24, v44, 1
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
; SI-NEXT: s_and_b32 s16, s20, 0xff
; SI-NEXT: s_lshl_b32 s17, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s22, 0xff
-; SI-NEXT: s_lshl_b32 s24, s24, 16
+; SI-NEXT: s_and_b32 s24, s24, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s23, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s24, s26, s24
+; SI-NEXT: s_lshl_b32 s24, s24, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -71181,7 +71182,6 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
; SI-NEXT: s_add_i32 s41, s24, 0x3000000
-; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[46:47], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[56:57], s[14:15], 16
@@ -71190,14 +71190,14 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshr_b64 s[62:63], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[72:73], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b32 s71, s43, 16
-; SI-NEXT: s_lshr_b32 s70, s41, 16
-; SI-NEXT: s_lshr_b32 s68, s15, 16
-; SI-NEXT: s_lshr_b32 s69, s13, 16
+; SI-NEXT: s_lshr_b32 s89, s43, 16
+; SI-NEXT: s_lshr_b32 s88, s41, 16
+; SI-NEXT: s_lshr_b32 s79, s15, 16
+; SI-NEXT: s_lshr_b32 s78, s13, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s11, 16
; SI-NEXT: s_lshr_b32 s67, s9, 16
-; SI-NEXT: s_lshr_b32 s80, s7, 16
-; SI-NEXT: s_lshr_b32 s81, s5, 16
+; SI-NEXT: s_lshr_b32 s90, s7, 16
+; SI-NEXT: s_lshr_b32 s45, s5, 16
; SI-NEXT: .LBB99_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
@@ -71206,25 +71206,25 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s17, s44, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s71, 16
+; SI-NEXT: s_lshl_b32 s18, s89, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s40, 0xffff
; SI-NEXT: s_lshl_b32 s19, s46, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s70, 16
+; SI-NEXT: s_lshl_b32 s20, s88, 16
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_lshl_b32 s20, s56, 16
; SI-NEXT: s_or_b32 s14, s14, s20
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s68, 16
+; SI-NEXT: s_lshl_b32 s20, s79, 16
; SI-NEXT: s_or_b32 s15, s15, s20
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_lshl_b32 s20, s58, 16
; SI-NEXT: s_or_b32 s12, s12, s20
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_lshl_b32 s20, s78, 16
; SI-NEXT: s_or_b32 s13, s13, s20
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_lshl_b32 s20, s60, 16
@@ -71242,15 +71242,15 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s20, s72, 16
; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s80, 16
+; SI-NEXT: s_lshl_b32 s20, s90, 16
; SI-NEXT: s_or_b32 s7, s7, s20
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s20, s74, 16
; SI-NEXT: s_or_b32 s4, s4, s20
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s81, 16
+; SI-NEXT: s_lshl_b32 s20, s45, 16
; SI-NEXT: s_or_b32 s5, s5, s20
-; SI-NEXT: v_readlane_b32 s30, v43, 34
+; SI-NEXT: v_readlane_b32 s30, v43, 29
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s18
@@ -71267,12 +71267,7 @@ define inreg <32 x i16> @bitcast_v64i8_to_v32i16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_mov_b32_e32 v13, s7
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_readlane_b32 s31, v43, 35
-; SI-NEXT: v_readlane_b32 s99, v43, 33
-; SI-NEXT: v_readlane_b32 s98, v43, 32
-; SI-NEXT: v_readlane_b32 s97, v43, 31
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
+; SI-NEXT: v_readlane_b32 s31, v43, 30
; SI-NEXT: v_readlane_b32 s86, v43, 28
; SI-NEXT: v_readlane_b32 s85, v43, 27
; SI-NEXT: v_readlane_b32 s84, v43, 26
@@ -78803,9 +78798,8 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_writelane_b32 v18, s55, 13
; SI-NEXT: v_writelane_b32 v18, s64, 14
; SI-NEXT: v_writelane_b32 v18, s65, 15
-; SI-NEXT: v_writelane_b32 v18, s66, 16
-; SI-NEXT: v_writelane_b32 v18, s30, 17
-; SI-NEXT: v_writelane_b32 v18, s31, 18
+; SI-NEXT: v_writelane_b32 v18, s30, 16
+; SI-NEXT: v_writelane_b32 v18, s31, 17
; SI-NEXT: v_readfirstlane_b32 s31, v2
; SI-NEXT: v_readfirstlane_b32 s44, v1
; SI-NEXT: s_lshr_b32 s75, s29, 16
@@ -78823,7 +78817,7 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshr_b32 s45, s17, 16
; SI-NEXT: s_lshr_b32 s47, s16, 16
; SI-NEXT: s_lshr_b32 s49, s31, 16
-; SI-NEXT: s_lshr_b32 s50, s44, 16
+; SI-NEXT: s_lshr_b32 s46, s44, 16
; SI-NEXT: v_readfirstlane_b32 s4, v3
; SI-NEXT: s_cmp_lg_u32 s4, 0
; SI-NEXT: ; implicit-def: $vgpr19 : SGPR spill to VGPR lane
@@ -78881,6 +78875,9 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_and_b32 s4, s23, 0xffff
; SI-NEXT: s_lshl_b32 s5, s38, 16
; SI-NEXT: s_or_b32 s11, s4, s5
+; SI-NEXT: s_lshr_b64 s[4:5], s[10:11], 24
+; SI-NEXT: v_writelane_b32 v19, s4, 18
+; SI-NEXT: v_writelane_b32 v19, s5, 19
; SI-NEXT: s_and_b32 s4, s24, 0xffff
; SI-NEXT: s_lshl_b32 s5, s63, 16
; SI-NEXT: s_or_b32 s12, s4, s5
@@ -78900,17 +78897,18 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshl_b32 s5, s75, 16
; SI-NEXT: s_or_b32 s7, s4, s5
; SI-NEXT: s_and_b32 s4, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s50, 16
+; SI-NEXT: s_lshl_b32 s5, s46, 16
; SI-NEXT: s_mov_b32 s76, s44
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s5, s31, 0xffff
; SI-NEXT: s_lshl_b32 s44, s49, 16
; SI-NEXT: s_or_b32 s5, s5, s44
; SI-NEXT: s_lshr_b32 s44, s43, 8
-; SI-NEXT: v_writelane_b32 v19, s44, 19
+; SI-NEXT: v_writelane_b32 v19, s44, 21
; SI-NEXT: s_lshr_b32 s44, s41, 8
-; SI-NEXT: v_writelane_b32 v19, s44, 18
-; SI-NEXT: s_bfe_u32 s51, s45, 0x80008
+; SI-NEXT: s_mov_b32 s78, s46
+; SI-NEXT: v_writelane_b32 v19, s44, 20
+; SI-NEXT: s_bfe_u32 s50, s45, 0x80008
; SI-NEXT: s_mov_b32 s46, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[12:13], 8
; SI-NEXT: s_mov_b32 s45, s46
@@ -78934,11 +78932,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_mov_b32 s59, s72
; SI-NEXT: s_mov_b32 s72, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[4:5], 24
-; SI-NEXT: s_bfe_u32 s65, s75, 0x80008
+; SI-NEXT: s_bfe_u32 s64, s75, 0x80008
; SI-NEXT: s_mov_b32 vcc_hi, s72
; SI-NEXT: s_mov_b32 s72, s75
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b64 s[78:79], s[10:11], 24
; SI-NEXT: s_lshr_b64 s[90:91], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[10:11], 8
; SI-NEXT: s_lshr_b64 s[88:89], s[12:13], 24
@@ -78952,47 +78949,52 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_lshr_b32 s89, s9, 8
; SI-NEXT: s_lshr_b32 s79, s7, 8
; SI-NEXT: s_lshr_b32 s77, s5, 8
-; SI-NEXT: s_bfe_u32 s52, s36, 0x80008
-; SI-NEXT: s_bfe_u32 s53, s37, 0x80008
-; SI-NEXT: s_bfe_u32 s54, s38, 0x80008
-; SI-NEXT: s_bfe_u32 s55, s39, 0x80008
-; SI-NEXT: s_bfe_u32 s64, s48, 0x80008
-; SI-NEXT: s_bfe_u32 s66, s49, 0x80008
+; SI-NEXT: s_bfe_u32 s51, s36, 0x80008
+; SI-NEXT: s_bfe_u32 s52, s37, 0x80008
+; SI-NEXT: s_bfe_u32 s53, s38, 0x80008
+; SI-NEXT: s_bfe_u32 s54, s39, 0x80008
+; SI-NEXT: s_bfe_u32 s55, s48, 0x80008
+; SI-NEXT: s_bfe_u32 s65, s49, 0x80008
; SI-NEXT: s_mov_b32 s73, s34
; SI-NEXT: s_mov_b64 s[34:35], 0
; SI-NEXT: s_branch .LBB105_3
; SI-NEXT: .LBB105_2:
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: s_mov_b32 s76, s44
+; SI-NEXT: s_mov_b32 s78, s46
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v19, s4, 0
; SI-NEXT: v_writelane_b32 v19, s5, 1
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: s_mov_b32 s76, s44
; SI-NEXT: v_writelane_b32 v19, s4, 2
; SI-NEXT: v_writelane_b32 v19, s5, 3
; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: v_writelane_b32 v19, s4, 4
+; SI-NEXT: v_writelane_b32 v19, s5, 5
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: ; kill: killed $sgpr4
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr42
-; SI-NEXT: ; implicit-def: $sgpr51
+; SI-NEXT: ; implicit-def: $sgpr50
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr52
+; SI-NEXT: ; implicit-def: $sgpr51
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr53
+; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr93
-; SI-NEXT: ; implicit-def: $sgpr54
+; SI-NEXT: ; implicit-def: $sgpr53
; SI-NEXT: ; implicit-def: $sgpr91
-; SI-NEXT: ; implicit-def: $sgpr55
+; SI-NEXT: ; implicit-def: $sgpr54
; SI-NEXT: ; implicit-def: $sgpr89
-; SI-NEXT: ; implicit-def: $sgpr64
+; SI-NEXT: ; implicit-def: $sgpr55
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr65
+; SI-NEXT: ; implicit-def: $sgpr64
; SI-NEXT: ; implicit-def: $sgpr77
-; SI-NEXT: ; implicit-def: $sgpr66
+; SI-NEXT: ; implicit-def: $sgpr65
; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr90
-; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr44
; SI-NEXT: ; implicit-def: $sgpr92
@@ -79008,11 +79010,6 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: v_writelane_b32 v19, s4, 4
-; SI-NEXT: v_writelane_b32 v19, s5, 5
-; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; kill: killed $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: v_writelane_b32 v19, s4, 6
; SI-NEXT: v_writelane_b32 v19, s5, 7
; SI-NEXT: ; implicit-def: $sgpr4
@@ -79033,6 +79030,9 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_writelane_b32 v19, s4, 16
; SI-NEXT: v_writelane_b32 v19, s5, 17
; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: v_writelane_b32 v19, s4, 18
+; SI-NEXT: v_writelane_b32 v19, s5, 19
+; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: .LBB105_3: ; %Flow
; SI-NEXT: s_and_b64 s[34:35], s[34:35], exec
; SI-NEXT: s_cselect_b32 s34, 1, 0
@@ -79040,7 +79040,7 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_cbranch_scc1 .LBB105_5
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_cvt_f32_f16_e32 v2, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s50
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s78
; SI-NEXT: v_cvt_f32_f16_e32 v3, s49
; SI-NEXT: v_cvt_f32_f16_e32 v9, s47
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
@@ -79241,9 +79241,11 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_bfe_u32 v7, v1, 8, 8
; SI-NEXT: s_branch .LBB105_6
; SI-NEXT: .LBB105_5:
-; SI-NEXT: v_mov_b32_e32 v2, s75
; SI-NEXT: v_mov_b32_e32 v6, s37
; SI-NEXT: v_mov_b32_e32 v8, s36
+; SI-NEXT: v_readlane_b32 s36, v19, 12
+; SI-NEXT: v_readlane_b32 s37, v19, 13
+; SI-NEXT: v_mov_b32_e32 v2, s75
; SI-NEXT: s_mov_b32 s16, s74
; SI-NEXT: s_mov_b32 s20, s62
; SI-NEXT: s_mov_b32 s18, s72
@@ -79259,20 +79261,21 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_readlane_b32 s46, v19, 16
; SI-NEXT: v_readlane_b32 s62, v19, 6
; SI-NEXT: v_readlane_b32 s56, v19, 14
-; SI-NEXT: v_readlane_b32 s36, v19, 12
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: v_readlane_b32 s36, v19, 18
; SI-NEXT: v_mov_b32_e32 v1, s49
; SI-NEXT: v_mov_b32_e32 v3, s48
; SI-NEXT: v_mov_b32_e32 v4, s39
; SI-NEXT: v_mov_b32_e32 v5, s38
; SI-NEXT: v_mov_b32_e32 v12, s45
-; SI-NEXT: v_mov_b32_e32 v7, s66
-; SI-NEXT: v_mov_b32_e32 v9, s65
-; SI-NEXT: v_mov_b32_e32 v10, s64
-; SI-NEXT: v_mov_b32_e32 v11, s55
-; SI-NEXT: v_mov_b32_e32 v13, s54
-; SI-NEXT: v_mov_b32_e32 v14, s53
-; SI-NEXT: v_mov_b32_e32 v15, s52
-; SI-NEXT: v_mov_b32_e32 v16, s51
+; SI-NEXT: v_mov_b32_e32 v7, s65
+; SI-NEXT: v_mov_b32_e32 v9, s64
+; SI-NEXT: v_mov_b32_e32 v10, s55
+; SI-NEXT: v_mov_b32_e32 v11, s54
+; SI-NEXT: v_mov_b32_e32 v13, s53
+; SI-NEXT: v_mov_b32_e32 v14, s52
+; SI-NEXT: v_mov_b32_e32 v15, s51
+; SI-NEXT: v_mov_b32_e32 v16, s50
; SI-NEXT: v_readlane_b32 s73, v19, 5
; SI-NEXT: v_readlane_b32 s75, v19, 3
; SI-NEXT: v_readlane_b32 s59, v19, 11
@@ -79281,10 +79284,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: v_readlane_b32 s47, v19, 17
; SI-NEXT: v_readlane_b32 s63, v19, 7
; SI-NEXT: v_readlane_b32 s57, v19, 15
-; SI-NEXT: v_readlane_b32 s17, v19, 18
-; SI-NEXT: v_readlane_b32 s19, v19, 19
-; SI-NEXT: v_readlane_b32 s37, v19, 13
-; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: v_readlane_b32 s17, v19, 20
+; SI-NEXT: v_readlane_b32 s19, v19, 21
+; SI-NEXT: v_readlane_b32 s37, v19, 19
+; SI-NEXT: s_mov_b32 s78, s36
; SI-NEXT: .LBB105_6: ; %end
; SI-NEXT: s_and_b32 s21, s42, 0xff
; SI-NEXT: s_lshl_b32 s23, s34, 8
@@ -79466,11 +79469,10 @@ define inreg <64 x i8> @bitcast_v32f16_to_v64i8_scalar(<32 x half> inreg %a, i32
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: v_or_b32_e32 v1, v2, v1
; SI-NEXT: v_or_b32_e32 v1, s4, v1
-; SI-NEXT: v_readlane_b32 s30, v18, 17
+; SI-NEXT: v_readlane_b32 s30, v18, 16
; SI-NEXT: buffer_store_dword v17, v0, s[0:3], 0 offen
; SI-NEXT: buffer_store_dword v1, v0, s[0:3], 0 offen offset:60
-; SI-NEXT: v_readlane_b32 s31, v18, 18
-; SI-NEXT: v_readlane_b32 s66, v18, 16
+; SI-NEXT: v_readlane_b32 s31, v18, 17
; SI-NEXT: v_readlane_b32 s65, v18, 15
; SI-NEXT: v_readlane_b32 s64, v18, 14
; SI-NEXT: v_readlane_b32 s55, v18, 13
@@ -83232,13 +83234,8 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v43, s84, 26
; SI-NEXT: v_writelane_b32 v43, s85, 27
; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s97, 31
-; SI-NEXT: v_writelane_b32 v43, s98, 32
-; SI-NEXT: v_writelane_b32 v43, s99, 33
-; SI-NEXT: v_writelane_b32 v43, s30, 34
-; SI-NEXT: v_writelane_b32 v43, s31, 35
+; SI-NEXT: v_writelane_b32 v43, s30, 29
+; SI-NEXT: v_writelane_b32 v43, s31, 30
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -83282,8 +83279,9 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_writelane_b32 v44, s10, 7
; SI-NEXT: v_readfirstlane_b32 s46, v16
; SI-NEXT: v_writelane_b32 v44, s11, 8
-; SI-NEXT: v_readfirstlane_b32 s60, v2
+; SI-NEXT: v_readfirstlane_b32 s45, v2
; SI-NEXT: v_writelane_b32 v44, s46, 9
+; SI-NEXT: v_writelane_b32 v44, s45, 10
; SI-NEXT: v_readfirstlane_b32 s63, v30
; SI-NEXT: v_readfirstlane_b32 s52, v29
; SI-NEXT: v_readfirstlane_b32 s51, v28
@@ -83303,8 +83301,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s59, v12
; SI-NEXT: v_readfirstlane_b32 s35, v11
; SI-NEXT: v_readfirstlane_b32 s36, v10
-; SI-NEXT: v_readfirstlane_b32 s45, v3
-; SI-NEXT: v_writelane_b32 v44, s60, 10
+; SI-NEXT: v_readfirstlane_b32 s68, v3
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
; SI-NEXT: s_and_b64 s[4:5], vcc, exec
@@ -83313,73 +83310,70 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readfirstlane_b32 s40, v34
; SI-NEXT: v_readfirstlane_b32 s41, v35
; SI-NEXT: v_readfirstlane_b32 s13, v36
-; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(12)
; SI-NEXT: v_readfirstlane_b32 s5, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s76, v39
-; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s31, v48
-; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s73, v49
-; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s77, v39
+; SI-NEXT: v_writelane_b32 v44, s77, 11
+; SI-NEXT: v_writelane_b32 v44, s5, 12
+; SI-NEXT: v_readfirstlane_b32 s15, v37
; SI-NEXT: s_waitcnt vmcnt(7)
; SI-NEXT: v_readfirstlane_b32 s44, v51
-; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: v_readfirstlane_b32 s31, v48
; SI-NEXT: s_waitcnt vmcnt(5)
; SI-NEXT: v_readfirstlane_b32 s43, v53
-; SI-NEXT: s_waitcnt vmcnt(4)
+; SI-NEXT: v_readfirstlane_b32 s73, v49
+; SI-NEXT: v_readfirstlane_b32 s75, v50
+; SI-NEXT: v_readfirstlane_b32 s42, v52
+; SI-NEXT: s_waitcnt vmcnt(1)
+; SI-NEXT: v_readfirstlane_b32 s76, v41
+; SI-NEXT: v_writelane_b32 v44, s76, 13
+; SI-NEXT: v_writelane_b32 v44, s44, 14
; SI-NEXT: v_readfirstlane_b32 s65, v54
-; SI-NEXT: s_waitcnt vmcnt(3)
; SI-NEXT: v_readfirstlane_b32 s66, v55
-; SI-NEXT: s_waitcnt vmcnt(2)
; SI-NEXT: v_readfirstlane_b32 s55, v40
-; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s82, v41
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: v_readfirstlane_b32 s64, v42
-; SI-NEXT: v_writelane_b32 v44, s76, 11
+; SI-NEXT: v_writelane_b32 v44, s43, 15
+; SI-NEXT: v_writelane_b32 v44, s42, 16
; SI-NEXT: s_cbranch_scc0 .LBB107_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s4, s16, 0xff
; SI-NEXT: s_lshl_b32 s6, s17, 8
-; SI-NEXT: s_or_b32 vcc_hi, s4, s6
+; SI-NEXT: s_or_b32 s88, s4, s6
; SI-NEXT: s_and_b32 s4, s18, 0xff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_lshl_b32 s6, s19, 24
; SI-NEXT: s_or_b32 s4, s6, s4
; SI-NEXT: s_and_b32 s6, s24, 0xff
-; SI-NEXT: s_lshl_b32 s62, s25, 8
-; SI-NEXT: s_or_b32 s67, s6, s62
+; SI-NEXT: s_lshl_b32 s60, s25, 8
+; SI-NEXT: s_or_b32 s89, s6, s60
; SI-NEXT: s_and_b32 s6, s26, 0xff
; SI-NEXT: s_and_b32 s72, s72, 0xff
; SI-NEXT: s_lshl_b32 s58, s58, 8
; SI-NEXT: s_and_b32 s56, s56, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s62, s27, 24
+; SI-NEXT: s_lshl_b32 s60, s27, 24
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_or_b32 s58, s72, s58
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s9, s9, 24
-; SI-NEXT: s_or_b32 s6, s62, s6
-; SI-NEXT: s_and_b32 s60, s60, 0xff
-; SI-NEXT: s_lshl_b32 s62, s45, 8
+; SI-NEXT: s_or_b32 s6, s60, s6
+; SI-NEXT: s_and_b32 s45, s45, 0xff
+; SI-NEXT: s_lshl_b32 s60, s68, 8
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_and_b32 s58, s58, 0xffff
-; SI-NEXT: s_or_b32 s87, s9, s56
-; SI-NEXT: s_or_b32 s70, s60, s62
+; SI-NEXT: s_or_b32 s81, s9, s56
+; SI-NEXT: s_or_b32 s45, s45, s60
; SI-NEXT: s_or_b32 s8, s8, s10
; SI-NEXT: s_and_b32 s10, s36, 0xff
; SI-NEXT: s_lshl_b32 s60, s35, 8
-; SI-NEXT: s_or_b32 s9, s58, s87
+; SI-NEXT: s_or_b32 s9, s58, s81
; SI-NEXT: s_and_b32 s56, s57, 0xff
; SI-NEXT: s_lshl_b32 s58, s47, 8
; SI-NEXT: s_and_b32 s46, s46, 0xff
-; SI-NEXT: s_or_b32 s71, s10, s60
+; SI-NEXT: s_or_b32 s90, s10, s60
; SI-NEXT: s_and_b32 s10, s59, 0xff
; SI-NEXT: s_or_b32 s56, s56, s58
; SI-NEXT: s_lshl_b32 s46, s46, 16
@@ -83387,14 +83381,14 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s10, s10, 16
; SI-NEXT: s_lshl_b32 s60, s30, 24
; SI-NEXT: s_and_b32 s56, s56, 0xffff
-; SI-NEXT: s_or_b32 s96, s11, s46
+; SI-NEXT: s_or_b32 s82, s11, s46
; SI-NEXT: s_or_b32 s10, s60, s10
; SI-NEXT: s_and_b32 s60, s50, 0xff
; SI-NEXT: s_lshl_b32 s62, s49, 8
-; SI-NEXT: s_or_b32 s11, s56, s96
+; SI-NEXT: s_or_b32 s11, s56, s82
; SI-NEXT: s_and_b32 s46, s38, 0xff
; SI-NEXT: s_lshl_b32 s56, s37, 8
-; SI-NEXT: s_or_b32 s80, s60, s62
+; SI-NEXT: s_or_b32 vcc_hi, s60, s62
; SI-NEXT: s_and_b32 s60, s39, 0xff
; SI-NEXT: s_or_b32 s46, s46, s56
; SI-NEXT: s_and_b32 s56, s61, 0xff
@@ -83404,102 +83398,95 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s58, s34, 24
; SI-NEXT: s_or_b32 s60, s62, s60
; SI-NEXT: s_and_b32 s62, s54, 0xff
-; SI-NEXT: s_lshl_b32 s77, s53, 8
+; SI-NEXT: s_lshl_b32 s78, s53, 8
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s97, s58, s56
-; SI-NEXT: s_or_b32 s81, s62, s77
+; SI-NEXT: s_or_b32 s83, s58, s56
+; SI-NEXT: s_or_b32 s67, s62, s78
; SI-NEXT: s_and_b32 s62, s51, 0xff
-; SI-NEXT: s_and_b32 s76, s76, 0xff
-; SI-NEXT: s_mov_b32 s79, s5
+; SI-NEXT: s_and_b32 s77, s77, 0xff
; SI-NEXT: s_lshl_b32 s5, s5, 8
; SI-NEXT: s_mov_b32 s72, s61
-; SI-NEXT: s_or_b32 s61, s46, s97
+; SI-NEXT: s_or_b32 s61, s46, s83
; SI-NEXT: s_and_b32 s46, s63, 0xff
; SI-NEXT: s_lshl_b32 s56, s64, 8
; SI-NEXT: s_lshl_b32 s62, s62, 16
-; SI-NEXT: s_lshl_b32 s77, s52, 24
-; SI-NEXT: s_or_b32 s84, s76, s5
+; SI-NEXT: s_lshl_b32 s78, s52, 24
+; SI-NEXT: s_or_b32 s70, s77, s5
; SI-NEXT: s_and_b32 s5, s20, 0xff
-; SI-NEXT: s_lshl_b32 s76, s21, 8
+; SI-NEXT: s_lshl_b32 s77, s21, 8
; SI-NEXT: s_or_b32 s46, s46, s56
-; SI-NEXT: s_and_b32 s56, s82, 0xff
-; SI-NEXT: s_or_b32 s62, s77, s62
-; SI-NEXT: s_and_b32 s77, s66, 0xff
-; SI-NEXT: s_lshl_b32 s78, s65, 8
-; SI-NEXT: s_or_b32 s5, s5, s76
-; SI-NEXT: s_and_b32 s76, s22, 0xff
+; SI-NEXT: s_and_b32 s56, s76, 0xff
+; SI-NEXT: s_or_b32 s62, s78, s62
+; SI-NEXT: s_and_b32 s78, s66, 0xff
+; SI-NEXT: s_lshl_b32 s79, s65, 8
+; SI-NEXT: s_or_b32 s5, s5, s77
+; SI-NEXT: s_and_b32 s77, s22, 0xff
; SI-NEXT: s_and_b32 s74, s74, 0xff
; SI-NEXT: s_lshl_b32 s56, s56, 16
; SI-NEXT: s_lshl_b32 s58, s55, 24
; SI-NEXT: s_mov_b32 s91, s14
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_or_b32 s83, s77, s78
-; SI-NEXT: s_lshl_b32 s76, s76, 16
-; SI-NEXT: s_lshl_b32 s77, s23, 24
+; SI-NEXT: s_or_b32 s69, s78, s79
+; SI-NEXT: s_lshl_b32 s77, s77, 16
+; SI-NEXT: s_lshl_b32 s78, s23, 24
; SI-NEXT: s_lshl_b32 s74, s74, 16
; SI-NEXT: s_lshl_b32 s7, s7, 24
; SI-NEXT: s_and_b32 s46, s46, 0xffff
-; SI-NEXT: s_or_b32 s98, s58, s56
+; SI-NEXT: s_or_b32 s84, s58, s56
; SI-NEXT: s_lshl_b32 s14, s14, 16
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_lshl_b32 s12, s12, 24
-; SI-NEXT: s_or_b32 s85, s77, s76
-; SI-NEXT: s_and_b32 s76, s28, 0xff
-; SI-NEXT: s_lshl_b32 s77, s29, 8
-; SI-NEXT: s_or_b32 s86, s7, s74
+; SI-NEXT: s_or_b32 s71, s78, s77
+; SI-NEXT: s_and_b32 s77, s28, 0xff
+; SI-NEXT: s_lshl_b32 s78, s29, 8
+; SI-NEXT: s_or_b32 s80, s7, s74
; SI-NEXT: s_mov_b32 s74, s63
-; SI-NEXT: s_or_b32 s63, s46, s98
-; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_or_b32 s63, s46, s84
; SI-NEXT: s_and_b32 s44, s44, 0xff
; SI-NEXT: s_lshl_b32 s46, s75, 8
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_and_b32 s41, s41, 0xff
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_lshl_b32 s40, s40, 8
-; SI-NEXT: s_or_b32 s78, s12, s14
+; SI-NEXT: s_or_b32 s86, s12, s14
; SI-NEXT: s_and_b32 s12, s15, 0xff
-; SI-NEXT: s_or_b32 s76, s76, s77
+; SI-NEXT: s_or_b32 s77, s77, s78
; SI-NEXT: s_or_b32 s44, s44, s46
; SI-NEXT: s_and_b32 s46, s73, 0xff
-; SI-NEXT: s_mov_b32 s89, s43
; SI-NEXT: s_and_b32 s43, s43, 0xff
; SI-NEXT: s_or_b32 s40, s41, s40
; SI-NEXT: s_lshl_b32 s12, s12, 16
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_lshl_b32 s13, s13, 24
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_and_b32 s76, s76, 0xffff
+; SI-NEXT: s_and_b32 s77, s77, 0xffff
; SI-NEXT: s_lshl_b32 s46, s46, 16
; SI-NEXT: s_lshl_b32 s56, s31, 24
; SI-NEXT: s_lshl_b32 s43, s43, 16
-; SI-NEXT: s_mov_b32 s90, s42
; SI-NEXT: s_lshl_b32 s42, s42, 24
-; SI-NEXT: s_or_b32 s68, s13, s12
+; SI-NEXT: s_or_b32 s78, s13, s12
; SI-NEXT: s_and_b32 s12, s40, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s85
-; SI-NEXT: s_or_b32 s7, s76, s86
-; SI-NEXT: s_or_b32 s99, s56, s46
+; SI-NEXT: s_or_b32 s5, s5, s71
+; SI-NEXT: s_or_b32 s7, s77, s80
+; SI-NEXT: s_or_b32 s85, s56, s46
; SI-NEXT: s_or_b32 s76, s42, s43
; SI-NEXT: s_and_b32 s42, s44, 0xffff
-; SI-NEXT: s_or_b32 s69, s12, s78
-; SI-NEXT: s_and_b32 s12, vcc_hi, 0xffff
-; SI-NEXT: s_or_b32 s77, s42, s99
+; SI-NEXT: s_or_b32 s79, s12, s86
+; SI-NEXT: s_and_b32 s12, s88, 0xffff
+; SI-NEXT: s_or_b32 s77, s42, s85
+; SI-NEXT: s_and_b32 s14, s45, 0xffff
; SI-NEXT: s_or_b32 s42, s12, s4
-; SI-NEXT: s_mov_b32 s12, s45
; SI-NEXT: s_lshr_b64 s[44:45], s[4:5], 16
; SI-NEXT: s_mov_b32 s4, s47
; SI-NEXT: s_lshr_b64 s[46:47], s[6:7], 16
-; SI-NEXT: s_and_b32 s13, s67, 0xffff
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
-; SI-NEXT: s_and_b32 s58, s71, 0xffff
-; SI-NEXT: s_and_b32 vcc_hi, s80, 0xffff
-; SI-NEXT: s_and_b32 s67, s81, 0xffff
+; SI-NEXT: s_and_b32 s13, s89, 0xffff
+; SI-NEXT: s_and_b32 s58, s90, 0xffff
+; SI-NEXT: s_and_b32 s88, vcc_hi, 0xffff
+; SI-NEXT: s_and_b32 s89, s67, 0xffff
; SI-NEXT: s_mov_b32 s47, s4
; SI-NEXT: s_mov_b32 s4, s57
; SI-NEXT: s_lshr_b64 s[56:57], s[8:9], 16
; SI-NEXT: s_mov_b32 s95, s15
-; SI-NEXT: s_and_b32 s70, s83, 0xffff
-; SI-NEXT: s_mov_b32 s45, s12
; SI-NEXT: s_or_b32 s40, s13, s6
; SI-NEXT: s_or_b32 s14, s14, s8
; SI-NEXT: s_mov_b32 s15, s9
@@ -83508,62 +83495,59 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_mov_b32 s13, s11
; SI-NEXT: s_mov_b32 s4, s59
; SI-NEXT: s_lshr_b64 s[58:59], s[10:11], 16
-; SI-NEXT: s_or_b32 s10, vcc_hi, s60
+; SI-NEXT: s_or_b32 s10, s88, s60
; SI-NEXT: s_mov_b32 s11, s61
; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_or_b32 s8, s67, s62
+; SI-NEXT: s_or_b32 s8, s89, s62
; SI-NEXT: s_mov_b32 s9, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
-; SI-NEXT: s_and_b32 s71, s84, 0xffff
+; SI-NEXT: s_and_b32 s90, s69, 0xffff
+; SI-NEXT: s_and_b32 vcc_hi, s70, 0xffff
; SI-NEXT: s_mov_b32 s59, s4
; SI-NEXT: s_mov_b32 s61, s72
; SI-NEXT: s_mov_b32 s63, s74
-; SI-NEXT: s_or_b32 s6, s70, s76
; SI-NEXT: s_mov_b32 s4, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[76:77], 16
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[68:69], 16
+; SI-NEXT: s_mov_b32 s45, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[78:79], 16
; SI-NEXT: s_mov_b32 s43, s5
; SI-NEXT: s_mov_b32 s41, s7
+; SI-NEXT: s_or_b32 s6, s90, s76
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_mov_b32 s73, s4
-; SI-NEXT: s_or_b32 s4, s71, s68
-; SI-NEXT: s_mov_b32 s5, s69
-; SI-NEXT: s_mov_b32 s75, s76
-; SI-NEXT: s_lshr_b32 s71, s85, 16
-; SI-NEXT: s_lshr_b32 s70, s86, 16
-; SI-NEXT: s_lshr_b32 s68, s87, 16
-; SI-NEXT: s_lshr_b32 s69, s96, 16
-; SI-NEXT: s_lshr_b32 vcc_hi, s97, 16
-; SI-NEXT: s_lshr_b32 s67, s98, 16
-; SI-NEXT: s_lshr_b32 s80, s99, 16
-; SI-NEXT: s_lshr_b32 s81, s78, 16
+; SI-NEXT: s_or_b32 s4, vcc_hi, s78
+; SI-NEXT: s_mov_b32 s5, s79
+; SI-NEXT: s_mov_b32 s75, s45
+; SI-NEXT: s_lshr_b32 s89, s71, 16
+; SI-NEXT: s_lshr_b32 s88, s80, 16
+; SI-NEXT: s_lshr_b32 s79, s81, 16
+; SI-NEXT: s_lshr_b32 s78, s82, 16
+; SI-NEXT: s_lshr_b32 vcc_hi, s83, 16
+; SI-NEXT: s_lshr_b32 s67, s84, 16
+; SI-NEXT: s_lshr_b32 s90, s85, 16
+; SI-NEXT: s_lshr_b32 s45, s86, 16
; SI-NEXT: s_mov_b64 s[76:77], 0
; SI-NEXT: s_branch .LBB107_3
; SI-NEXT: .LBB107_2:
-; SI-NEXT: s_mov_b32 s79, s5
; SI-NEXT: s_mov_b32 vcc_lo, s13
; SI-NEXT: s_mov_b32 s95, s15
; SI-NEXT: s_mov_b32 s92, s41
; SI-NEXT: s_mov_b32 s93, s40
; SI-NEXT: s_mov_b32 s94, s12
; SI-NEXT: s_mov_b32 s91, s14
-; SI-NEXT: s_mov_b32 s90, s42
-; SI-NEXT: s_mov_b32 s89, s43
-; SI-NEXT: s_mov_b32 s88, s44
; SI-NEXT: s_mov_b64 s[76:77], -1
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr71
+; SI-NEXT: ; implicit-def: $sgpr89
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr88
; SI-NEXT: ; implicit-def: $sgpr14
; SI-NEXT: ; implicit-def: $sgpr56
-; SI-NEXT: ; implicit-def: $sgpr68
+; SI-NEXT: ; implicit-def: $sgpr79
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr58
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr10
; SI-NEXT: ; implicit-def: $sgpr60
; SI-NEXT: ; implicit-def: $vcc_hi
@@ -83572,10 +83556,10 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; implicit-def: $sgpr67
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr80
+; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr4
; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr81
+; SI-NEXT: ; implicit-def: $sgpr45
; SI-NEXT: .LBB107_3: ; %Flow
; SI-NEXT: s_and_b64 s[76:77], s[76:77], exec
; SI-NEXT: s_cselect_b32 s76, 1, 0
@@ -83584,10 +83568,11 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: v_readlane_b32 s4, v44, 11
; SI-NEXT: s_add_i32 s4, s4, 3
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s5, s79, 8
+; SI-NEXT: s_and_b32 s5, s4, 0xff
+; SI-NEXT: v_readlane_b32 s4, v44, 12
+; SI-NEXT: s_lshl_b32 s4, s4, 8
; SI-NEXT: s_add_i32 s6, s95, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_addk_i32 s4, 0x300
; SI-NEXT: s_lshl_b32 s5, vcc_lo, 24
@@ -83610,16 +83595,19 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s6, s66, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s7, s65, 8
-; SI-NEXT: s_add_i32 s8, s89, 3
; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_and_b32 s8, s8, 0xff
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s7, s90, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_or_b32 s6, s7, s6
-; SI-NEXT: s_add_i32 s7, s88, 3
+; SI-NEXT: s_and_b32 s7, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_lshl_b32 s8, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s8, s6
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: v_readlane_b32 s7, v44, 14
+; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xff
; SI-NEXT: s_lshl_b32 s8, s75, 8
; SI-NEXT: s_add_i32 s9, s73, 3
@@ -83646,15 +83634,16 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s9, s63, 3
; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_lshl_b32 s10, s64, 8
-; SI-NEXT: s_add_i32 s82, s82, 3
; SI-NEXT: s_or_b32 s9, s10, s9
-; SI-NEXT: s_and_b32 s11, s82, 0xff
; SI-NEXT: s_addk_i32 s9, 0x300
-; SI-NEXT: s_lshl_b32 s10, s55, 24
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_or_b32 s10, s10, s11
-; SI-NEXT: s_or_b32 s9, s10, s9
+; SI-NEXT: s_and_b32 s10, s9, 0xffff
+; SI-NEXT: v_readlane_b32 s9, v44, 13
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_lshl_b32 s11, s55, 24
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s9, s11, s9
+; SI-NEXT: s_or_b32 s9, s9, s10
; SI-NEXT: s_add_i32 s10, s50, 3
; SI-NEXT: s_and_b32 s10, s10, 0xff
; SI-NEXT: s_lshl_b32 s11, s49, 8
@@ -83708,7 +83697,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_readlane_b32 s14, v44, 10
; SI-NEXT: s_add_i32 s14, s14, 3
; SI-NEXT: s_and_b32 s14, s14, 0xff
-; SI-NEXT: s_lshl_b32 s15, s45, 8
+; SI-NEXT: s_lshl_b32 s15, s68, 8
; SI-NEXT: s_or_b32 s14, s15, s14
; SI-NEXT: s_addk_i32 s14, 0x300
; SI-NEXT: s_and_b32 s15, s14, 0xffff
@@ -83745,49 +83734,50 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s15, s15, 16
; SI-NEXT: s_and_b32 s24, s24, 0xffff
; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_or_b32 s15, s41, s15
; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s16, s16, 3
+; SI-NEXT: s_and_b32 s16, s16, 0xff
+; SI-NEXT: s_lshl_b32 s17, s17, 8
+; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s15, s15, s40
; SI-NEXT: s_add_i32 s40, s24, 0x3000000
; SI-NEXT: s_and_b32 s24, s28, 0xff
; SI-NEXT: s_lshl_b32 s25, s29, 8
-; SI-NEXT: s_and_b32 s16, s16, 0xff
-; SI-NEXT: s_lshl_b32 s17, s17, 8
-; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_or_b32 s24, s25, s24
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s25, s24, 0xffff
-; SI-NEXT: v_readlane_b32 s24, v44, 0
+; SI-NEXT: s_addk_i32 s24, 0x300
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_lshl_b32 s26, s24, 24
-; SI-NEXT: v_readlane_b32 s24, v44, 1
+; SI-NEXT: s_and_b32 s25, s24, 0xffff
+; SI-NEXT: v_readlane_b32 s24, v44, 0
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s20, s20, 3
-; SI-NEXT: s_add_i32 s24, s24, 3
+; SI-NEXT: s_lshl_b32 s26, s24, 24
+; SI-NEXT: v_readlane_b32 s24, v44, 1
; SI-NEXT: s_add_i32 s42, s16, 0x3000000
; SI-NEXT: s_and_b32 s16, s20, 0xff
; SI-NEXT: s_lshl_b32 s17, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
-; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s22, 0xff
-; SI-NEXT: s_lshl_b32 s24, s24, 16
+; SI-NEXT: s_and_b32 s24, s24, 0xff
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s23, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_or_b32 s24, s26, s24
+; SI-NEXT: s_lshl_b32 s24, s24, 16
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_or_b32 s24, s26, s24
; SI-NEXT: s_or_b32 s16, s17, s16
+; SI-NEXT: s_or_b32 s24, s24, s25
+; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
@@ -83801,7 +83791,6 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_add_i32 s14, s14, 0x3000000
; SI-NEXT: s_add_i32 s15, s15, 0x3000000
; SI-NEXT: s_add_i32 s41, s24, 0x3000000
-; SI-NEXT: s_add_i32 s43, s16, 0x3000000
; SI-NEXT: s_lshr_b64 s[44:45], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[46:47], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[56:57], s[14:15], 16
@@ -83810,14 +83799,14 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshr_b64 s[62:63], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[72:73], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b32 s71, s43, 16
-; SI-NEXT: s_lshr_b32 s70, s41, 16
-; SI-NEXT: s_lshr_b32 s68, s15, 16
-; SI-NEXT: s_lshr_b32 s69, s13, 16
+; SI-NEXT: s_lshr_b32 s89, s43, 16
+; SI-NEXT: s_lshr_b32 s88, s41, 16
+; SI-NEXT: s_lshr_b32 s79, s15, 16
+; SI-NEXT: s_lshr_b32 s78, s13, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s11, 16
; SI-NEXT: s_lshr_b32 s67, s9, 16
-; SI-NEXT: s_lshr_b32 s80, s7, 16
-; SI-NEXT: s_lshr_b32 s81, s5, 16
+; SI-NEXT: s_lshr_b32 s90, s7, 16
+; SI-NEXT: s_lshr_b32 s45, s5, 16
; SI-NEXT: .LBB107_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
@@ -83826,25 +83815,25 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s17, s44, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s71, 16
+; SI-NEXT: s_lshl_b32 s18, s89, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s40, 0xffff
; SI-NEXT: s_lshl_b32 s19, s46, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s41, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s70, 16
+; SI-NEXT: s_lshl_b32 s20, s88, 16
; SI-NEXT: s_or_b32 s19, s19, s20
; SI-NEXT: s_and_b32 s14, s14, 0xffff
; SI-NEXT: s_lshl_b32 s20, s56, 16
; SI-NEXT: s_or_b32 s14, s14, s20
; SI-NEXT: s_and_b32 s15, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s68, 16
+; SI-NEXT: s_lshl_b32 s20, s79, 16
; SI-NEXT: s_or_b32 s15, s15, s20
; SI-NEXT: s_and_b32 s12, s12, 0xffff
; SI-NEXT: s_lshl_b32 s20, s58, 16
; SI-NEXT: s_or_b32 s12, s12, s20
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s69, 16
+; SI-NEXT: s_lshl_b32 s20, s78, 16
; SI-NEXT: s_or_b32 s13, s13, s20
; SI-NEXT: s_and_b32 s10, s10, 0xffff
; SI-NEXT: s_lshl_b32 s20, s60, 16
@@ -83862,15 +83851,15 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: s_lshl_b32 s20, s72, 16
; SI-NEXT: s_or_b32 s6, s6, s20
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s80, 16
+; SI-NEXT: s_lshl_b32 s20, s90, 16
; SI-NEXT: s_or_b32 s7, s7, s20
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s20, s74, 16
; SI-NEXT: s_or_b32 s4, s4, s20
; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_lshl_b32 s20, s81, 16
+; SI-NEXT: s_lshl_b32 s20, s45, 16
; SI-NEXT: s_or_b32 s5, s5, s20
-; SI-NEXT: v_readlane_b32 s30, v43, 34
+; SI-NEXT: v_readlane_b32 s30, v43, 29
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s18
@@ -83887,12 +83876,7 @@ define inreg <32 x half> @bitcast_v64i8_to_v32f16_scalar(<64 x i8> inreg %a, i32
; SI-NEXT: v_mov_b32_e32 v13, s7
; SI-NEXT: v_mov_b32_e32 v14, s4
; SI-NEXT: v_mov_b32_e32 v15, s5
-; SI-NEXT: v_readlane_b32 s31, v43, 35
-; SI-NEXT: v_readlane_b32 s99, v43, 33
-; SI-NEXT: v_readlane_b32 s98, v43, 32
-; SI-NEXT: v_readlane_b32 s97, v43, 31
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
+; SI-NEXT: v_readlane_b32 s31, v43, 30
; SI-NEXT: v_readlane_b32 s86, v43, 28
; SI-NEXT: v_readlane_b32 s85, v43, 27
; SI-NEXT: v_readlane_b32 s84, v43, 26
@@ -93878,26 +93862,8 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_writelane_b32 v43, s52, 10
; SI-NEXT: v_writelane_b32 v43, s53, 11
; SI-NEXT: v_writelane_b32 v43, s54, 12
-; SI-NEXT: v_writelane_b32 v43, s55, 13
-; SI-NEXT: v_writelane_b32 v43, s64, 14
-; SI-NEXT: v_writelane_b32 v43, s65, 15
-; SI-NEXT: v_writelane_b32 v43, s66, 16
-; SI-NEXT: v_writelane_b32 v43, s67, 17
-; SI-NEXT: v_writelane_b32 v43, s68, 18
-; SI-NEXT: v_writelane_b32 v43, s69, 19
-; SI-NEXT: v_writelane_b32 v43, s70, 20
-; SI-NEXT: v_writelane_b32 v43, s71, 21
-; SI-NEXT: v_writelane_b32 v43, s80, 22
-; SI-NEXT: v_writelane_b32 v43, s81, 23
-; SI-NEXT: v_writelane_b32 v43, s82, 24
-; SI-NEXT: v_writelane_b32 v43, s83, 25
-; SI-NEXT: v_writelane_b32 v43, s84, 26
-; SI-NEXT: v_writelane_b32 v43, s85, 27
-; SI-NEXT: v_writelane_b32 v43, s86, 28
-; SI-NEXT: v_writelane_b32 v43, s87, 29
-; SI-NEXT: v_writelane_b32 v43, s96, 30
-; SI-NEXT: v_writelane_b32 v43, s30, 31
-; SI-NEXT: v_writelane_b32 v43, s31, 32
+; SI-NEXT: v_writelane_b32 v43, s30, 13
+; SI-NEXT: v_writelane_b32 v43, s31, 14
; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:76
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 offset:72
; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:68
@@ -93921,526 +93887,607 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:4
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32
+; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: ; implicit-def: $vgpr44 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s9, v1
-; SI-NEXT: v_readfirstlane_b32 s6, v5
-; SI-NEXT: v_readfirstlane_b32 s7, v4
-; SI-NEXT: v_readfirstlane_b32 s8, v3
+; SI-NEXT: v_readfirstlane_b32 s92, v5
+; SI-NEXT: v_writelane_b32 v44, s91, 0
+; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_readfirstlane_b32 s90, v3
+; SI-NEXT: v_readfirstlane_b32 s94, v9
+; SI-NEXT: v_readfirstlane_b32 s38, v8
+; SI-NEXT: v_readfirstlane_b32 s39, v7
+; SI-NEXT: v_readfirstlane_b32 s95, v13
+; SI-NEXT: v_readfirstlane_b32 s36, v12
+; SI-NEXT: v_readfirstlane_b32 s48, v6
+; SI-NEXT: v_readfirstlane_b32 s93, v11
+; SI-NEXT: v_readfirstlane_b32 s45, v17
+; SI-NEXT: v_readfirstlane_b32 s58, v16
+; SI-NEXT: v_readfirstlane_b32 s37, v10
+; SI-NEXT: v_readfirstlane_b32 s59, v15
+; SI-NEXT: v_readfirstlane_b32 s14, v21
+; SI-NEXT: v_readfirstlane_b32 s40, v20
+; SI-NEXT: v_readfirstlane_b32 s60, v14
+; SI-NEXT: v_readfirstlane_b32 s43, v19
+; SI-NEXT: v_readfirstlane_b32 s10, v25
+; SI-NEXT: v_readfirstlane_b32 s11, v24
+; SI-NEXT: v_readfirstlane_b32 s44, v18
+; SI-NEXT: v_readfirstlane_b32 s12, v23
+; SI-NEXT: v_readfirstlane_b32 s13, v22
+; SI-NEXT: v_readfirstlane_b32 s7, v28
+; SI-NEXT: v_readfirstlane_b32 s8, v27
; SI-NEXT: v_readfirstlane_b32 s89, v30
-; SI-NEXT: v_readfirstlane_b32 s63, v29
-; SI-NEXT: v_readfirstlane_b32 s72, v28
-; SI-NEXT: v_readfirstlane_b32 s75, v27
-; SI-NEXT: v_readfirstlane_b32 s77, v26
-; SI-NEXT: v_readfirstlane_b32 s59, v25
-; SI-NEXT: v_readfirstlane_b32 s60, v24
-; SI-NEXT: v_readfirstlane_b32 s62, v23
-; SI-NEXT: v_readfirstlane_b32 s73, v22
-; SI-NEXT: v_readfirstlane_b32 s47, v21
-; SI-NEXT: v_readfirstlane_b32 s56, v20
-; SI-NEXT: v_readfirstlane_b32 s58, v19
-; SI-NEXT: v_readfirstlane_b32 s61, v18
-; SI-NEXT: v_readfirstlane_b32 s43, v17
-; SI-NEXT: v_readfirstlane_b32 s44, v16
-; SI-NEXT: v_readfirstlane_b32 s46, v15
-; SI-NEXT: v_readfirstlane_b32 s57, v14
-; SI-NEXT: v_readfirstlane_b32 s15, v13
-; SI-NEXT: v_readfirstlane_b32 s40, v12
-; SI-NEXT: v_readfirstlane_b32 s42, v11
-; SI-NEXT: v_readfirstlane_b32 s45, v10
-; SI-NEXT: v_readfirstlane_b32 s5, v9
-; SI-NEXT: v_readfirstlane_b32 s87, v8
-; SI-NEXT: v_readfirstlane_b32 s14, v7
-; SI-NEXT: v_readfirstlane_b32 s41, v6
-; SI-NEXT: v_readfirstlane_b32 s13, v2
+; SI-NEXT: v_readfirstlane_b32 s6, v29
+; SI-NEXT: v_readfirstlane_b32 s9, v26
; SI-NEXT: s_waitcnt vmcnt(14)
; SI-NEXT: v_cmp_ne_u32_e32 vcc, 0, v31
-; SI-NEXT: s_and_b64 s[10:11], vcc, exec
-; SI-NEXT: v_readfirstlane_b32 s10, v0
-; SI-NEXT: v_writelane_b32 v44, s10, 0
-; SI-NEXT: v_writelane_b32 v44, s9, 1
-; SI-NEXT: v_writelane_b32 v44, s6, 2
-; SI-NEXT: v_writelane_b32 v44, s7, 3
-; SI-NEXT: v_readfirstlane_b32 s55, v32
-; SI-NEXT: v_readfirstlane_b32 s54, v33
-; SI-NEXT: v_readfirstlane_b32 s64, v34
-; SI-NEXT: v_readfirstlane_b32 s65, v35
-; SI-NEXT: v_readfirstlane_b32 s51, v36
+; SI-NEXT: s_and_b64 s[4:5], vcc, exec
+; SI-NEXT: v_readfirstlane_b32 vcc_hi, v0
+; SI-NEXT: v_writelane_b32 v44, vcc_hi, 1
+; SI-NEXT: v_writelane_b32 v44, s92, 2
+; SI-NEXT: v_writelane_b32 v44, s76, 3
+; SI-NEXT: v_writelane_b32 v44, s90, 4
+; SI-NEXT: v_writelane_b32 v44, s94, 5
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v44, s38, 6
+; SI-NEXT: v_writelane_b32 v44, vcc_lo, 7
+; SI-NEXT: v_writelane_b32 v44, s39, 8
+; SI-NEXT: v_writelane_b32 v44, s95, 9
+; SI-NEXT: v_writelane_b32 v44, s36, 10
+; SI-NEXT: v_writelane_b32 v44, s48, 11
+; SI-NEXT: v_writelane_b32 v44, s93, 12
+; SI-NEXT: v_writelane_b32 v44, s45, 13
+; SI-NEXT: v_writelane_b32 v44, s58, 14
+; SI-NEXT: v_writelane_b32 v44, s37, 15
+; SI-NEXT: v_writelane_b32 v44, s59, 16
+; SI-NEXT: v_writelane_b32 v44, s14, 17
+; SI-NEXT: v_writelane_b32 v44, s40, 18
+; SI-NEXT: v_writelane_b32 v44, s60, 19
+; SI-NEXT: v_writelane_b32 v44, s43, 20
+; SI-NEXT: v_writelane_b32 v44, s10, 21
+; SI-NEXT: v_writelane_b32 v44, s11, 22
+; SI-NEXT: v_writelane_b32 v44, s44, 23
+; SI-NEXT: v_writelane_b32 v44, s12, 24
+; SI-NEXT: v_writelane_b32 v44, s13, 25
+; SI-NEXT: v_writelane_b32 v44, s7, 26
+; SI-NEXT: v_readfirstlane_b32 s4, v32
+; SI-NEXT: v_readfirstlane_b32 s5, v33
+; SI-NEXT: v_readfirstlane_b32 s47, v34
+; SI-NEXT: v_readfirstlane_b32 s57, v35
+; SI-NEXT: v_readfirstlane_b32 s46, v36
; SI-NEXT: s_waitcnt vmcnt(13)
-; SI-NEXT: v_readfirstlane_b32 s50, v37
+; SI-NEXT: v_readfirstlane_b32 s61, v37
; SI-NEXT: s_waitcnt vmcnt(12)
-; SI-NEXT: v_readfirstlane_b32 s52, v38
+; SI-NEXT: v_readfirstlane_b32 s41, v38
; SI-NEXT: s_waitcnt vmcnt(11)
-; SI-NEXT: v_readfirstlane_b32 s53, v39
+; SI-NEXT: v_readfirstlane_b32 s42, v39
; SI-NEXT: s_waitcnt vmcnt(10)
-; SI-NEXT: v_readfirstlane_b32 s39, v48
+; SI-NEXT: v_readfirstlane_b32 s56, v48
; SI-NEXT: s_waitcnt vmcnt(9)
-; SI-NEXT: v_readfirstlane_b32 s38, v49
+; SI-NEXT: v_readfirstlane_b32 s62, v49
; SI-NEXT: s_waitcnt vmcnt(8)
-; SI-NEXT: v_readfirstlane_b32 s48, v50
+; SI-NEXT: v_readfirstlane_b32 s15, v50
; SI-NEXT: s_waitcnt vmcnt(7)
-; SI-NEXT: v_readfirstlane_b32 s49, v51
+; SI-NEXT: v_readfirstlane_b32 s63, v51
; SI-NEXT: s_waitcnt vmcnt(6)
-; SI-NEXT: v_readfirstlane_b32 s35, v52
+; SI-NEXT: v_readfirstlane_b32 s74, v52
; SI-NEXT: s_waitcnt vmcnt(5)
-; SI-NEXT: v_readfirstlane_b32 s34, v53
+; SI-NEXT: v_readfirstlane_b32 s75, v53
; SI-NEXT: s_waitcnt vmcnt(4)
-; SI-NEXT: v_readfirstlane_b32 s36, v54
+; SI-NEXT: v_readfirstlane_b32 s78, v54
; SI-NEXT: s_waitcnt vmcnt(3)
-; SI-NEXT: v_readfirstlane_b32 s37, v55
+; SI-NEXT: v_readfirstlane_b32 s30, v55
; SI-NEXT: s_waitcnt vmcnt(2)
-; SI-NEXT: v_readfirstlane_b32 s30, v40
+; SI-NEXT: v_readfirstlane_b32 s31, v40
; SI-NEXT: s_waitcnt vmcnt(1)
-; SI-NEXT: v_readfirstlane_b32 s4, v41
+; SI-NEXT: v_readfirstlane_b32 s34, v41
; SI-NEXT: s_waitcnt vmcnt(0)
-; SI-NEXT: v_readfirstlane_b32 s31, v42
-; SI-NEXT: v_writelane_b32 v44, s8, 4
-; SI-NEXT: v_writelane_b32 v44, s5, 5
+; SI-NEXT: v_readfirstlane_b32 s35, v42
+; SI-NEXT: v_writelane_b32 v44, s8, 27
+; SI-NEXT: v_writelane_b32 v44, s6, 28
; SI-NEXT: s_cbranch_scc0 .LBB111_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s11, s16, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s12, s17, 24
-; SI-NEXT: s_or_b32 s74, s12, s11
-; SI-NEXT: s_and_b32 s11, s18, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s12, s19, 24
-; SI-NEXT: s_or_b32 s76, s12, s11
-; SI-NEXT: s_and_b32 s11, s20, 0xff
-; SI-NEXT: s_lshl_b32 s12, s21, 8
-; SI-NEXT: s_or_b32 s12, s11, s12
-; SI-NEXT: s_and_b32 s11, s22, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s78, s23, 24
-; SI-NEXT: s_or_b32 s79, s78, s11
-; SI-NEXT: s_and_b32 s11, s24, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s78, s25, 24
-; SI-NEXT: s_and_b32 s10, s10, 0xff
-; SI-NEXT: s_or_b32 s78, s78, s11
-; SI-NEXT: s_and_b32 s11, s26, 0xff
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_lshl_b32 s9, s9, 24
+; SI-NEXT: s_and_b32 s72, s16, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s73, s17, 24
+; SI-NEXT: s_or_b32 s73, s73, s72
+; SI-NEXT: s_and_b32 s72, s18, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s77, s19, 24
+; SI-NEXT: s_or_b32 s77, s77, s72
+; SI-NEXT: s_and_b32 s72, s20, 0xff
+; SI-NEXT: s_lshl_b32 s79, s21, 8
+; SI-NEXT: s_or_b32 s49, s72, s79
+; SI-NEXT: s_and_b32 s72, s22, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s79, s23, 24
+; SI-NEXT: s_or_b32 s79, s79, s72
+; SI-NEXT: s_and_b32 s72, s24, 0xff
+; SI-NEXT: s_lshl_b32 s72, s72, 16
+; SI-NEXT: s_lshl_b32 s88, s25, 24
+; SI-NEXT: s_or_b32 s72, s88, s72
+; SI-NEXT: s_and_b32 s88, s26, 0xff
+; SI-NEXT: s_and_b32 vcc_lo, vcc_lo, 0xff
+; SI-NEXT: s_and_b32 s76, s76, 0xff
+; SI-NEXT: s_lshl_b32 s88, s88, 16
+; SI-NEXT: s_lshl_b32 s50, s27, 24
+; SI-NEXT: s_lshl_b32 vcc_lo, vcc_lo, 16
+; SI-NEXT: s_lshl_b32 s90, s90, 24
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s92, s92, 24
+; SI-NEXT: s_or_b32 s88, s50, s88
+; SI-NEXT: s_and_b32 s50, s28, 0xff
+; SI-NEXT: s_lshl_b32 s51, s29, 8
+; SI-NEXT: s_or_b32 s90, s90, vcc_lo
+; SI-NEXT: s_or_b32 s92, s92, s76
+; SI-NEXT: s_and_b32 s76, s48, 0xff
+; SI-NEXT: s_lshl_b32 vcc_lo, s39, 8
+; SI-NEXT: s_or_b32 s50, s50, s51
+; SI-NEXT: s_or_b32 s51, s76, vcc_lo
+; SI-NEXT: s_and_b32 s76, s38, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s94, s94, 24
+; SI-NEXT: s_or_b32 s94, s94, s76
+; SI-NEXT: s_and_b32 s76, s37, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s93, s93, 24
+; SI-NEXT: s_or_b32 s93, s93, s76
+; SI-NEXT: s_and_b32 s76, s36, 0xff
+; SI-NEXT: s_lshl_b32 s76, s76, 16
+; SI-NEXT: s_lshl_b32 s95, s95, 24
+; SI-NEXT: s_or_b32 s95, s95, s76
+; SI-NEXT: s_mov_b32 s76, s9
+; SI-NEXT: s_and_b32 s9, s9, 0xff
; SI-NEXT: s_and_b32 s7, s7, 0xff
-; SI-NEXT: s_lshl_b32 s11, s11, 16
-; SI-NEXT: s_lshl_b32 s88, s27, 24
-; SI-NEXT: s_or_b32 s91, s9, s10
-; SI-NEXT: s_and_b32 s9, s13, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_or_b32 s88, s88, s11
-; SI-NEXT: s_and_b32 s11, s28, 0xff
-; SI-NEXT: s_lshl_b32 s90, s29, 8
; SI-NEXT: s_lshl_b32 s9, s9, 16
; SI-NEXT: s_lshl_b32 s8, s8, 24
-; SI-NEXT: s_or_b32 s92, s6, s7
-; SI-NEXT: s_and_b32 s6, s41, 0xff
-; SI-NEXT: s_lshl_b32 s7, s14, 8
-; SI-NEXT: s_or_b32 s67, s11, s90
-; SI-NEXT: s_or_b32 s90, s8, s9
-; SI-NEXT: s_or_b32 s8, s6, s7
-; SI-NEXT: s_and_b32 s6, s87, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s5, s5, 24
-; SI-NEXT: s_or_b32 s94, s5, s6
-; SI-NEXT: s_and_b32 s5, s45, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s42, 24
-; SI-NEXT: s_or_b32 s93, s6, s5
-; SI-NEXT: s_and_b32 s5, s40, 0xff
-; SI-NEXT: s_lshl_b32 s5, s5, 16
-; SI-NEXT: s_lshl_b32 s6, s15, 24
-; SI-NEXT: s_or_b32 s95, s6, s5
-; SI-NEXT: s_and_b32 s5, s57, 0xff
-; SI-NEXT: s_lshl_b32 s6, s46, 8
-; SI-NEXT: s_or_b32 s5, s5, s6
-; SI-NEXT: s_and_b32 s6, s44, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s43, 24
-; SI-NEXT: s_or_b32 vcc_hi, s7, s6
-; SI-NEXT: s_and_b32 s6, s61, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s58, 24
-; SI-NEXT: s_or_b32 vcc_lo, s7, s6
-; SI-NEXT: s_and_b32 s6, s56, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s47, 24
-; SI-NEXT: s_or_b32 s86, s7, s6
-; SI-NEXT: s_and_b32 s6, s73, 0xff
-; SI-NEXT: s_lshl_b32 s7, s62, 8
-; SI-NEXT: s_or_b32 s10, s6, s7
-; SI-NEXT: s_and_b32 s6, s60, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s59, 24
-; SI-NEXT: s_or_b32 s85, s7, s6
-; SI-NEXT: s_and_b32 s6, s77, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s75, 24
-; SI-NEXT: s_or_b32 s83, s7, s6
-; SI-NEXT: s_and_b32 s6, s72, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_lshl_b32 s7, s63, 24
-; SI-NEXT: s_or_b32 s84, s7, s6
-; SI-NEXT: s_and_b32 s6, s89, 0xff
-; SI-NEXT: s_lshl_b32 s7, s31, 8
-; SI-NEXT: s_or_b32 s81, s6, s7
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s30, 24
-; SI-NEXT: s_or_b32 s82, s7, s4
-; SI-NEXT: s_and_b32 s4, s37, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s36, 24
-; SI-NEXT: s_or_b32 s71, s7, s4
-; SI-NEXT: s_and_b32 s4, s34, 0xff
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_lshl_b32 s7, s35, 24
-; SI-NEXT: s_or_b32 s80, s7, s4
-; SI-NEXT: s_and_b32 s4, s49, 0xff
-; SI-NEXT: s_lshl_b32 s7, s48, 8
-; SI-NEXT: s_or_b32 s4, s4, s7
-; SI-NEXT: s_and_b32 s7, s38, 0xff
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s9, s39, 24
-; SI-NEXT: s_or_b32 s68, s9, s7
-; SI-NEXT: s_and_b32 s7, s53, 0xff
; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_lshl_b32 s9, s52, 24
-; SI-NEXT: s_or_b32 s7, s9, s7
-; SI-NEXT: s_and_b32 s9, s50, 0xff
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_lshl_b32 s11, s51, 24
-; SI-NEXT: s_or_b32 s66, s11, s9
-; SI-NEXT: s_and_b32 s9, s65, 0xff
-; SI-NEXT: s_lshl_b32 s11, s64, 8
-; SI-NEXT: s_or_b32 s96, s9, s11
-; SI-NEXT: s_and_b32 s9, s54, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 24
+; SI-NEXT: s_and_b32 s11, s11, 0xff
+; SI-NEXT: s_or_b32 s36, s8, s9
+; SI-NEXT: s_or_b32 s37, s6, s7
+; SI-NEXT: s_and_b32 s6, s89, 0xff
+; SI-NEXT: s_lshl_b32 s8, s35, 8
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_lshl_b32 s10, s10, 24
+; SI-NEXT: s_or_b32 s53, s6, s8
+; SI-NEXT: s_and_b32 s8, s34, 0xff
+; SI-NEXT: s_or_b32 s39, s10, s11
+; SI-NEXT: s_lshl_b32 s9, s8, 16
+; SI-NEXT: s_lshl_b32 s10, s31, 24
+; SI-NEXT: s_mov_b32 s7, s35
+; SI-NEXT: s_or_b32 s35, s10, s9
+; SI-NEXT: s_and_b32 s9, s30, 0xff
+; SI-NEXT: s_and_b32 s40, s40, 0xff
+; SI-NEXT: s_and_b32 s13, s13, 0xff
+; SI-NEXT: s_lshl_b32 s12, s12, 8
; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_lshl_b32 s11, s55, 24
-; SI-NEXT: s_or_b32 s11, s11, s9
-; SI-NEXT: s_lshl_b32 s69, s12, 16
-; SI-NEXT: s_lshl_b32 s70, s67, 16
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_lshl_b32 s9, s5, 16
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_lshl_b32 s81, s81, 16
-; SI-NEXT: s_lshl_b32 s67, s4, 16
-; SI-NEXT: s_lshl_b32 s12, s96, 16
+; SI-NEXT: s_lshl_b32 s11, s78, 24
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_lshl_b32 s14, s14, 24
+; SI-NEXT: s_or_b32 s52, s13, s12
+; SI-NEXT: s_mov_b32 s12, s30
+; SI-NEXT: s_or_b32 s30, s11, s9
+; SI-NEXT: s_and_b32 s11, s75, 0xff
+; SI-NEXT: s_or_b32 s48, s14, s40
+; SI-NEXT: s_lshl_b32 s13, s11, 16
+; SI-NEXT: s_lshl_b32 s14, s74, 24
+; SI-NEXT: s_and_b32 s44, s44, 0xff
+; SI-NEXT: s_mov_b32 s8, s31
+; SI-NEXT: s_or_b32 s31, s14, s13
+; SI-NEXT: s_and_b32 s13, s63, 0xff
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 8
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_lshl_b32 s43, s43, 24
+; SI-NEXT: s_mov_b32 s10, s78
+; SI-NEXT: s_or_b32 s78, s13, s15
+; SI-NEXT: s_and_b32 s15, s62, 0xff
+; SI-NEXT: s_and_b32 vcc_hi, vcc_hi, 0xff
+; SI-NEXT: s_and_b32 s58, s58, 0xff
+; SI-NEXT: s_or_b32 vcc_lo, s43, s44
+; SI-NEXT: s_lshl_b32 s43, s15, 16
+; SI-NEXT: s_lshl_b32 s44, s56, 24
+; SI-NEXT: s_lshl_b32 vcc_hi, vcc_hi, 16
+; SI-NEXT: s_lshl_b32 s91, s91, 24
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_lshl_b32 s45, s45, 24
+; SI-NEXT: s_mov_b32 s15, s56
+; SI-NEXT: s_or_b32 s56, s44, s43
+; SI-NEXT: s_and_b32 s44, s61, 0xff
+; SI-NEXT: s_or_b32 s91, s91, vcc_hi
+; SI-NEXT: s_and_b32 s60, s60, 0xff
+; SI-NEXT: s_lshl_b32 s59, s59, 8
+; SI-NEXT: s_or_b32 vcc_hi, s45, s58
+; SI-NEXT: s_lshl_b32 s58, s44, 16
+; SI-NEXT: s_mov_b32 s44, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 24
+; SI-NEXT: s_or_b32 s38, s60, s59
+; SI-NEXT: s_or_b32 s46, s46, s58
+; SI-NEXT: s_mov_b32 s59, s57
+; SI-NEXT: s_and_b32 s57, s57, 0xff
+; SI-NEXT: s_mov_b32 s58, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 8
+; SI-NEXT: s_mov_b32 s45, s42
+; SI-NEXT: s_and_b32 s42, s42, 0xff
+; SI-NEXT: s_or_b32 s54, s57, s47
+; SI-NEXT: s_mov_b32 s57, s5
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 24
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_mov_b32 s60, s4
+; SI-NEXT: s_lshl_b32 s4, s4, 24
+; SI-NEXT: s_mov_b32 s6, s34
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_mov_b32 s11, s74
+; SI-NEXT: s_mov_b32 s40, s63
+; SI-NEXT: s_mov_b32 s13, s62
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_mov_b32 s42, s61
+; SI-NEXT: s_or_b32 s75, s4, s5
+; SI-NEXT: s_lshl_b32 s61, s49, 16
+; SI-NEXT: s_lshl_b32 s62, s50, 16
+; SI-NEXT: s_lshl_b32 s63, s51, 16
+; SI-NEXT: s_lshl_b32 s74, s38, 16
+; SI-NEXT: s_lshl_b32 s38, s52, 16
+; SI-NEXT: s_lshl_b32 s34, s53, 16
+; SI-NEXT: s_lshl_b32 s47, s78, 16
+; SI-NEXT: s_lshl_b32 s78, s54, 16
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB111_3
; SI-NEXT: .LBB111_2:
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_mov_b32 s76, s9
+; SI-NEXT: s_mov_b32 s59, s57
+; SI-NEXT: s_mov_b32 s58, s47
+; SI-NEXT: s_mov_b32 s60, s4
+; SI-NEXT: s_mov_b32 s57, s5
+; SI-NEXT: s_mov_b32 s45, s42
+; SI-NEXT: s_mov_b32 s43, s41
+; SI-NEXT: s_mov_b32 s44, s46
+; SI-NEXT: s_mov_b32 s42, s61
+; SI-NEXT: s_mov_b32 s40, s63
+; SI-NEXT: s_mov_b32 s14, s15
+; SI-NEXT: s_mov_b32 s15, s56
+; SI-NEXT: s_mov_b32 s13, s62
+; SI-NEXT: s_mov_b32 s12, s30
+; SI-NEXT: s_mov_b32 s10, s78
+; SI-NEXT: s_mov_b32 s11, s74
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_mov_b32 s7, s35
+; SI-NEXT: s_mov_b32 s8, s31
+; SI-NEXT: s_mov_b32 s6, s34
; SI-NEXT: s_mov_b64 s[4:5], -1
-; SI-NEXT: ; implicit-def: $sgpr74
-; SI-NEXT: ; implicit-def: $sgpr76
-; SI-NEXT: ; implicit-def: $sgpr69
+; SI-NEXT: ; implicit-def: $sgpr73
+; SI-NEXT: ; implicit-def: $sgpr77
+; SI-NEXT: ; implicit-def: $sgpr61
; SI-NEXT: ; implicit-def: $sgpr79
-; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr88
-; SI-NEXT: ; implicit-def: $sgpr70
+; SI-NEXT: ; implicit-def: $sgpr62
; SI-NEXT: ; implicit-def: $sgpr91
; SI-NEXT: ; implicit-def: $sgpr90
; SI-NEXT: ; implicit-def: $sgpr92
-; SI-NEXT: ; implicit-def: $sgpr8
+; SI-NEXT: ; implicit-def: $sgpr63
; SI-NEXT: ; implicit-def: $sgpr94
; SI-NEXT: ; implicit-def: $sgpr93
; SI-NEXT: ; implicit-def: $sgpr95
-; SI-NEXT: ; implicit-def: $sgpr9
+; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $vcc_hi
; SI-NEXT: ; implicit-def: $vcc_lo
-; SI-NEXT: ; implicit-def: $sgpr86
-; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr85
-; SI-NEXT: ; implicit-def: $sgpr83
-; SI-NEXT: ; implicit-def: $sgpr84
-; SI-NEXT: ; implicit-def: $sgpr81
-; SI-NEXT: ; implicit-def: $sgpr82
-; SI-NEXT: ; implicit-def: $sgpr71
-; SI-NEXT: ; implicit-def: $sgpr80
-; SI-NEXT: ; implicit-def: $sgpr67
-; SI-NEXT: ; implicit-def: $sgpr68
-; SI-NEXT: ; implicit-def: $sgpr7
-; SI-NEXT: ; implicit-def: $sgpr66
-; SI-NEXT: ; implicit-def: $sgpr12
-; SI-NEXT: ; implicit-def: $sgpr11
+; SI-NEXT: ; implicit-def: $sgpr48
+; SI-NEXT: ; implicit-def: $sgpr38
+; SI-NEXT: ; implicit-def: $sgpr39
+; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr37
+; SI-NEXT: ; implicit-def: $sgpr34
+; SI-NEXT: ; implicit-def: $sgpr35
+; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr31
+; SI-NEXT: ; implicit-def: $sgpr47
+; SI-NEXT: ; implicit-def: $sgpr56
+; SI-NEXT: ; implicit-def: $sgpr41
+; SI-NEXT: ; implicit-def: $sgpr46
+; SI-NEXT: ; implicit-def: $sgpr78
+; SI-NEXT: ; implicit-def: $sgpr75
; SI-NEXT: .LBB111_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
-; SI-NEXT: s_mov_b32 s5, s11
-; SI-NEXT: s_mov_b32 s11, s10
-; SI-NEXT: s_mov_b32 s10, s9
+; SI-NEXT: s_mov_b32 s5, s75
+; SI-NEXT: s_mov_b32 s75, s74
; SI-NEXT: s_cmp_lg_u32 s4, 1
-; SI-NEXT: s_mov_b32 s9, s8
-; SI-NEXT: s_mov_b32 s8, s10
-; SI-NEXT: s_mov_b32 s10, s69
-; SI-NEXT: s_mov_b32 s69, s11
+; SI-NEXT: s_mov_b32 s74, s61
+; SI-NEXT: s_mov_b32 s61, s75
; SI-NEXT: s_cbranch_scc1 .LBB111_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s65, s65, 3
-; SI-NEXT: s_and_b32 s4, s65, 0xff
-; SI-NEXT: s_lshl_b32 s5, s64, 8
-; SI-NEXT: s_add_i32 s54, s54, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s7, s54, 0xff
-; SI-NEXT: s_addk_i32 s4, 0x300
-; SI-NEXT: s_lshl_b32 s5, s55, 24
-; SI-NEXT: s_lshl_b32 s7, s7, 16
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_or_b32 s5, s5, s7
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_and_b32 s5, s53, 0xff
-; SI-NEXT: s_lshl_b32 s7, s52, 8
-; SI-NEXT: s_add_i32 s50, s50, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s8, s50, 0xff
-; SI-NEXT: s_addk_i32 s5, 0x300
-; SI-NEXT: s_lshl_b32 s7, s51, 24
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
-; SI-NEXT: s_or_b32 s7, s7, s8
-; SI-NEXT: s_add_i32 s49, s49, 3
-; SI-NEXT: s_or_b32 s5, s7, s5
-; SI-NEXT: s_and_b32 s7, s49, 0xff
-; SI-NEXT: s_lshl_b32 s8, s48, 8
-; SI-NEXT: s_add_i32 s38, s38, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s9, s38, 0xff
-; SI-NEXT: s_addk_i32 s7, 0x300
-; SI-NEXT: s_lshl_b32 s8, s39, 24
+; SI-NEXT: s_add_i32 s12, s12, 3
+; SI-NEXT: s_and_b32 s12, s12, 0xff
+; SI-NEXT: s_lshl_b32 s10, s10, 8
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_or_b32 s10, s10, s12
+; SI-NEXT: s_and_b32 s9, s9, 0xff
+; SI-NEXT: s_addk_i32 s10, 0x300
+; SI-NEXT: s_lshl_b32 s11, s11, 24
; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_add_i32 s37, s37, 3
-; SI-NEXT: s_or_b32 s7, s8, s7
-; SI-NEXT: s_and_b32 s8, s37, 0xff
-; SI-NEXT: s_lshl_b32 s9, s36, 8
-; SI-NEXT: s_add_i32 s34, s34, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
-; SI-NEXT: s_and_b32 s10, s34, 0xff
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_lshl_b32 s9, s35, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_or_b32 s8, s9, s8
+; SI-NEXT: s_and_b32 s10, s10, 0xffff
+; SI-NEXT: s_or_b32 s9, s11, s9
; SI-NEXT: s_add_i32 s89, s89, 3
-; SI-NEXT: s_add_i32 s11, s8, 0x3000000
-; SI-NEXT: s_and_b32 s8, s89, 0xff
-; SI-NEXT: s_lshl_b32 s9, s31, 8
+; SI-NEXT: s_or_b32 s9, s9, s10
+; SI-NEXT: s_and_b32 s10, s89, 0xff
+; SI-NEXT: s_lshl_b32 s7, s7, 8
; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_or_b32 s8, s9, s8
+; SI-NEXT: s_or_b32 s7, s7, s10
; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_addk_i32 s8, 0x300
-; SI-NEXT: s_lshl_b32 s9, s30, 24
+; SI-NEXT: s_addk_i32 s7, 0x300
+; SI-NEXT: s_lshl_b32 s8, s8, 24
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_or_b32 s6, s6, s8
-; SI-NEXT: s_add_i32 s77, s77, 3
-; SI-NEXT: s_add_i32 s12, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s77, 0xff
-; SI-NEXT: s_lshl_b32 s8, s75, 8
-; SI-NEXT: s_add_i32 s72, s72, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s72, 0xff
-; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s63, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s73, s73, 3
-; SI-NEXT: s_add_i32 s63, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s73, 0xff
-; SI-NEXT: s_lshl_b32 s8, s62, 8
-; SI-NEXT: s_add_i32 s60, s60, 3
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s60, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s7
+; SI-NEXT: s_add_i32 s7, s6, 0x3000000
+; SI-NEXT: s_add_i32 s6, s76, 3
+; SI-NEXT: s_and_b32 s8, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 27
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s8
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s59, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s61, s61, 3
-; SI-NEXT: s_add_i32 s59, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s61, 0xff
-; SI-NEXT: s_lshl_b32 s8, s58, 8
-; SI-NEXT: s_add_i32 s56, s56, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s56, 0xff
+; SI-NEXT: s_and_b32 s8, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 28
+; SI-NEXT: s_lshl_b32 s10, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 26
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s10, s6
+; SI-NEXT: s_or_b32 s6, s6, s8
+; SI-NEXT: s_add_i32 s8, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 25
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s10, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 24
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s10
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s47, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s57, s57, 3
-; SI-NEXT: s_add_i32 s47, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s57, 0xff
-; SI-NEXT: s_lshl_b32 s8, s46, 8
-; SI-NEXT: s_add_i32 s44, s44, 3
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_and_b32 s9, s44, 0xff
+; SI-NEXT: s_and_b32 s10, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 21
+; SI-NEXT: s_lshl_b32 s11, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 22
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s11, s6
+; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_add_i32 s10, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 23
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s11, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 20
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s11
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s8, s43, 24
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s8, s8, s9
-; SI-NEXT: s_or_b32 s6, s8, s6
-; SI-NEXT: s_add_i32 s45, s45, 3
-; SI-NEXT: s_add_i32 s8, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s45, 0xff
-; SI-NEXT: s_lshl_b32 s9, s42, 8
+; SI-NEXT: s_and_b32 s11, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 17
+; SI-NEXT: s_lshl_b32 s12, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 18
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s12, s6
+; SI-NEXT: s_or_b32 s6, s6, s11
+; SI-NEXT: s_add_i32 s11, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 19
+; SI-NEXT: s_add_i32 s6, s6, 3
; SI-NEXT: s_add_i32 s40, s40, 3
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_and_b32 s10, s40, 0xff
+; SI-NEXT: s_and_b32 s12, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 16
+; SI-NEXT: s_and_b32 s40, s40, 0xff
+; SI-NEXT: s_lshl_b32 s14, s14, 8
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s14, s14, s40
+; SI-NEXT: s_and_b32 s13, s13, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s12
+; SI-NEXT: s_addk_i32 s14, 0x300
+; SI-NEXT: s_lshl_b32 s15, s15, 24
+; SI-NEXT: s_lshl_b32 s13, s13, 16
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_lshl_b32 s9, s15, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s6, s6, 0xffff
-; SI-NEXT: s_or_b32 s9, s9, s10
-; SI-NEXT: s_or_b32 s6, s9, s6
-; SI-NEXT: s_add_i32 s41, s41, 3
-; SI-NEXT: s_add_i32 s15, s6, 0x3000000
-; SI-NEXT: s_and_b32 s6, s41, 0xff
-; SI-NEXT: s_lshl_b32 s9, s14, 8
-; SI-NEXT: s_or_b32 s6, s9, s6
+; SI-NEXT: s_and_b32 s14, s14, 0xffff
+; SI-NEXT: s_or_b32 s13, s15, s13
+; SI-NEXT: s_and_b32 s12, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 13
+; SI-NEXT: s_or_b32 s13, s13, s14
+; SI-NEXT: s_lshl_b32 s14, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 14
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s14, s6
+; SI-NEXT: s_or_b32 s6, s6, s12
+; SI-NEXT: s_add_i32 s12, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 15
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s14, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 12
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s14
+; SI-NEXT: s_addk_i32 s6, 0x300
+; SI-NEXT: s_and_b32 s14, s6, 0xffff
+; SI-NEXT: v_readlane_b32 s6, v44, 9
+; SI-NEXT: s_lshl_b32 s15, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 10
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s6, s15, s6
+; SI-NEXT: s_or_b32 s6, s6, s14
+; SI-NEXT: s_add_i32 s14, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 11
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s15, s6, 0xff
+; SI-NEXT: v_readlane_b32 s6, v44, 8
+; SI-NEXT: s_lshl_b32 s6, s6, 8
+; SI-NEXT: s_or_b32 s6, s6, s15
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_add_i32 s87, s87, 3
-; SI-NEXT: s_and_b32 s9, s6, 0xffff
+; SI-NEXT: s_and_b32 s15, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 5
-; SI-NEXT: s_and_b32 s10, s87, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 24
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_or_b32 s6, s6, s10
-; SI-NEXT: s_or_b32 s6, s6, s9
-; SI-NEXT: s_add_i32 s9, s6, 0x3000000
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_lshl_b32 s40, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 6
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_add_i32 s4, s59, 3
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xff
+; SI-NEXT: s_lshl_b32 s5, s58, 8
+; SI-NEXT: s_add_i32 s41, s57, 3
+; SI-NEXT: s_or_b32 s6, s40, s6
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_and_b32 s41, s41, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s15
+; SI-NEXT: s_addk_i32 s4, 0x300
+; SI-NEXT: s_lshl_b32 s5, s60, 24
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_add_i32 s15, s6, 0x3000000
+; SI-NEXT: v_readlane_b32 s6, v44, 7
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_or_b32 s5, s5, s41
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s5, s45, 3
+; SI-NEXT: s_and_b32 s40, s6, 0xff
; SI-NEXT: v_readlane_b32 s6, v44, 4
-; SI-NEXT: s_and_b32 s10, s13, 0xff
+; SI-NEXT: s_and_b32 s5, s5, 0xff
+; SI-NEXT: s_lshl_b32 s41, s43, 8
+; SI-NEXT: s_add_i32 s42, s42, 3
; SI-NEXT: s_lshl_b32 s6, s6, 8
-; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_or_b32 s5, s41, s5
+; SI-NEXT: s_and_b32 s42, s42, 0xff
+; SI-NEXT: s_or_b32 s6, s6, s40
+; SI-NEXT: s_addk_i32 s5, 0x300
+; SI-NEXT: s_lshl_b32 s41, s44, 24
+; SI-NEXT: s_lshl_b32 s42, s42, 16
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_and_b32 s10, s6, 0xffff
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s40, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 2
-; SI-NEXT: s_lshl_b32 s13, s6, 24
+; SI-NEXT: s_or_b32 s5, s41, s5
+; SI-NEXT: s_lshl_b32 s41, s6, 24
; SI-NEXT: v_readlane_b32 s6, v44, 3
; SI-NEXT: s_add_i32 s6, s6, 3
; SI-NEXT: s_and_b32 s6, s6, 0xff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_or_b32 s6, s13, s6
-; SI-NEXT: s_or_b32 s6, s6, s10
+; SI-NEXT: s_or_b32 s6, s41, s6
+; SI-NEXT: s_or_b32 s6, s6, s40
; SI-NEXT: s_add_i32 s28, s28, 3
-; SI-NEXT: s_add_i32 s13, s6, 0x3000000
+; SI-NEXT: s_add_i32 s40, s6, 0x3000000
; SI-NEXT: s_and_b32 s6, s28, 0xff
-; SI-NEXT: s_lshl_b32 s10, s29, 8
-; SI-NEXT: s_or_b32 s6, s10, s6
+; SI-NEXT: s_lshl_b32 s28, s29, 8
+; SI-NEXT: s_or_b32 s6, s28, s6
; SI-NEXT: s_addk_i32 s6, 0x300
-; SI-NEXT: s_and_b32 s10, s6, 0xffff
-; SI-NEXT: v_readlane_b32 s6, v44, 1
-; SI-NEXT: s_lshl_b32 s14, s6, 24
+; SI-NEXT: s_and_b32 s28, s6, 0xffff
; SI-NEXT: v_readlane_b32 s6, v44, 0
-; SI-NEXT: s_add_i32 s6, s6, 3
-; SI-NEXT: s_and_b32 s6, s6, 0xff
-; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_or_b32 s6, s14, s6
+; SI-NEXT: s_lshl_b32 s29, s6, 24
+; SI-NEXT: v_readlane_b32 s6, v44, 1
; SI-NEXT: s_add_i32 s24, s24, 3
-; SI-NEXT: s_or_b32 s6, s6, s10
-; SI-NEXT: s_and_b32 s10, s24, 0xff
-; SI-NEXT: s_lshl_b32 s14, s25, 8
-; SI-NEXT: s_add_i32 s26, s26, 3
-; SI-NEXT: s_or_b32 s10, s14, s10
-; SI-NEXT: s_and_b32 s24, s26, 0xff
-; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s14, s27, 24
-; SI-NEXT: s_lshl_b32 s24, s24, 16
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_or_b32 s14, s14, s24
-; SI-NEXT: s_or_b32 s10, s14, s10
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s16, s16, 3
-; SI-NEXT: s_add_i32 s14, s10, 0x3000000
-; SI-NEXT: s_and_b32 s10, s20, 0xff
-; SI-NEXT: s_lshl_b32 s20, s21, 8
+; SI-NEXT: s_add_i32 s6, s6, 3
+; SI-NEXT: s_and_b32 s24, s24, 0xff
+; SI-NEXT: s_lshl_b32 s25, s25, 8
+; SI-NEXT: s_add_i32 s26, s26, 3
+; SI-NEXT: s_and_b32 s20, s20, 0xff
+; SI-NEXT: s_lshl_b32 s21, s21, 8
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_and_b32 s16, s16, 0xff
; SI-NEXT: s_lshl_b32 s17, s17, 8
; SI-NEXT: s_add_i32 s18, s18, 3
-; SI-NEXT: s_or_b32 s10, s20, s10
-; SI-NEXT: s_and_b32 s21, s22, 0xff
+; SI-NEXT: s_and_b32 s6, s6, 0xff
+; SI-NEXT: s_or_b32 s24, s25, s24
+; SI-NEXT: s_and_b32 s26, s26, 0xff
+; SI-NEXT: s_or_b32 s20, s21, s20
+; SI-NEXT: s_and_b32 s22, s22, 0xff
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_and_b32 s18, s18, 0xff
-; SI-NEXT: s_addk_i32 s10, 0x300
-; SI-NEXT: s_lshl_b32 s20, s23, 24
-; SI-NEXT: s_lshl_b32 s21, s21, 16
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_addk_i32 s24, 0x300
+; SI-NEXT: s_lshl_b32 s25, s27, 24
+; SI-NEXT: s_lshl_b32 s26, s26, 16
+; SI-NEXT: s_addk_i32 s20, 0x300
+; SI-NEXT: s_lshl_b32 s21, s23, 24
+; SI-NEXT: s_lshl_b32 s22, s22, 16
; SI-NEXT: s_addk_i32 s16, 0x300
; SI-NEXT: s_lshl_b32 s17, s19, 24
; SI-NEXT: s_lshl_b32 s18, s18, 16
-; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_or_b32 s20, s20, s21
+; SI-NEXT: s_or_b32 s6, s29, s6
+; SI-NEXT: s_and_b32 s24, s24, 0xffff
+; SI-NEXT: s_or_b32 s25, s25, s26
+; SI-NEXT: s_and_b32 s20, s20, 0xffff
+; SI-NEXT: s_or_b32 s21, s21, s22
; SI-NEXT: s_and_b32 s16, s16, 0xffff
; SI-NEXT: s_or_b32 s17, s17, s18
-; SI-NEXT: s_or_b32 s10, s20, s10
+; SI-NEXT: s_or_b32 s6, s6, s28
+; SI-NEXT: s_or_b32 s24, s25, s24
+; SI-NEXT: s_or_b32 s20, s21, s20
; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x3000000
; SI-NEXT: s_add_i32 s5, s5, 0x3000000
-; SI-NEXT: s_add_i32 s7, s7, 0x3000000
+; SI-NEXT: s_add_i32 s13, s13, 0x3000000
+; SI-NEXT: s_add_i32 s9, s9, 0x3000000
; SI-NEXT: s_add_i32 s6, s6, 0x3000000
-; SI-NEXT: s_add_i32 s10, s10, 0x3000000
+; SI-NEXT: s_add_i32 s24, s24, 0x3000000
+; SI-NEXT: s_add_i32 s20, s20, 0x3000000
; SI-NEXT: s_add_i32 s16, s16, 0x3000000
-; SI-NEXT: s_and_b32 s76, s16, 0xffff0000
-; SI-NEXT: s_lshl_b32 s74, s16, 16
-; SI-NEXT: s_and_b32 s79, s10, 0xffff0000
-; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_and_b32 s88, s14, 0xffff0000
-; SI-NEXT: s_lshl_b32 s78, s14, 16
+; SI-NEXT: s_and_b32 s77, s16, 0xffff0000
+; SI-NEXT: s_lshl_b32 s73, s16, 16
+; SI-NEXT: s_and_b32 s79, s20, 0xffff0000
+; SI-NEXT: s_lshl_b32 s74, s20, 16
+; SI-NEXT: s_and_b32 s88, s24, 0xffff0000
+; SI-NEXT: s_lshl_b32 s72, s24, 16
; SI-NEXT: s_and_b32 s91, s6, 0xffff0000
-; SI-NEXT: s_lshl_b32 s70, s6, 16
-; SI-NEXT: s_and_b32 s92, s13, 0xffff0000
-; SI-NEXT: s_lshl_b32 s90, s13, 16
-; SI-NEXT: s_and_b32 s94, s9, 0xffff0000
-; SI-NEXT: s_lshl_b32 s9, s9, 16
-; SI-NEXT: s_and_b32 s95, s15, 0xffff0000
-; SI-NEXT: s_lshl_b32 s93, s15, 16
-; SI-NEXT: s_and_b32 vcc_hi, s8, 0xffff0000
-; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_and_b32 s86, s47, 0xffff0000
-; SI-NEXT: s_lshl_b32 vcc_lo, s47, 16
-; SI-NEXT: s_and_b32 s85, s59, 0xffff0000
-; SI-NEXT: s_lshl_b32 s69, s59, 16
-; SI-NEXT: s_and_b32 s84, s63, 0xffff0000
-; SI-NEXT: s_lshl_b32 s83, s63, 16
-; SI-NEXT: s_and_b32 s82, s12, 0xffff0000
-; SI-NEXT: s_lshl_b32 s81, s12, 16
-; SI-NEXT: s_and_b32 s80, s11, 0xffff0000
-; SI-NEXT: s_lshl_b32 s71, s11, 16
-; SI-NEXT: s_and_b32 s68, s7, 0xffff0000
-; SI-NEXT: s_lshl_b32 s67, s7, 16
-; SI-NEXT: s_and_b32 s66, s5, 0xffff0000
-; SI-NEXT: s_lshl_b32 s7, s5, 16
+; SI-NEXT: s_lshl_b32 s62, s6, 16
+; SI-NEXT: s_and_b32 s92, s40, 0xffff0000
+; SI-NEXT: s_lshl_b32 s90, s40, 16
+; SI-NEXT: s_and_b32 s94, s15, 0xffff0000
+; SI-NEXT: s_lshl_b32 s63, s15, 16
+; SI-NEXT: s_and_b32 s95, s14, 0xffff0000
+; SI-NEXT: s_lshl_b32 s93, s14, 16
+; SI-NEXT: s_and_b32 vcc_hi, s12, 0xffff0000
+; SI-NEXT: s_lshl_b32 s61, s12, 16
+; SI-NEXT: s_and_b32 s48, s11, 0xffff0000
+; SI-NEXT: s_lshl_b32 vcc_lo, s11, 16
+; SI-NEXT: s_and_b32 s39, s10, 0xffff0000
+; SI-NEXT: s_lshl_b32 s38, s10, 16
+; SI-NEXT: s_and_b32 s37, s8, 0xffff0000
+; SI-NEXT: s_lshl_b32 s36, s8, 16
+; SI-NEXT: s_and_b32 s35, s7, 0xffff0000
+; SI-NEXT: s_lshl_b32 s34, s7, 16
+; SI-NEXT: s_and_b32 s31, s9, 0xffff0000
+; SI-NEXT: s_lshl_b32 s30, s9, 16
+; SI-NEXT: s_and_b32 s56, s13, 0xffff0000
+; SI-NEXT: s_lshl_b32 s47, s13, 16
+; SI-NEXT: s_and_b32 s46, s5, 0xffff0000
+; SI-NEXT: s_lshl_b32 s41, s5, 16
; SI-NEXT: s_and_b32 s5, s4, 0xffff0000
-; SI-NEXT: s_lshl_b32 s12, s4, 16
+; SI-NEXT: s_lshl_b32 s78, s4, 16
; SI-NEXT: .LBB111_5: ; %end
; SI-NEXT: buffer_load_dword v42, off, s[0:3], s32 offset:80 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v41, off, s[0:3], s32 offset:84 ; 4-byte Folded Reload
; SI-NEXT: buffer_load_dword v40, off, s[0:3], s32 offset:88 ; 4-byte Folded Reload
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s76
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s77
; SI-NEXT: v_lshrrev_b32_e32 v1, 16, v0
-; SI-NEXT: v_mul_f32_e64 v0, 1.0, s74
+; SI-NEXT: v_mul_f32_e64 v0, 1.0, s73
; SI-NEXT: v_lshr_b64 v[0:1], v[0:1], 16
; SI-NEXT: v_mul_f32_e64 v1, 1.0, s79
; SI-NEXT: v_lshrrev_b32_e32 v2, 16, v1
-; SI-NEXT: v_mul_f32_e64 v1, 1.0, s10
+; SI-NEXT: v_mul_f32_e64 v1, 1.0, s74
; SI-NEXT: v_lshr_b64 v[1:2], v[1:2], 16
; SI-NEXT: v_mul_f32_e64 v2, 1.0, s88
; SI-NEXT: v_lshrrev_b32_e32 v3, 16, v2
-; SI-NEXT: v_mul_f32_e64 v2, 1.0, s78
+; SI-NEXT: v_mul_f32_e64 v2, 1.0, s72
; SI-NEXT: v_lshr_b64 v[2:3], v[2:3], 16
; SI-NEXT: v_mul_f32_e64 v3, 1.0, s91
; SI-NEXT: v_lshrrev_b32_e32 v4, 16, v3
-; SI-NEXT: v_mul_f32_e64 v3, 1.0, s70
+; SI-NEXT: v_mul_f32_e64 v3, 1.0, s62
; SI-NEXT: v_lshr_b64 v[3:4], v[3:4], 16
; SI-NEXT: v_mul_f32_e64 v4, 1.0, s92
; SI-NEXT: v_lshrrev_b32_e32 v5, 16, v4
@@ -94448,7 +94495,7 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_lshr_b64 v[4:5], v[4:5], 16
; SI-NEXT: v_mul_f32_e64 v5, 1.0, s94
; SI-NEXT: v_lshrrev_b32_e32 v6, 16, v5
-; SI-NEXT: v_mul_f32_e64 v5, 1.0, s9
+; SI-NEXT: v_mul_f32_e64 v5, 1.0, s63
; SI-NEXT: v_lshr_b64 v[5:6], v[5:6], 16
; SI-NEXT: v_mul_f32_e64 v6, 1.0, s95
; SI-NEXT: v_lshrrev_b32_e32 v7, 16, v6
@@ -94456,60 +94503,42 @@ define inreg <32 x bfloat> @bitcast_v64i8_to_v32bf16_scalar(<64 x i8> inreg %a,
; SI-NEXT: v_lshr_b64 v[6:7], v[6:7], 16
; SI-NEXT: v_mul_f32_e64 v7, 1.0, vcc_hi
; SI-NEXT: v_lshrrev_b32_e32 v8, 16, v7
-; SI-NEXT: v_mul_f32_e64 v7, 1.0, s8
+; SI-NEXT: v_mul_f32_e64 v7, 1.0, s61
; SI-NEXT: v_lshr_b64 v[7:8], v[7:8], 16
-; SI-NEXT: v_mul_f32_e64 v8, 1.0, s86
+; SI-NEXT: v_mul_f32_e64 v8, 1.0, s48
; SI-NEXT: v_lshrrev_b32_e32 v9, 16, v8
; SI-NEXT: v_mul_f32_e64 v8, 1.0, vcc_lo
; SI-NEXT: v_lshr_b64 v[8:9], v[8:9], 16
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s85
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s39
; SI-NEXT: v_lshrrev_b32_e32 v10, 16, v9
-; SI-NEXT: v_mul_f32_e64 v9, 1.0, s69
+; SI-NEXT: v_mul_f32_e64 v9, 1.0, s38
; SI-NEXT: v_lshr_b64 v[9:10], v[9:10], 16
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s84
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s37
; SI-NEXT: v_lshrrev_b32_e32 v11, 16, v10
-; SI-NEXT: v_mul_f32_e64 v10, 1.0, s83
+; SI-NEXT: v_mul_f32_e64 v10, 1.0, s36
; SI-NEXT: v_lshr_b64 v[10:11], v[10:11], 16
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s82
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s35
; SI-NEXT: v_lshrrev_b32_e32 v12, 16, v11
-; SI-NEXT: v_mul_f32_e64 v11, 1.0, s81
+; SI-NEXT: v_mul_f32_e64 v11, 1.0, s34
; SI-NEXT: v_lshr_b64 v[11:12], v[11:12], 16
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s80
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s31
; SI-NEXT: v_lshrrev_b32_e32 v13, 16, v12
-; SI-NEXT: v_mul_f32_e64 v12, 1.0, s71
+; SI-NEXT: v_mul_f32_e64 v12, 1.0, s30
; SI-NEXT: v_lshr_b64 v[12:13], v[12:13], 16
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s68
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s56
; SI-NEXT: v_lshrrev_b32_e32 v14, 16, v13
-; SI-NEXT: v_mul_f32_e64 v13, 1.0, s67
+; SI-NEXT: v_mul_f32_e64 v13, 1.0, s47
; SI-NEXT: v_lshr_b64 v[13:14], v[13:14], 16
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s66
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s46
; SI-NEXT: v_lshrrev_b32_e32 v15, 16, v14
-; SI-NEXT: v_mul_f32_e64 v14, 1.0, s7
+; SI-NEXT: v_mul_f32_e64 v14, 1.0, s41
; SI-NEXT: v_lshr_b64 v[14:15], v[14:15], 16
; SI-NEXT: v_mul_f32_e64 v15, 1.0, s5
; SI-NEXT: v_lshrrev_b32_e32 v16, 16, v15
-; SI-NEXT: v_mul_f32_e64 v15, 1.0, s12
-; SI-NEXT: v_readlane_b32 s30, v43, 31
+; SI-NEXT: v_mul_f32_e64 v15, 1.0, s78
+; SI-NEXT: v_readlane_b32 s30, v43, 13
; SI-NEXT: v_lshr_b64 v[15:16], v[15:16], 16
-; SI-NEXT: v_readlane_b32 s31, v43, 32
-; SI-NEXT: v_readlane_b32 s96, v43, 30
-; SI-NEXT: v_readlane_b32 s87, v43, 29
-; SI-NEXT: v_readlane_b32 s86, v43, 28
-; SI-NEXT: v_readlane_b32 s85, v43, 27
-; SI-NEXT: v_readlane_b32 s84, v43, 26
-; SI-NEXT: v_readlane_b32 s83, v43, 25
-; SI-NEXT: v_readlane_b32 s82, v43, 24
-; SI-NEXT: v_readlane_b32 s81, v43, 23
-; SI-NEXT: v_readlane_b32 s80, v43, 22
-; SI-NEXT: v_readlane_b32 s71, v43, 21
-; SI-NEXT: v_readlane_b32 s70, v43, 20
-; SI-NEXT: v_readlane_b32 s69, v43, 19
-; SI-NEXT: v_readlane_b32 s68, v43, 18
-; SI-NEXT: v_readlane_b32 s67, v43, 17
-; SI-NEXT: v_readlane_b32 s66, v43, 16
-; SI-NEXT: v_readlane_b32 s65, v43, 15
-; SI-NEXT: v_readlane_b32 s64, v43, 14
-; SI-NEXT: v_readlane_b32 s55, v43, 13
+; SI-NEXT: v_readlane_b32 s31, v43, 14
; SI-NEXT: v_readlane_b32 s54, v43, 12
; SI-NEXT: v_readlane_b32 s53, v43, 11
; SI-NEXT: v_readlane_b32 s52, v43, 10
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
index a2d89fb22315e..3bd3eec83eb13 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
@@ -8354,6 +8354,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -8371,16 +8372,14 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -8391,83 +8390,94 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -8477,7 +8487,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -8525,11 +8535,11 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -8545,15 +8555,17 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -8592,8 +8604,6 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -8612,6 +8622,7 @@ define inreg <20 x i32> @bitcast_v40f16_to_v20i32_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -17066,6 +17077,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -17083,16 +17095,14 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -17103,83 +17113,94 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -17189,7 +17210,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -17237,11 +17258,11 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -17257,15 +17278,17 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -17304,8 +17327,6 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -17324,6 +17345,7 @@ define inreg <20 x float> @bitcast_v40f16_to_v20f32_scalar(<40 x half> inreg %a,
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -24069,6 +24091,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -24086,16 +24109,14 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -24106,83 +24127,94 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -24192,7 +24224,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -24240,11 +24272,11 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -24260,15 +24292,17 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -24307,8 +24341,6 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -24327,6 +24359,7 @@ define inreg <10 x i64> @bitcast_v40f16_to_v10i64_scalar(<40 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -30346,6 +30379,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s36, 0
; VI-NEXT: v_writelane_b32 v32, s37, 1
@@ -30363,16 +30397,14 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 13
; VI-NEXT: v_writelane_b32 v32, s66, 14
; VI-NEXT: v_writelane_b32 v32, s67, 15
-; VI-NEXT: v_writelane_b32 v32, s30, 16
-; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v5
; VI-NEXT: v_readfirstlane_b32 s8, v4
; VI-NEXT: v_readfirstlane_b32 s11, v3
; VI-NEXT: v_readfirstlane_b32 s14, v2
; VI-NEXT: v_readfirstlane_b32 s73, v1
; VI-NEXT: v_readfirstlane_b32 s76, v0
-; VI-NEXT: s_lshr_b32 s10, s29, 16
-; VI-NEXT: s_lshr_b32 s13, s28, 16
+; VI-NEXT: s_lshr_b32 s5, s29, 16
+; VI-NEXT: s_lshr_b32 s12, s28, 16
; VI-NEXT: s_lshr_b32 s72, s27, 16
; VI-NEXT: s_lshr_b32 s74, s26, 16
; VI-NEXT: s_lshr_b32 s77, s25, 16
@@ -30383,83 +30415,94 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_lshr_b32 s91, s20, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; VI-NEXT: s_lshr_b32 s4, s17, 16
-; VI-NEXT: s_lshr_b32 s30, s16, 16
-; VI-NEXT: s_lshr_b32 s31, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
+; VI-NEXT: s_lshr_b32 s40, s17, 16
+; VI-NEXT: s_lshr_b32 s7, s16, 16
+; VI-NEXT: s_lshr_b32 s4, s6, 16
+; VI-NEXT: s_lshr_b32 s56, s8, 16
+; VI-NEXT: s_lshr_b32 s57, s11, 16
; VI-NEXT: s_lshr_b32 s15, s14, 16
; VI-NEXT: s_lshr_b32 s75, s73, 16
; VI-NEXT: s_lshr_b32 s78, s76, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v6
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s9, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_cmp_lg_u32 s9, 0
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: v_writelane_b32 v33, s56, 1
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s9, 0xffff, s16
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s9, s7
+; VI-NEXT: s_and_b32 s9, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s13, s40, 16
+; VI-NEXT: s_or_b32 s37, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s13, vcc_hi, 16
+; VI-NEXT: s_or_b32 s38, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s13, vcc_lo, 16
+; VI-NEXT: s_or_b32 s39, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s13, s91, 16
+; VI-NEXT: s_mov_b32 s7, s40
+; VI-NEXT: s_or_b32 s40, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s13, s90, 16
+; VI-NEXT: s_or_b32 s41, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s13, s89, 16
+; VI-NEXT: s_or_b32 s42, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s13, s88, 16
+; VI-NEXT: s_or_b32 s43, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s13, s79, 16
+; VI-NEXT: s_or_b32 s44, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s13, s77, 16
+; VI-NEXT: s_or_b32 s45, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s13, s74, 16
+; VI-NEXT: s_or_b32 s46, s9, s13
+; VI-NEXT: s_and_b32 s9, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s13, s72, 16
+; VI-NEXT: s_or_b32 s47, s9, s13
+; VI-NEXT: s_and_b32 s13, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s48, s13, s12
+; VI-NEXT: s_and_b32 s13, 0xffff, s29
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s49, s13, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s13, s78, 16
+; VI-NEXT: s_or_b32 s50, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s13, s75, 16
+; VI-NEXT: s_or_b32 s51, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s13, s15, 16
+; VI-NEXT: s_or_b32 s52, s5, s13
+; VI-NEXT: s_and_b32 s5, 0xffff, s11
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s53, s5, s57
+; VI-NEXT: s_and_b32 s5, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s54, s5, s56
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s37, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s38, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s39, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
+; VI-NEXT: s_or_b32 s55, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s9, s12
+; VI-NEXT: s_mov_b32 s13, s57
+; VI-NEXT: s_mov_b32 s12, s5
+; VI-NEXT: s_mov_b32 s10, s7
+; VI-NEXT: s_mov_b32 s7, s40
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -30469,7 +30512,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v19, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s30
+; VI-NEXT: v_mov_b32_e32 v0, s10
; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v0, v0, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v19
@@ -30517,11 +30560,11 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v11, v11, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v19
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s13
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v19
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s10
+; VI-NEXT: v_mov_b32_e32 v13, s12
; VI-NEXT: v_add_f16_sdwa v13, v13, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -30537,15 +30580,17 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v16, v16, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s14, v19
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s12
+; VI-NEXT: v_mov_b32_e32 v17, s13
; VI-NEXT: v_add_f16_sdwa v17, v17, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s11, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s9
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s8, v19
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v18, v20, v18
-; VI-NEXT: v_mov_b32_e32 v20, s31
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v19 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s6, v19
; VI-NEXT: v_or_b32_e32 v19, v19, v20
@@ -30584,8 +30629,6 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 16
-; VI-NEXT: v_readlane_b32 s31, v32, 17
; VI-NEXT: v_readlane_b32 s67, v32, 15
; VI-NEXT: v_readlane_b32 s66, v32, 14
; VI-NEXT: v_readlane_b32 s65, v32, 13
@@ -30604,6 +30647,7 @@ define inreg <10 x double> @bitcast_v40f16_to_v10f64_scalar(<40 x half> inreg %a
; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
index 048bfa8132a1d..c0741937d045b 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
@@ -8750,8 +8750,9 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -8768,8 +8769,6 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -8790,91 +8789,97 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -8883,7 +8888,7 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -9040,12 +9045,14 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -9094,8 +9101,6 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -9114,6 +9119,7 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -9123,134 +9129,152 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -9260,23 +9284,23 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -9304,15 +9328,15 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -9323,28 +9347,34 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -9383,30 +9413,25 @@ define inreg <22 x i32> @bitcast_v44f16_to_v22i32_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -18245,8 +18270,9 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -18263,8 +18289,6 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -18285,91 +18309,97 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -18378,7 +18408,7 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -18535,12 +18565,14 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -18589,8 +18621,6 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -18609,6 +18639,7 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -18618,134 +18649,152 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -18755,23 +18804,23 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -18799,15 +18848,15 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -18818,28 +18867,34 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -18878,30 +18933,25 @@ define inreg <22 x float> @bitcast_v44f16_to_v22f32_scalar(<44 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -25947,8 +25997,9 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -25965,8 +26016,6 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -25987,91 +26036,97 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -26080,7 +26135,7 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -26237,12 +26292,14 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -26291,8 +26348,6 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -26311,6 +26366,7 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -26320,134 +26376,152 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -26457,23 +26531,23 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -26501,15 +26575,15 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -26520,28 +26594,34 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -26580,30 +26660,25 @@ define inreg <11 x i64> @bitcast_v44f16_to_v11i64_scalar(<44 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -32897,8 +32972,9 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s36, 0
; SI-NEXT: v_writelane_b32 v32, s37, 1
; SI-NEXT: v_writelane_b32 v32, s38, 2
@@ -32915,8 +32991,6 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 13
; SI-NEXT: v_writelane_b32 v32, s66, 14
; SI-NEXT: v_writelane_b32 v32, s67, 15
-; SI-NEXT: v_writelane_b32 v32, s30, 16
-; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v7
; SI-NEXT: v_readfirstlane_b32 s8, v6
; SI-NEXT: v_readfirstlane_b32 s10, v5
@@ -32937,91 +33011,97 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_lshr_b32 s95, s20, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s19, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s18, 16
-; SI-NEXT: s_lshr_b32 s4, s17, 16
-; SI-NEXT: s_lshr_b32 s30, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s7, s16, 16
+; SI-NEXT: s_lshr_b32 s4, s6, 16
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: s_lshr_b32 s11, s10, 16
; SI-NEXT: s_lshr_b32 s13, s12, 16
; SI-NEXT: s_lshr_b32 s15, s14, 16
; SI-NEXT: s_lshr_b32 s73, s72, 16
; SI-NEXT: s_lshr_b32 s76, s74, 16
; SI-NEXT: s_lshr_b32 s79, s77, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v8
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s9, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_cmp_lg_u32 s9, 0
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: v_writelane_b32 v33, s5, 1
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s40, s30, 16
-; SI-NEXT: s_and_b32 s41, s17, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s41, s16, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s42, s7, 16
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s7, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s36, s41, s42
+; SI-NEXT: s_or_b32 s37, s43, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
+; SI-NEXT: s_or_b32 s38, s40, s41
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s41, vcc_lo, 16
+; SI-NEXT: s_or_b32 s39, s40, s41
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s41, s95, 16
+; SI-NEXT: s_or_b32 s40, s40, s41
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s42, s94, 16
+; SI-NEXT: s_or_b32 s41, s41, s42
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s92, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s91, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s90, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s89, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s88, 16
+; SI-NEXT: s_or_b32 s47, s47, s56
+; SI-NEXT: s_and_b32 s56, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s78, 16
+; SI-NEXT: s_or_b32 s48, s56, s57
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s75, 16
+; SI-NEXT: s_or_b32 s49, s56, s57
+; SI-NEXT: s_and_b32 s56, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s79, 16
+; SI-NEXT: s_or_b32 s50, s56, s57
+; SI-NEXT: s_and_b32 s56, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s76, 16
+; SI-NEXT: s_or_b32 s51, s56, s57
+; SI-NEXT: s_and_b32 s56, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s73, 16
+; SI-NEXT: s_or_b32 s52, s56, s57
+; SI-NEXT: s_and_b32 s56, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s15, 16
+; SI-NEXT: s_or_b32 s53, s56, s57
+; SI-NEXT: s_and_b32 s56, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s13, 16
+; SI-NEXT: s_or_b32 s54, s56, s57
+; SI-NEXT: s_and_b32 s56, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s11, 16
+; SI-NEXT: s_or_b32 s55, s56, s57
+; SI-NEXT: s_and_b32 s56, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s56, s56, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s36, s5, s40
-; SI-NEXT: s_or_b32 s37, s41, s4
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s38, s4, s5
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s39, s4, s5
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s40, s4, s5
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s41, s4, s5
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
+; SI-NEXT: s_or_b32 s57, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s40
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -33030,7 +33110,7 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s9
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
; SI-NEXT: v_cvt_f32_f16_e32 v2, s7
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
@@ -33187,12 +33267,14 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s6
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -33241,8 +33323,6 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 16
-; SI-NEXT: v_readlane_b32 s31, v32, 17
; SI-NEXT: v_readlane_b32 s67, v32, 15
; SI-NEXT: v_readlane_b32 s66, v32, 14
; SI-NEXT: v_readlane_b32 s65, v32, 13
@@ -33261,6 +33341,7 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -33270,134 +33351,152 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s30, 20
-; VI-NEXT: v_writelane_b32 v32, s31, 21
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
; VI-NEXT: v_readfirstlane_b32 s6, v7
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v6
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v5
-; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s15, v3
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: v_readfirstlane_b32 s12, v4
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s75, v2
; VI-NEXT: v_readfirstlane_b32 s78, v1
; VI-NEXT: v_readfirstlane_b32 s89, v0
-; VI-NEXT: s_lshr_b32 s13, s29, 16
-; VI-NEXT: s_lshr_b32 s72, s28, 16
-; VI-NEXT: s_lshr_b32 s74, s27, 16
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s14, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
; VI-NEXT: s_lshr_b32 s77, s26, 16
; VI-NEXT: s_lshr_b32 s88, s25, 16
; VI-NEXT: s_lshr_b32 s90, s24, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; VI-NEXT: s_lshr_b32 s4, s21, 16
-; VI-NEXT: s_lshr_b32 s30, s20, 16
-; VI-NEXT: s_lshr_b32 s31, s19, 16
-; VI-NEXT: s_lshr_b32 s34, s18, 16
-; VI-NEXT: s_lshr_b32 s35, s17, 16
-; VI-NEXT: s_lshr_b32 s68, s16, 16
-; VI-NEXT: s_lshr_b32 s69, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s14, s12, 16
-; VI-NEXT: s_lshr_b32 s73, s15, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s7, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s13, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s43, s16, 16
+; VI-NEXT: s_lshr_b32 s58, s12, 16
+; VI-NEXT: s_lshr_b32 s60, s75, 16
; VI-NEXT: s_lshr_b32 s79, s78, 16
; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v8
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v33, s59, 3
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 4
+; VI-NEXT: v_writelane_b32 v33, s60, 5
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s35, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s30, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s36, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s38, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_and_b32 s44, 0xffff, s17
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s40, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s7, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s37, s44, s42
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s42, vcc_hi, 16
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s43, vcc_lo, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, s90, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s88, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s46, s77, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s61, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s48, s61, s14
+; VI-NEXT: s_and_b32 s61, 0xffff, s29
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s61, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s61, s91, 16
+; VI-NEXT: s_or_b32 s50, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s61, s79, 16
+; VI-NEXT: s_or_b32 s51, s56, s61
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s53, s56, s59
+; VI-NEXT: s_and_b32 s56, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s54, s56, s58
+; VI-NEXT: s_and_b32 s56, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s56, s56, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s41, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
+; VI-NEXT: s_or_b32 s57, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s14
+; VI-NEXT: s_mov_b32 s14, s56
+; VI-NEXT: s_mov_b32 s76, s43
+; VI-NEXT: s_mov_b32 s72, s42
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s40
+; VI-NEXT: s_mov_b32 s73, s7
+; VI-NEXT: s_mov_b32 s7, s41
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -33407,23 +33506,23 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v21, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s68
-; VI-NEXT: v_mov_b32_e32 v2, s35
+; VI-NEXT: v_mov_b32_e32 v0, s76
+; VI-NEXT: v_mov_b32_e32 v2, s72
; VI-NEXT: v_add_f16_sdwa v0, v0, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v21
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v21
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s34
+; VI-NEXT: v_mov_b32_e32 v2, s74
; VI-NEXT: v_add_f16_sdwa v2, v2, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v21
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s31
+; VI-NEXT: v_mov_b32_e32 v3, s13
; VI-NEXT: v_add_f16_sdwa v3, v3, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v21
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s30
+; VI-NEXT: v_mov_b32_e32 v4, s73
; VI-NEXT: v_add_f16_sdwa v4, v4, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v21
; VI-NEXT: v_or_b32_e32 v4, v5, v4
@@ -33451,15 +33550,15 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v10, v10, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v21
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s74
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v21
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s72
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s13
+; VI-NEXT: v_mov_b32_e32 v13, s14
; VI-NEXT: v_add_f16_sdwa v13, v13, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v21
; VI-NEXT: v_or_b32_e32 v13, v14, v13
@@ -33470,28 +33569,34 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v15, s79
; VI-NEXT: v_add_f16_sdwa v15, v15, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v16, s78, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s76
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s75, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s73
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s15, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s14
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s12, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s11
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s10, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s9
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s8, v21
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v20, v22, v20
-; VI-NEXT: v_mov_b32_e32 v22, s69
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v21 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s6, v21
; VI-NEXT: v_or_b32_e32 v21, v21, v22
@@ -33530,30 +33635,25 @@ define inreg <11 x double> @bitcast_v44f16_to_v11f64_scalar(<44 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 20
-; VI-NEXT: v_readlane_b32 s31, v32, 21
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
index ddee91e20f7dd..0a06115202c94 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
@@ -9454,40 +9454,44 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -9496,103 +9500,112 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -9601,9 +9614,9 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -9613,7 +9626,7 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -9623,14 +9636,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -9739,10 +9752,11 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -9750,12 +9764,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -9763,10 +9779,11 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -9774,12 +9791,14 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -9828,30 +9847,25 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -9861,148 +9875,172 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -10012,35 +10050,35 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -10052,25 +10090,28 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -10079,32 +10120,39 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -10143,34 +10191,27 @@ define inreg <24 x i32> @bitcast_v48f16_to_v24i32_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -19701,40 +19742,44 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -19743,103 +19788,112 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -19848,9 +19902,9 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -19860,7 +19914,7 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -19870,14 +19924,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -19986,10 +20040,11 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -19997,12 +20052,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -20010,10 +20067,11 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -20021,12 +20079,14 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -20075,30 +20135,25 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -20108,148 +20163,172 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -20259,35 +20338,35 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -20299,25 +20378,28 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -20326,32 +20408,39 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -20390,34 +20479,27 @@ define inreg <24 x float> @bitcast_v48f16_to_v24f32_scalar(<48 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -28073,40 +28155,44 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -28115,103 +28201,112 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -28220,9 +28315,9 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -28232,7 +28327,7 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -28242,14 +28337,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -28358,10 +28453,11 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -28369,12 +28465,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -28382,10 +28480,11 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -28393,12 +28492,14 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -28447,30 +28548,25 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -28480,148 +28576,172 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -28631,35 +28751,35 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -28671,25 +28791,28 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -28698,32 +28821,39 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -28762,34 +28892,27 @@ define inreg <12 x i64> @bitcast_v48f16_to_v12i64_scalar(<48 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -35666,40 +35789,44 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s30, 20
-; SI-NEXT: v_writelane_b32 v32, s31, 21
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
; SI-NEXT: v_readfirstlane_b32 s6, v9
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v8
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v7
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s57, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v6
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s56, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v5
; SI-NEXT: v_readfirstlane_b32 s72, v4
; SI-NEXT: v_readfirstlane_b32 s74, v3
; SI-NEXT: v_readfirstlane_b32 s76, v2
; SI-NEXT: v_readfirstlane_b32 s79, v1
; SI-NEXT: v_readfirstlane_b32 s89, v0
+; SI-NEXT: v_writelane_b32 v33, s57, 2
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s90, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
@@ -35708,103 +35835,112 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_lshr_b32 s95, s24, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s23, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s30, s20, 16
-; SI-NEXT: s_lshr_b32 s31, s19, 16
-; SI-NEXT: s_lshr_b32 s34, s18, 16
-; SI-NEXT: s_lshr_b32 s68, s17, 16
-; SI-NEXT: s_lshr_b32 s69, s16, 16
-; SI-NEXT: s_lshr_b32 s35, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s9, s19, 16
+; SI-NEXT: s_lshr_b32 s11, s18, 16
+; SI-NEXT: s_lshr_b32 s13, s17, 16
+; SI-NEXT: s_lshr_b32 s15, s16, 16
+; SI-NEXT: s_lshr_b32 s58, s14, 16
+; SI-NEXT: s_lshr_b32 s59, s72, 16
; SI-NEXT: s_lshr_b32 s75, s74, 16
; SI-NEXT: s_lshr_b32 s77, s76, 16
; SI-NEXT: s_lshr_b32 s88, s79, 16
; SI-NEXT: s_lshr_b32 s91, s89, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v10
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s40, v10
+; SI-NEXT: v_writelane_b32 v33, s56, 3
+; SI-NEXT: s_cmp_lg_u32 s40, 0
+; SI-NEXT: v_writelane_b32 v33, s58, 4
+; SI-NEXT: v_writelane_b32 v33, s59, 5
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s40, s16, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s40, s15
+; SI-NEXT: s_and_b32 s40, s17, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s37, s40, s13
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s38, s40, s11
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s39, s40, s9
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s43, vcc_hi, 16
+; SI-NEXT: s_or_b32 s42, s42, s43
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s44, vcc_lo, 16
+; SI-NEXT: s_or_b32 s43, s43, s44
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s95, 16
+; SI-NEXT: s_or_b32 s44, s44, s45
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s46, s94, 16
+; SI-NEXT: s_or_b32 s45, s45, s46
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s93, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s92, 16
+; SI-NEXT: s_or_b32 s47, s47, s60
+; SI-NEXT: s_and_b32 s60, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s90, 16
+; SI-NEXT: s_or_b32 s48, s60, s61
+; SI-NEXT: s_and_b32 s60, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s78, 16
+; SI-NEXT: s_or_b32 s49, s60, s61
+; SI-NEXT: s_and_b32 s60, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s91, 16
+; SI-NEXT: s_or_b32 s50, s60, s61
+; SI-NEXT: s_and_b32 s60, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s88, 16
+; SI-NEXT: s_or_b32 s51, s60, s61
+; SI-NEXT: s_and_b32 s60, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s77, 16
+; SI-NEXT: s_or_b32 s52, s60, s61
+; SI-NEXT: s_and_b32 s60, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s75, 16
+; SI-NEXT: s_or_b32 s53, s60, s61
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s54, s60, s59
+; SI-NEXT: s_and_b32 s59, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s55, s59, s58
+; SI-NEXT: s_and_b32 s58, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s58, s56
+; SI-NEXT: s_and_b32 s58, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s58, s57
+; SI-NEXT: s_and_b32 s58, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s58, s58, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s94, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
+; SI-NEXT: s_or_b32 s59, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s41
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -35813,9 +35949,9 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s73
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s15
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -35825,7 +35961,7 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s13
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -35835,14 +35971,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s11
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s9
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
@@ -35951,10 +36087,11 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v19, s74
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s72
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -35962,12 +36099,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -35975,10 +36114,11 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v22, s12
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s10
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -35986,12 +36126,14 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s6
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -36040,30 +36182,25 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 20
-; SI-NEXT: v_readlane_b32 s31, v32, 21
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -36073,148 +36210,172 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
-; VI-NEXT: v_writelane_b32 v32, s34, 0
-; VI-NEXT: v_writelane_b32 v32, s35, 1
-; VI-NEXT: v_writelane_b32 v32, s36, 2
-; VI-NEXT: v_writelane_b32 v32, s37, 3
-; VI-NEXT: v_writelane_b32 v32, s38, 4
-; VI-NEXT: v_writelane_b32 v32, s39, 5
-; VI-NEXT: v_writelane_b32 v32, s48, 6
-; VI-NEXT: v_writelane_b32 v32, s49, 7
-; VI-NEXT: v_writelane_b32 v32, s50, 8
-; VI-NEXT: v_writelane_b32 v32, s51, 9
-; VI-NEXT: v_writelane_b32 v32, s52, 10
-; VI-NEXT: v_writelane_b32 v32, s53, 11
-; VI-NEXT: v_writelane_b32 v32, s54, 12
-; VI-NEXT: v_writelane_b32 v32, s55, 13
-; VI-NEXT: v_writelane_b32 v32, s64, 14
-; VI-NEXT: v_writelane_b32 v32, s65, 15
-; VI-NEXT: v_writelane_b32 v32, s66, 16
-; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s30, 24
-; VI-NEXT: v_writelane_b32 v32, s31, 25
+; VI-NEXT: v_writelane_b32 v32, s36, 0
+; VI-NEXT: v_writelane_b32 v32, s37, 1
+; VI-NEXT: v_writelane_b32 v32, s38, 2
+; VI-NEXT: v_writelane_b32 v32, s39, 3
+; VI-NEXT: v_writelane_b32 v32, s48, 4
+; VI-NEXT: v_writelane_b32 v32, s49, 5
+; VI-NEXT: v_writelane_b32 v32, s50, 6
+; VI-NEXT: v_writelane_b32 v32, s51, 7
+; VI-NEXT: v_writelane_b32 v32, s52, 8
+; VI-NEXT: v_writelane_b32 v32, s53, 9
+; VI-NEXT: v_writelane_b32 v32, s54, 10
+; VI-NEXT: v_writelane_b32 v32, s55, 11
+; VI-NEXT: v_writelane_b32 v32, s64, 12
+; VI-NEXT: v_writelane_b32 v32, s65, 13
+; VI-NEXT: v_writelane_b32 v32, s66, 14
+; VI-NEXT: v_writelane_b32 v32, s67, 15
+; VI-NEXT: v_writelane_b32 v32, s30, 16
+; VI-NEXT: v_writelane_b32 v32, s31, 17
; VI-NEXT: v_readfirstlane_b32 s6, v9
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v8
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v7
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s57, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v6
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s56, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 2
+; VI-NEXT: s_lshr_b32 s60, s14, 16
; VI-NEXT: v_readfirstlane_b32 s73, v4
+; VI-NEXT: v_writelane_b32 v33, s56, 3
+; VI-NEXT: s_lshr_b32 s61, s73, 16
; VI-NEXT: v_readfirstlane_b32 s76, v3
+; VI-NEXT: v_writelane_b32 v33, s60, 4
+; VI-NEXT: s_lshr_b32 s62, s76, 16
+; VI-NEXT: v_writelane_b32 v33, s61, 5
; VI-NEXT: v_readfirstlane_b32 s79, v2
; VI-NEXT: v_readfirstlane_b32 s91, v1
-; VI-NEXT: v_readfirstlane_b32 s4, v0
-; VI-NEXT: s_lshr_b32 s72, s29, 16
-; VI-NEXT: s_lshr_b32 s74, s28, 16
-; VI-NEXT: s_lshr_b32 s77, s27, 16
-; VI-NEXT: s_lshr_b32 s88, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s63, v0
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: s_lshr_b32 s58, s29, 16
+; VI-NEXT: s_lshr_b32 s59, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
; VI-NEXT: s_lshr_b32 s90, s25, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
-; VI-NEXT: s_lshr_b32 s31, s23, 16
-; VI-NEXT: s_lshr_b32 s34, s22, 16
-; VI-NEXT: s_lshr_b32 s35, s21, 16
-; VI-NEXT: s_lshr_b32 s68, s20, 16
-; VI-NEXT: s_lshr_b32 s69, s19, 16
-; VI-NEXT: s_lshr_b32 s70, s18, 16
-; VI-NEXT: s_lshr_b32 s71, s17, 16
-; VI-NEXT: s_lshr_b32 s80, s16, 16
-; VI-NEXT: s_lshr_b32 s81, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s75, s73, 16
-; VI-NEXT: s_lshr_b32 s78, s76, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s13, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s42, s18, 16
+; VI-NEXT: s_lshr_b32 s7, s17, 16
+; VI-NEXT: s_lshr_b32 s9, s16, 16
; VI-NEXT: s_lshr_b32 s89, s79, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v10
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s30, s63, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v10
+; VI-NEXT: v_writelane_b32 v33, s63, 7
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s58, 8
+; VI-NEXT: v_writelane_b32 v33, s30, 9
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s37, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s68, 16
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s72, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_mov_b32 s7, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s36, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s17
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s37, s11, s7
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s38, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s42, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s13, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s44, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s11, s44
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s45, s90, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s75, 0xffff, s26
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s75, s46
+; VI-NEXT: s_and_b32 s75, 0xffff, s27
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s75, s47
+; VI-NEXT: s_and_b32 s31, 0xffff, s28
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s48, s31, s59
+; VI-NEXT: s_and_b32 s59, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s49, s59, s58
+; VI-NEXT: s_and_b32 s58, 0xffff, s63
+; VI-NEXT: s_lshl_b32 s59, s30, 16
+; VI-NEXT: s_or_b32 s50, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s59, vcc_lo, 16
+; VI-NEXT: s_or_b32 s51, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s59, s89, 16
+; VI-NEXT: s_or_b32 s52, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s59, s62, 16
+; VI-NEXT: s_or_b32 s53, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s59, s61, 16
+; VI-NEXT: s_or_b32 s54, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s60, 16
+; VI-NEXT: s_or_b32 s55, s58, s59
+; VI-NEXT: s_and_b32 s58, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s56, s58, s56
+; VI-NEXT: s_and_b32 s58, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s58, s57
+; VI-NEXT: s_and_b32 s58, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s58, s58, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s59, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s11, s46
+; VI-NEXT: s_mov_b32 s78, s47
+; VI-NEXT: s_mov_b32 s75, s59
+; VI-NEXT: s_mov_b32 s88, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s42
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 s72, s41
+; VI-NEXT: s_mov_b32 s74, s13
+; VI-NEXT: s_mov_b32 s13, s43
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -36224,35 +36385,35 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v23, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s80
-; VI-NEXT: v_mov_b32_e32 v2, s71
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, s9
; VI-NEXT: v_add_f16_sdwa v0, v0, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v23
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v23
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s70
+; VI-NEXT: v_mov_b32_e32 v2, s7
; VI-NEXT: v_add_f16_sdwa v2, v2, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v23
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s69
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v23
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s68
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v23
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s35
+; VI-NEXT: v_mov_b32_e32 v5, s72
; VI-NEXT: v_add_f16_sdwa v5, v5, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v23
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s34
+; VI-NEXT: v_mov_b32_e32 v6, s74
; VI-NEXT: v_add_f16_sdwa v6, v6, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v23
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s31
+; VI-NEXT: v_mov_b32_e32 v7, s13
; VI-NEXT: v_add_f16_sdwa v7, v7, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v23
; VI-NEXT: v_or_b32_e32 v7, v8, v7
@@ -36264,25 +36425,28 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v9, v9, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v23
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s88
+; VI-NEXT: v_mov_b32_e32 v10, s11
; VI-NEXT: v_add_f16_sdwa v10, v10, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v23
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s77
+; VI-NEXT: v_mov_b32_e32 v11, s78
; VI-NEXT: v_add_f16_sdwa v11, v11, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v23
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s74
+; VI-NEXT: v_mov_b32_e32 v12, s75
; VI-NEXT: v_add_f16_sdwa v12, v12, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s72
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s30
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_add_f16_sdwa v14, v14, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s7, v23
+; VI-NEXT: v_add_f16_e32 v15, s4, v23
; VI-NEXT: v_or_b32_e32 v14, v15, v14
; VI-NEXT: v_mov_b32_e32 v15, vcc_lo
; VI-NEXT: v_add_f16_sdwa v15, v15, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
@@ -36291,32 +36455,39 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v16, s89
; VI-NEXT: v_add_f16_sdwa v16, v16, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, s79, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s78
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s76, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s75
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s73, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s15
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s14, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s13
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s12, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s11
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s10, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s9
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s8, v23
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v22, v24, v22
-; VI-NEXT: v_mov_b32_e32 v24, s81
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v23 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s6, v23
; VI-NEXT: v_or_b32_e32 v23, v23, v24
@@ -36355,34 +36526,27 @@ define inreg <12 x double> @bitcast_v48f16_to_v12f64_scalar(<48 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 24
-; VI-NEXT: v_readlane_b32 s31, v32, 25
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
-; VI-NEXT: v_readlane_b32 s67, v32, 17
-; VI-NEXT: v_readlane_b32 s66, v32, 16
-; VI-NEXT: v_readlane_b32 s65, v32, 15
-; VI-NEXT: v_readlane_b32 s64, v32, 14
-; VI-NEXT: v_readlane_b32 s55, v32, 13
-; VI-NEXT: v_readlane_b32 s54, v32, 12
-; VI-NEXT: v_readlane_b32 s53, v32, 11
-; VI-NEXT: v_readlane_b32 s52, v32, 10
-; VI-NEXT: v_readlane_b32 s51, v32, 9
-; VI-NEXT: v_readlane_b32 s50, v32, 8
-; VI-NEXT: v_readlane_b32 s49, v32, 7
-; VI-NEXT: v_readlane_b32 s48, v32, 6
-; VI-NEXT: v_readlane_b32 s39, v32, 5
-; VI-NEXT: v_readlane_b32 s38, v32, 4
-; VI-NEXT: v_readlane_b32 s37, v32, 3
-; VI-NEXT: v_readlane_b32 s36, v32, 2
-; VI-NEXT: v_readlane_b32 s35, v32, 1
-; VI-NEXT: v_readlane_b32 s34, v32, 0
+; VI-NEXT: v_readlane_b32 s30, v32, 16
+; VI-NEXT: v_readlane_b32 s31, v32, 17
+; VI-NEXT: v_readlane_b32 s67, v32, 15
+; VI-NEXT: v_readlane_b32 s66, v32, 14
+; VI-NEXT: v_readlane_b32 s65, v32, 13
+; VI-NEXT: v_readlane_b32 s64, v32, 12
+; VI-NEXT: v_readlane_b32 s55, v32, 11
+; VI-NEXT: v_readlane_b32 s54, v32, 10
+; VI-NEXT: v_readlane_b32 s53, v32, 9
+; VI-NEXT: v_readlane_b32 s52, v32, 8
+; VI-NEXT: v_readlane_b32 s51, v32, 7
+; VI-NEXT: v_readlane_b32 s50, v32, 6
+; VI-NEXT: v_readlane_b32 s49, v32, 5
+; VI-NEXT: v_readlane_b32 s48, v32, 4
+; VI-NEXT: v_readlane_b32 s39, v32, 3
+; VI-NEXT: v_readlane_b32 s38, v32, 2
+; VI-NEXT: v_readlane_b32 s37, v32, 1
+; VI-NEXT: v_readlane_b32 s36, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -37817,9 +37981,9 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v25, s56, 5
; SI-NEXT: s_lshr_b32 s41, s45, 16
; SI-NEXT: v_writelane_b32 v25, s47, 6
-; SI-NEXT: v_readfirstlane_b32 s48, v6
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v6
; SI-NEXT: v_writelane_b32 v25, s41, 7
-; SI-NEXT: s_lshr_b32 s15, s48, 16
+; SI-NEXT: s_lshr_b32 s15, vcc_lo, 16
; SI-NEXT: v_writelane_b32 v25, s45, 8
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v25, s15, 9
@@ -37827,8 +37991,8 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v25, s43, 10
; SI-NEXT: v_readfirstlane_b32 s11, v5
; SI-NEXT: v_writelane_b32 v25, s13, 11
-; SI-NEXT: v_readfirstlane_b32 s5, v9
-; SI-NEXT: v_readfirstlane_b32 s37, v8
+; SI-NEXT: v_readfirstlane_b32 s7, v9
+; SI-NEXT: v_readfirstlane_b32 s30, v8
; SI-NEXT: v_readfirstlane_b32 s9, v7
; SI-NEXT: v_writelane_b32 v25, s11, 12
; SI-NEXT: s_lshr_b32 s89, s29, 16
@@ -37842,17 +38006,17 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s50, s18, 16
; SI-NEXT: s_lshr_b32 s61, s17, 16
; SI-NEXT: s_lshr_b32 s36, s16, 16
-; SI-NEXT: s_lshr_b32 s31, s5, 16
-; SI-NEXT: s_lshr_b32 s7, s37, 16
+; SI-NEXT: s_lshr_b32 s31, s7, 16
+; SI-NEXT: s_lshr_b32 s5, s30, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s9, 16
; SI-NEXT: s_lshr_b32 s51, s11, 16
-; SI-NEXT: s_lshr_b32 s49, s13, 16
-; SI-NEXT: s_lshr_b32 s91, s43, 16
+; SI-NEXT: s_lshr_b32 s93, s13, 16
+; SI-NEXT: s_lshr_b32 s37, s43, 16
; SI-NEXT: v_readfirstlane_b32 s10, v10
; SI-NEXT: v_writelane_b32 v25, s9, 13
; SI-NEXT: s_cmp_lg_u32 s10, 0
-; SI-NEXT: v_writelane_b32 v25, s5, 14
-; SI-NEXT: v_writelane_b32 v25, s7, 15
+; SI-NEXT: v_writelane_b32 v25, s7, 14
+; SI-NEXT: v_writelane_b32 v25, s5, 15
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s10, s16, 0xffff
@@ -37899,28 +38063,26 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s73, s45, s47
; SI-NEXT: s_and_b32 s45, s23, 0xffff
; SI-NEXT: s_lshl_b32 s47, s75, 16
-; SI-NEXT: s_mov_b32 vcc_lo, s75
+; SI-NEXT: s_mov_b32 s34, s75
; SI-NEXT: s_or_b32 s75, s45, s47
; SI-NEXT: s_and_b32 s45, s25, 0xffff
; SI-NEXT: s_lshl_b32 s47, s77, 16
-; SI-NEXT: s_mov_b32 s30, s77
+; SI-NEXT: s_mov_b32 s35, s77
; SI-NEXT: s_or_b32 s77, s45, s47
; SI-NEXT: s_and_b32 s45, s27, 0xffff
; SI-NEXT: s_lshl_b32 s47, s79, 16
-; SI-NEXT: s_mov_b32 s34, s79
+; SI-NEXT: s_mov_b32 s48, s79
; SI-NEXT: s_or_b32 s79, s45, s47
; SI-NEXT: s_and_b32 s45, s29, 0xffff
; SI-NEXT: s_lshl_b32 s47, s89, 16
-; SI-NEXT: s_mov_b32 s35, s89
+; SI-NEXT: s_mov_b32 s49, s89
; SI-NEXT: s_or_b32 s89, s45, s47
; SI-NEXT: s_and_b32 s43, s43, 0xffff
-; SI-NEXT: s_lshl_b32 s45, s91, 16
-; SI-NEXT: s_mov_b32 s95, s37
-; SI-NEXT: s_mov_b32 s37, s48
-; SI-NEXT: s_mov_b32 s48, s91
+; SI-NEXT: s_lshl_b32 s45, s37, 16
; SI-NEXT: s_or_b32 s91, s43, s45
; SI-NEXT: s_and_b32 s13, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s43, s49, 16
+; SI-NEXT: s_lshl_b32 s43, s93, 16
+; SI-NEXT: s_mov_b32 s95, s93
; SI-NEXT: s_or_b32 s93, s13, s43
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s13, s51, 16
@@ -37928,48 +38090,51 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, vcc_hi, 16
; SI-NEXT: s_or_b32 s57, s9, s11
-; SI-NEXT: s_and_b32 s5, s5, 0xffff
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s9, s31, 16
-; SI-NEXT: s_or_b32 s53, s5, s9
+; SI-NEXT: s_mov_b32 s47, s61
+; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
+; SI-NEXT: s_or_b32 s53, s7, s9
+; SI-NEXT: v_writelane_b32 v25, s60, 16
; SI-NEXT: s_mov_b32 s45, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
; SI-NEXT: s_mov_b32 s43, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
-; SI-NEXT: s_mov_b32 s9, s91
-; SI-NEXT: s_lshr_b64 s[90:91], s[90:91], 16
-; SI-NEXT: s_mov_b32 s47, s61
-; SI-NEXT: s_lshr_b64 s[60:61], s[60:61], 16
-; SI-NEXT: s_mov_b32 s63, s41
+; SI-NEXT: s_mov_b32 s13, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
+; SI-NEXT: s_mov_b32 s7, s93
+; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 16
+; SI-NEXT: s_or_b32 s4, s4, s58
+; SI-NEXT: s_lshl_b32 s52, s5, 16
+; SI-NEXT: v_writelane_b32 v25, s61, 17
; SI-NEXT: s_mov_b32 s73, s94
-; SI-NEXT: s_mov_b32 s41, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
-; SI-NEXT: s_mov_b32 s91, s48
-; SI-NEXT: s_mov_b32 s48, s37
-; SI-NEXT: s_mov_b32 s37, s95
+; SI-NEXT: s_mov_b32 s79, s48
+; SI-NEXT: s_mov_b32 s48, s62
+; SI-NEXT: s_mov_b32 s93, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[58:59], 16
-; SI-NEXT: s_or_b32 s4, s4, s58
-; SI-NEXT: s_lshl_b32 s52, s7, 16
+; SI-NEXT: s_mov_b32 s60, vcc_lo
+; SI-NEXT: s_and_b32 s58, vcc_lo, 0xffff
+; SI-NEXT: s_mov_b32 s62, vcc_hi
+; SI-NEXT: s_lshr_b64 vcc, s[56:57], 16
; SI-NEXT: s_mov_b32 s61, s15
-; SI-NEXT: s_mov_b32 s75, vcc_lo
+; SI-NEXT: s_mov_b32 s63, s41
+; SI-NEXT: s_mov_b32 s41, s75
+; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
; SI-NEXT: s_mov_b32 s15, s77
; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_and_b32 s58, s48, 0xffff
-; SI-NEXT: s_mov_b32 s95, vcc_hi
-; SI-NEXT: s_lshr_b64 vcc, s[56:57], 16
-; SI-NEXT: s_mov_b32 s77, s30
-; SI-NEXT: s_mov_b32 s13, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
; SI-NEXT: s_mov_b32 s11, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 16
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 vcc_hi, s95
-; SI-NEXT: s_and_b32 s56, s37, 0xffff
+; SI-NEXT: s_mov_b32 vcc_hi, s62
+; SI-NEXT: s_mov_b32 s62, s30
+; SI-NEXT: s_and_b32 s56, s30, 0xffff
; SI-NEXT: s_mov_b32 s95, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[52:53], 16
-; SI-NEXT: s_mov_b32 s79, s34
-; SI-NEXT: s_mov_b32 s89, s35
-; SI-NEXT: s_mov_b32 s7, s93
-; SI-NEXT: s_lshr_b64 s[92:93], s[92:93], 16
+; SI-NEXT: s_mov_b32 s75, s34
+; SI-NEXT: s_mov_b32 s77, s35
+; SI-NEXT: s_mov_b32 s89, s49
+; SI-NEXT: s_mov_b32 s9, s91
+; SI-NEXT: s_lshr_b64 s[90:91], s[90:91], 16
; SI-NEXT: s_mov_b32 s5, s59
; SI-NEXT: s_mov_b32 s59, s57
; SI-NEXT: s_or_b32 s56, s56, s52
@@ -37978,11 +38143,15 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_mov_b64 s[34:35], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s62, s30
+; SI-NEXT: s_mov_b32 s60, vcc_lo
; SI-NEXT: s_mov_b64 s[34:35], -1
+; SI-NEXT: v_writelane_b32 v25, s4, 16
; SI-NEXT: ; implicit-def: $sgpr46
-; SI-NEXT: ; implicit-def: $sgpr60
+; SI-NEXT: v_writelane_b32 v25, s5, 17
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr72
; SI-NEXT: ; implicit-def: $sgpr40
@@ -38007,9 +38176,11 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_and_b64 s[34:35], s[34:35], exec
; SI-NEXT: s_cselect_b32 s95, 1, 0
; SI-NEXT: s_cmp_lg_u32 s95, 1
+; SI-NEXT: v_readlane_b32 s34, v25, 16
+; SI-NEXT: v_readlane_b32 s35, v25, 17
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s4, s37, 3
+; SI-NEXT: s_add_i32 s4, s62, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v25, 15
; SI-NEXT: s_lshl_b32 s4, s4, 16
@@ -38021,7 +38192,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s48, 3
+; SI-NEXT: s_add_i32 s4, s60, 3
; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v25, 9
; SI-NEXT: s_lshl_b32 s4, s4, 16
@@ -38053,7 +38224,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_readlane_b32 s7, v25, 11
; SI-NEXT: s_add_i32 s7, s7, 3
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s8, s49, 16
+; SI-NEXT: s_lshl_b32 s8, s93, 16
; SI-NEXT: s_or_b32 s7, s8, s7
; SI-NEXT: v_readlane_b32 s8, v25, 4
; SI-NEXT: s_add_i32 s8, s8, 3
@@ -38064,7 +38235,7 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: v_readlane_b32 s9, v25, 10
; SI-NEXT: s_add_i32 s9, s9, 3
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s10, s91, 16
+; SI-NEXT: s_lshl_b32 s10, s37, 16
; SI-NEXT: s_or_b32 s9, s10, s9
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: v_readlane_b32 s10, v25, 2
@@ -38131,31 +38302,31 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s46, s16, 0x30000
; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s61, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
-; SI-NEXT: s_add_i32 s8, s8, 0x30000
-; SI-NEXT: s_add_i32 s9, s9, 0x30000
+; SI-NEXT: s_add_i32 s6, s6, 0x30000
+; SI-NEXT: s_add_i32 s7, s7, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
; SI-NEXT: s_add_i32 s12, s12, 0x30000
; SI-NEXT: s_add_i32 s13, s13, 0x30000
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
-; SI-NEXT: s_add_i32 s47, s16, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_add_i32 s6, s6, 0x30000
-; SI-NEXT: s_add_i32 s7, s7, 0x30000
-; SI-NEXT: s_lshr_b64 s[60:61], s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[62:63], s[44:45], 16
+; SI-NEXT: s_add_i32 s8, s8, 0x30000
+; SI-NEXT: s_add_i32 s9, s9, 0x30000
+; SI-NEXT: s_add_i32 s47, s16, 0x30000
; SI-NEXT: s_lshr_b64 s[72:73], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[90:91], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[92:93], s[6:7], 16
; SI-NEXT: s_lshr_b64 vcc, s[58:59], 16
; SI-NEXT: s_lshr_b64 s[30:31], s[56:57], 16
-; SI-NEXT: s_lshr_b64 s[92:93], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[34:35], s[46:47], 16
+; SI-NEXT: s_lshr_b64 s[48:49], s[44:45], 16
+; SI-NEXT: s_lshr_b64 s[90:91], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[4:5], 16
; SI-NEXT: s_lshr_b32 s61, s47, 16
; SI-NEXT: s_lshr_b32 s63, s45, 16
@@ -38164,20 +38335,20 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s77, s15, 16
; SI-NEXT: s_lshr_b32 s79, s13, 16
; SI-NEXT: s_lshr_b32 s89, s11, 16
-; SI-NEXT: s_lshr_b32 s91, s9, 16
-; SI-NEXT: s_lshr_b32 s49, s7, 16
+; SI-NEXT: s_lshr_b32 s37, s9, 16
+; SI-NEXT: s_lshr_b32 s93, s7, 16
; SI-NEXT: s_lshr_b32 s51, s5, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s59, 16
; SI-NEXT: s_lshr_b32 s31, s57, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s16, s46, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s60, 16
+; SI-NEXT: s_lshl_b32 s17, s34, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s47, 0xffff
; SI-NEXT: s_lshl_b32 s18, s61, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s18, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s19, s62, 16
+; SI-NEXT: s_lshl_b32 s19, s48, 16
; SI-NEXT: s_or_b32 s18, s18, s19
; SI-NEXT: s_and_b32 s19, s45, 0xffff
; SI-NEXT: s_lshl_b32 s20, s63, 16
@@ -38216,13 +38387,13 @@ define inreg <48 x half> @bitcast_v48i16_to_v48f16_scalar(<48 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s24, s90, 16
; SI-NEXT: s_or_b32 s8, s8, s24
; SI-NEXT: s_and_b32 s9, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s91, 16
+; SI-NEXT: s_lshl_b32 s24, s37, 16
; SI-NEXT: s_or_b32 s9, s9, s24
; SI-NEXT: s_and_b32 s6, s6, 0xffff
; SI-NEXT: s_lshl_b32 s24, s92, 16
; SI-NEXT: s_or_b32 s6, s6, s24
; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s24, s49, 16
+; SI-NEXT: s_lshl_b32 s24, s93, 16
; SI-NEXT: s_or_b32 s7, s7, s24
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s24, s94, 16
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
index 54180d9a67207..3e413ade56702 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
@@ -6585,6 +6585,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -6604,145 +6605,189 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -6750,43 +6795,47 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_cbranch_scc1 .LBB15_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -6796,27 +6845,27 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -6826,61 +6875,66 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -6907,17 +6961,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -6938,6 +6982,7 @@ define inreg <26 x i32> @bitcast_v52i16_to_v26i32_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -10167,159 +10212,182 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -10328,9 +10396,9 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -10342,7 +10410,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -10350,11 +10418,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -10365,7 +10433,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -10374,11 +10442,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -10389,7 +10457,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -10448,9 +10516,10 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -10458,10 +10527,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -10469,12 +10539,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -10482,10 +10554,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -10493,12 +10566,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -10506,10 +10581,11 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -10517,12 +10593,14 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -10571,34 +10649,27 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -10608,6 +10679,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -10627,143 +10699,172 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -10773,39 +10874,39 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -10813,67 +10914,81 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -10912,18 +11027,8 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -10944,6 +11049,7 @@ define inreg <26 x i32> @bitcast_v52f16_to_v26i32_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -17340,6 +17446,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -17359,145 +17466,189 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -17505,43 +17656,47 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB31_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -17551,27 +17706,27 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -17581,61 +17736,66 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -17662,17 +17822,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -17693,6 +17843,7 @@ define inreg <26 x float> @bitcast_v52i16_to_v26f32_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -21241,159 +21392,182 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -21402,9 +21576,9 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -21416,7 +21590,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -21424,11 +21598,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -21439,7 +21613,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -21448,11 +21622,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -21463,7 +21637,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -21522,9 +21696,10 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -21532,10 +21707,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -21543,12 +21719,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -21556,10 +21734,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -21567,12 +21746,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -21580,10 +21761,11 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -21591,12 +21773,14 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -21645,34 +21829,27 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -21682,6 +21859,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -21701,143 +21879,172 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -21847,39 +22054,39 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -21887,67 +22094,81 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -21986,18 +22207,8 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -22018,6 +22229,7 @@ define inreg <26 x float> @bitcast_v52f16_to_v26f32_scalar(<52 x half> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -26704,6 +26916,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -26723,145 +26936,189 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -26869,43 +27126,47 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_cbranch_scc1 .LBB43_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -26915,27 +27176,27 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -26945,61 +27206,66 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -27026,17 +27292,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -27057,6 +27313,7 @@ define inreg <13 x i64> @bitcast_v52i16_to_v13i64_scalar(<52 x i16> inreg %a, i3
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -30316,159 +30573,182 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -30477,9 +30757,9 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -30491,7 +30771,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -30499,11 +30779,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -30514,7 +30794,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -30523,11 +30803,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -30538,7 +30818,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -30597,9 +30877,10 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -30607,10 +30888,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -30618,12 +30900,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -30631,10 +30915,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -30642,12 +30927,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -30655,10 +30942,11 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -30666,12 +30954,14 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -30720,34 +31010,27 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -30757,6 +31040,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -30776,143 +31060,172 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -30922,39 +31235,39 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -30962,67 +31275,81 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -31061,18 +31388,8 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -31093,6 +31410,7 @@ define inreg <13 x i64> @bitcast_v52f16_to_v13i64_scalar(<52 x half> inreg %a, i
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -34832,6 +35150,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v26, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v26, s34, 0
; VI-NEXT: v_writelane_b32 v26, s35, 1
@@ -34851,145 +35170,189 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v26, s65, 15
; VI-NEXT: v_writelane_b32 v26, s66, 16
; VI-NEXT: v_writelane_b32 v26, s67, 17
-; VI-NEXT: v_writelane_b32 v26, s68, 18
-; VI-NEXT: v_writelane_b32 v26, s69, 19
-; VI-NEXT: v_writelane_b32 v26, s70, 20
-; VI-NEXT: v_writelane_b32 v26, s71, 21
-; VI-NEXT: v_writelane_b32 v26, s80, 22
-; VI-NEXT: v_writelane_b32 v26, s81, 23
-; VI-NEXT: v_writelane_b32 v26, s82, 24
-; VI-NEXT: v_writelane_b32 v26, s83, 25
-; VI-NEXT: v_writelane_b32 v26, s84, 26
-; VI-NEXT: v_writelane_b32 v26, s85, 27
-; VI-NEXT: v_writelane_b32 v26, s30, 28
-; VI-NEXT: v_writelane_b32 v26, s31, 29
+; VI-NEXT: v_writelane_b32 v26, s30, 18
+; VI-NEXT: v_writelane_b32 v26, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v11
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v10
-; VI-NEXT: v_readfirstlane_b32 s11, v9
-; VI-NEXT: v_readfirstlane_b32 s13, v8
-; VI-NEXT: v_readfirstlane_b32 s15, v7
-; VI-NEXT: v_readfirstlane_b32 s73, v6
-; VI-NEXT: v_readfirstlane_b32 s75, v5
-; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s90, v3
-; VI-NEXT: v_readfirstlane_b32 vcc_hi, v2
+; VI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v27, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v8
+; VI-NEXT: v_writelane_b32 v27, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s11, 16
+; VI-NEXT: v_readfirstlane_b32 s13, v7
+; VI-NEXT: v_writelane_b32 v27, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s13, 16
+; VI-NEXT: v_writelane_b32 v27, s58, 3
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: v_writelane_b32 v27, s57, 4
+; VI-NEXT: s_lshr_b32 s8, s16, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v6
+; VI-NEXT: v_readfirstlane_b32 s73, v5
+; VI-NEXT: v_readfirstlane_b32 s75, v4
+; VI-NEXT: v_readfirstlane_b32 vcc_hi, v3
+; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
; VI-NEXT: v_readfirstlane_b32 s30, v1
; VI-NEXT: v_readfirstlane_b32 s35, v0
-; VI-NEXT: s_lshr_b32 s76, s29, 16
-; VI-NEXT: s_lshr_b32 s79, s28, 16
-; VI-NEXT: s_lshr_b32 s89, s27, 16
-; VI-NEXT: s_lshr_b32 s91, s26, 16
-; VI-NEXT: s_lshr_b32 s71, s25, 16
+; VI-NEXT: v_writelane_b32 v27, s40, 5
+; VI-NEXT: s_lshr_b32 s74, s29, 16
+; VI-NEXT: s_lshr_b32 s77, s28, 16
+; VI-NEXT: s_lshr_b32 s88, s27, 16
+; VI-NEXT: s_lshr_b32 s89, s26, 16
+; VI-NEXT: s_lshr_b32 s62, s25, 16
; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s82, s22, 16
-; VI-NEXT: s_lshr_b32 s84, s21, 16
-; VI-NEXT: s_lshr_b32 s83, s20, 16
-; VI-NEXT: s_lshr_b32 s4, s19, 16
-; VI-NEXT: s_lshr_b32 s68, s18, 16
-; VI-NEXT: s_lshr_b32 s69, s17, 16
-; VI-NEXT: s_lshr_b32 s70, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s7, 16
-; VI-NEXT: s_lshr_b32 s8, s9, 16
-; VI-NEXT: s_lshr_b32 s10, s11, 16
-; VI-NEXT: s_lshr_b32 s12, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s72, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s77, s78, 16
-; VI-NEXT: s_lshr_b32 s88, s90, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
-; VI-NEXT: s_lshr_b32 s80, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s23, 16
+; VI-NEXT: s_lshr_b32 s65, s22, 16
+; VI-NEXT: s_lshr_b32 s66, s21, 16
+; VI-NEXT: s_lshr_b32 s67, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s14, s17, 16
+; VI-NEXT: s_lshr_b32 s72, s15, 16
+; VI-NEXT: s_lshr_b32 s12, s73, 16
+; VI-NEXT: s_lshr_b32 s76, s75, 16
+; VI-NEXT: s_lshr_b32 s79, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s91, vcc_lo, 16
+; VI-NEXT: s_lshr_b32 s63, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: v_readfirstlane_b32 s41, v12
+; VI-NEXT: v_writelane_b32 v27, s8, 6
+; VI-NEXT: s_cmp_lg_u32 s41, 0
+; VI-NEXT: v_writelane_b32 v27, s6, 7
+; VI-NEXT: v_writelane_b32 v27, s14, 8
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s68, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s41, 0xffff, s16
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s41, s8
+; VI-NEXT: s_and_b32 s8, 0xffff, s18
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s38, s8, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s19
+; VI-NEXT: s_lshl_b32 s8, s40, 16
+; VI-NEXT: s_or_b32 s39, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s20
+; VI-NEXT: s_lshl_b32 s8, s67, 16
+; VI-NEXT: s_or_b32 s40, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s21
+; VI-NEXT: s_lshl_b32 s8, s66, 16
+; VI-NEXT: s_and_b32 s42, 0xffff, s17
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s41, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s22
+; VI-NEXT: s_lshl_b32 s8, s65, 16
+; VI-NEXT: s_or_b32 s37, s42, s14
+; VI-NEXT: s_or_b32 s42, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s23
+; VI-NEXT: s_lshl_b32 s8, s64, 16
+; VI-NEXT: s_or_b32 s43, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s8, s31, 16
+; VI-NEXT: s_or_b32 s44, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s8, s62, 16
+; VI-NEXT: s_or_b32 s45, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s8, s89, 16
+; VI-NEXT: s_or_b32 s46, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s8, s88, 16
+; VI-NEXT: s_or_b32 s47, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s8, s77, 16
+; VI-NEXT: s_or_b32 s48, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s8, s74, 16
+; VI-NEXT: s_or_b32 s49, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s8, s34, 16
+; VI-NEXT: s_or_b32 s50, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s8, s63, 16
+; VI-NEXT: s_or_b32 s51, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s8, s91, 16
+; VI-NEXT: s_or_b32 s52, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s8, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s8, s76, 16
+; VI-NEXT: s_or_b32 s54, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s8, s12, 16
+; VI-NEXT: s_or_b32 s55, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s8, s72, 16
+; VI-NEXT: s_or_b32 s56, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s8, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s8, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s8, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s8
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s69, 16
-; VI-NEXT: s_or_b32 s39, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s5, s83, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s5, s84, 16
-; VI-NEXT: s_or_b32 s41, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s42, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s5, s81, 16
-; VI-NEXT: s_or_b32 s43, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, s80, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s90
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s77, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s6, s4
-; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: v_writelane_b32 v27, s89, 9
+; VI-NEXT: s_mov_b32 s89, s79
+; VI-NEXT: s_mov_b32 s79, s76
+; VI-NEXT: s_mov_b32 s76, s12
+; VI-NEXT: s_mov_b32 s12, s72
+; VI-NEXT: s_mov_b32 s72, s62
+; VI-NEXT: s_mov_b32 s6, s91
+; VI-NEXT: s_mov_b32 s91, s88
+; VI-NEXT: s_mov_b32 s88, s77
+; VI-NEXT: s_mov_b32 s77, s74
+; VI-NEXT: s_mov_b32 s74, s63
+; VI-NEXT: s_mov_b32 s78, s73
+; VI-NEXT: s_mov_b32 s73, s13
+; VI-NEXT: s_mov_b32 s13, s10
+; VI-NEXT: s_mov_b32 s10, s64
+; VI-NEXT: s_mov_b32 s14, s65
+; VI-NEXT: s_mov_b32 s90, s75
+; VI-NEXT: s_mov_b32 s75, s15
+; VI-NEXT: s_mov_b32 s15, s11
+; VI-NEXT: s_mov_b32 s11, s9
+; VI-NEXT: s_mov_b32 s9, s7
+; VI-NEXT: s_mov_b32 s7, s66
+; VI-NEXT: s_mov_b32 s8, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; VI-NEXT: s_mov_b64 s[4:5], -1
+; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s11
+; VI-NEXT: s_mov_b32 s11, s15
+; VI-NEXT: s_mov_b32 s15, s75
+; VI-NEXT: s_mov_b32 s75, s90
+; VI-NEXT: s_mov_b32 s65, s14
+; VI-NEXT: s_mov_b32 s64, s10
+; VI-NEXT: s_mov_b32 s10, s13
+; VI-NEXT: s_mov_b32 s13, s73
+; VI-NEXT: s_mov_b32 s73, s78
+; VI-NEXT: s_mov_b32 s63, s74
+; VI-NEXT: s_mov_b32 s74, s77
+; VI-NEXT: s_mov_b32 s77, s88
+; VI-NEXT: s_mov_b32 s88, s91
+; VI-NEXT: s_mov_b32 s91, s6
+; VI-NEXT: s_mov_b32 s62, s72
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s76
+; VI-NEXT: s_mov_b32 s76, s79
+; VI-NEXT: s_mov_b32 s79, s89
+; VI-NEXT: v_readlane_b32 s89, v27, 9
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -34997,43 +35360,47 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB51_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
-; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 6
+; VI-NEXT: s_and_b32 s5, s16, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s36, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s69, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: s_add_i32 s17, s17, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 8
+; VI-NEXT: s_and_b32 s5, s17, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s37, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: s_add_i32 s18, s18, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 7
+; VI-NEXT: s_and_b32 s5, s18, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s38, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s19, s19, 3
+; VI-NEXT: v_readlane_b32 s4, v27, 5
+; VI-NEXT: s_and_b32 s5, s19, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s83, 16
+; VI-NEXT: s_lshl_b32 s5, s67, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s66, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -35043,27 +35410,27 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s62, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s26, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
+; VI-NEXT: s_lshl_b32 s5, s89, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s27, s27, 3
; VI-NEXT: s_add_i32 s46, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s27, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
+; VI-NEXT: s_lshl_b32 s5, s77, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s49, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -35073,61 +35440,66 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: s_add_i32 s50, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s63, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_add_i32 s4, vcc_lo, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s91, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s90, s90, 3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s90, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
+; VI-NEXT: s_add_i32 s4, vcc_hi, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s78, s78, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s78, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s77, 16
+; VI-NEXT: s_add_i32 s4, s75, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
+; VI-NEXT: s_add_i32 s4, s73, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
+; VI-NEXT: s_add_i32 s4, s15, 3
+; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s4, s13, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 4
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: s_add_i32 s4, s11, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 3
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s4, s10, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 2
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s8, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s4, s9, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 1
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s4, s7, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v27, 0
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v26, 28
+; VI-NEXT: v_readlane_b32 s30, v26, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -35154,17 +35526,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v23, s59
; VI-NEXT: v_mov_b32_e32 v24, s60
; VI-NEXT: v_mov_b32_e32 v25, s61
-; VI-NEXT: v_readlane_b32 s31, v26, 29
-; VI-NEXT: v_readlane_b32 s85, v26, 27
-; VI-NEXT: v_readlane_b32 s84, v26, 26
-; VI-NEXT: v_readlane_b32 s83, v26, 25
-; VI-NEXT: v_readlane_b32 s82, v26, 24
-; VI-NEXT: v_readlane_b32 s81, v26, 23
-; VI-NEXT: v_readlane_b32 s80, v26, 22
-; VI-NEXT: v_readlane_b32 s71, v26, 21
-; VI-NEXT: v_readlane_b32 s70, v26, 20
-; VI-NEXT: v_readlane_b32 s69, v26, 19
-; VI-NEXT: v_readlane_b32 s68, v26, 18
+; VI-NEXT: v_readlane_b32 s31, v26, 19
; VI-NEXT: v_readlane_b32 s67, v26, 17
; VI-NEXT: v_readlane_b32 s66, v26, 16
; VI-NEXT: v_readlane_b32 s65, v26, 15
@@ -35185,6 +35547,7 @@ define inreg <13 x double> @bitcast_v52i16_to_v13f64_scalar(<52 x i16> inreg %a,
; VI-NEXT: v_readlane_b32 s34, v26, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v26, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v27, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -38632,159 +38995,182 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
-; SI-NEXT: v_writelane_b32 v32, s34, 0
-; SI-NEXT: v_writelane_b32 v32, s35, 1
-; SI-NEXT: v_writelane_b32 v32, s36, 2
-; SI-NEXT: v_writelane_b32 v32, s37, 3
-; SI-NEXT: v_writelane_b32 v32, s38, 4
-; SI-NEXT: v_writelane_b32 v32, s39, 5
-; SI-NEXT: v_writelane_b32 v32, s48, 6
-; SI-NEXT: v_writelane_b32 v32, s49, 7
-; SI-NEXT: v_writelane_b32 v32, s50, 8
-; SI-NEXT: v_writelane_b32 v32, s51, 9
-; SI-NEXT: v_writelane_b32 v32, s52, 10
-; SI-NEXT: v_writelane_b32 v32, s53, 11
-; SI-NEXT: v_writelane_b32 v32, s54, 12
-; SI-NEXT: v_writelane_b32 v32, s55, 13
-; SI-NEXT: v_writelane_b32 v32, s64, 14
-; SI-NEXT: v_writelane_b32 v32, s65, 15
-; SI-NEXT: v_writelane_b32 v32, s66, 16
-; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s30, 24
-; SI-NEXT: v_writelane_b32 v32, s31, 25
+; SI-NEXT: s_waitcnt expcnt(1)
+; SI-NEXT: v_writelane_b32 v32, s36, 0
+; SI-NEXT: v_writelane_b32 v32, s37, 1
+; SI-NEXT: v_writelane_b32 v32, s38, 2
+; SI-NEXT: v_writelane_b32 v32, s39, 3
+; SI-NEXT: v_writelane_b32 v32, s48, 4
+; SI-NEXT: v_writelane_b32 v32, s49, 5
+; SI-NEXT: v_writelane_b32 v32, s50, 6
+; SI-NEXT: v_writelane_b32 v32, s51, 7
+; SI-NEXT: v_writelane_b32 v32, s52, 8
+; SI-NEXT: v_writelane_b32 v32, s53, 9
+; SI-NEXT: v_writelane_b32 v32, s54, 10
+; SI-NEXT: v_writelane_b32 v32, s55, 11
+; SI-NEXT: v_writelane_b32 v32, s64, 12
+; SI-NEXT: v_writelane_b32 v32, s65, 13
+; SI-NEXT: v_writelane_b32 v32, s66, 14
+; SI-NEXT: v_writelane_b32 v32, s67, 15
+; SI-NEXT: v_writelane_b32 v32, s30, 16
+; SI-NEXT: v_writelane_b32 v32, s31, 17
; SI-NEXT: v_readfirstlane_b32 s6, v11
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v10
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v9
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s59, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v8
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s58, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 2
+; SI-NEXT: s_lshr_b32 s57, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 3
+; SI-NEXT: s_lshr_b32 s56, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v5
+; SI-NEXT: v_writelane_b32 v33, s57, 4
+; SI-NEXT: s_lshr_b32 s60, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v4
+; SI-NEXT: v_writelane_b32 v33, s56, 5
+; SI-NEXT: s_lshr_b32 s61, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v3
; SI-NEXT: v_readfirstlane_b32 s88, v2
; SI-NEXT: v_readfirstlane_b32 s91, v1
; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_writelane_b32 v33, s60, 6
; SI-NEXT: s_lshr_b32 s90, s29, 16
; SI-NEXT: s_lshr_b32 s93, s28, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; SI-NEXT: s_lshr_b32 s4, s25, 16
-; SI-NEXT: s_lshr_b32 s30, s24, 16
-; SI-NEXT: s_lshr_b32 s31, s23, 16
-; SI-NEXT: s_lshr_b32 s34, s22, 16
-; SI-NEXT: s_lshr_b32 s35, s21, 16
-; SI-NEXT: s_lshr_b32 s69, s20, 16
-; SI-NEXT: s_lshr_b32 s70, s19, 16
-; SI-NEXT: s_lshr_b32 s71, s18, 16
-; SI-NEXT: s_lshr_b32 s80, s17, 16
-; SI-NEXT: s_lshr_b32 s81, s16, 16
-; SI-NEXT: s_lshr_b32 s68, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s7, s24, 16
+; SI-NEXT: s_lshr_b32 s9, s23, 16
+; SI-NEXT: s_lshr_b32 s11, s22, 16
+; SI-NEXT: s_lshr_b32 s13, s21, 16
+; SI-NEXT: s_lshr_b32 s15, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s43, s16, 16
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: s_lshr_b32 s92, s91, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v12
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s44, v12
+; SI-NEXT: v_writelane_b32 v33, s61, 7
+; SI-NEXT: s_cmp_lg_u32 s44, 0
+; SI-NEXT: v_writelane_b32 v33, s62, 8
+; SI-NEXT: v_writelane_b32 v33, s63, 9
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_and_b32 s44, s16, 0xffff
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s36, s44, s43
+; SI-NEXT: s_and_b32 s43, s17, 0xffff
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s37, s43, s42
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s42, s41
+; SI-NEXT: s_and_b32 s41, s19, 0xffff
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s41, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s40, s40, s15
+; SI-NEXT: s_and_b32 s41, s21, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s41, s41, s13
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s42, s42, s11
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s43, s43, s9
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s44, s44, s7
+; SI-NEXT: s_and_b32 s46, s25, 0xffff
+; SI-NEXT: s_mov_b32 s7, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s46, s45
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s47, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s46, s47
+; SI-NEXT: s_and_b32 s47, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s30, vcc_lo, 16
+; SI-NEXT: s_or_b32 s47, s47, s30
+; SI-NEXT: s_and_b32 s30, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s93, 16
+; SI-NEXT: s_or_b32 s48, s30, s31
+; SI-NEXT: s_and_b32 s30, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s90, 16
+; SI-NEXT: s_or_b32 s49, s30, s31
+; SI-NEXT: s_and_b32 s30, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s95, 16
+; SI-NEXT: s_or_b32 s50, s30, s31
+; SI-NEXT: s_and_b32 s30, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s92, 16
+; SI-NEXT: s_or_b32 s51, s30, s31
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s52, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s53, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s54, s62, s61
+; SI-NEXT: s_and_b32 s61, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s55, s61, s60
+; SI-NEXT: s_and_b32 s60, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s60, s56
+; SI-NEXT: s_and_b32 s60, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s60, s57
+; SI-NEXT: s_and_b32 s60, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s60, s58
+; SI-NEXT: s_and_b32 s60, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s60, s59
+; SI-NEXT: s_and_b32 s60, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s60, s60, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s45, s5, s4
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
+; SI-NEXT: s_or_b32 s61, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s89, s43
+; SI-NEXT: s_mov_b32 s79, s42
+; SI-NEXT: s_mov_b32 s77, s41
+; SI-NEXT: s_mov_b32 s75, s40
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s45
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -38793,9 +39179,9 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s89
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s79
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -38807,7 +39193,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v4, s18
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s77
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -38815,11 +39201,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_or_b32_e32 v1, v3, v1
-; SI-NEXT: v_cvt_f32_f16_e32 v3, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v3, s75
; SI-NEXT: v_lshlrev_b32_e32 v2, 16, v2
; SI-NEXT: v_or_b32_e32 v2, v4, v2
; SI-NEXT: v_cvt_f32_f16_e32 v4, s19
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s73
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
@@ -38830,7 +39216,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_or_b32_e32 v3, v4, v3
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s15
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
@@ -38839,11 +39225,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_or_b32_e32 v4, v6, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v5
; SI-NEXT: v_or_b32_e32 v5, v7, v5
; SI-NEXT: v_cvt_f32_f16_e32 v7, s22
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s11
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
@@ -38854,7 +39240,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_or_b32_e32 v6, v7, v6
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s9
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
@@ -38913,9 +39299,10 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v13, v15, v13
; SI-NEXT: v_cvt_f32_f16_e32 v15, s92
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
; SI-NEXT: v_cvt_f32_f16_e32 v16, s91
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
@@ -38923,10 +39310,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v18, s88
; SI-NEXT: v_or_b32_e32 v15, v16, v15
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_cvt_f32_f16_e32 v19, s78
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
@@ -38934,12 +39322,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v16, v18, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v17, v19, v17
; SI-NEXT: v_cvt_f32_f16_e32 v19, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
@@ -38947,10 +39337,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v21, s74
; SI-NEXT: v_or_b32_e32 v18, v19, v18
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_cvt_f32_f16_e32 v22, s72
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
@@ -38958,12 +39349,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v19, v21, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v20, v22, v20
; SI-NEXT: v_cvt_f32_f16_e32 v22, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
@@ -38971,10 +39364,11 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v24, s12
; SI-NEXT: v_or_b32_e32 v21, v22, v21
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_cvt_f32_f16_e32 v25, s10
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -38982,12 +39376,14 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v22, v24, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v23, v25, v23
; SI-NEXT: v_cvt_f32_f16_e32 v25, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s6
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
@@ -39036,34 +39432,27 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 24
-; SI-NEXT: v_readlane_b32 s31, v32, 25
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
-; SI-NEXT: v_readlane_b32 s67, v32, 17
-; SI-NEXT: v_readlane_b32 s66, v32, 16
-; SI-NEXT: v_readlane_b32 s65, v32, 15
-; SI-NEXT: v_readlane_b32 s64, v32, 14
-; SI-NEXT: v_readlane_b32 s55, v32, 13
-; SI-NEXT: v_readlane_b32 s54, v32, 12
-; SI-NEXT: v_readlane_b32 s53, v32, 11
-; SI-NEXT: v_readlane_b32 s52, v32, 10
-; SI-NEXT: v_readlane_b32 s51, v32, 9
-; SI-NEXT: v_readlane_b32 s50, v32, 8
-; SI-NEXT: v_readlane_b32 s49, v32, 7
-; SI-NEXT: v_readlane_b32 s48, v32, 6
-; SI-NEXT: v_readlane_b32 s39, v32, 5
-; SI-NEXT: v_readlane_b32 s38, v32, 4
-; SI-NEXT: v_readlane_b32 s37, v32, 3
-; SI-NEXT: v_readlane_b32 s36, v32, 2
-; SI-NEXT: v_readlane_b32 s35, v32, 1
-; SI-NEXT: v_readlane_b32 s34, v32, 0
+; SI-NEXT: v_readlane_b32 s30, v32, 16
+; SI-NEXT: v_readlane_b32 s31, v32, 17
+; SI-NEXT: v_readlane_b32 s67, v32, 15
+; SI-NEXT: v_readlane_b32 s66, v32, 14
+; SI-NEXT: v_readlane_b32 s65, v32, 13
+; SI-NEXT: v_readlane_b32 s64, v32, 12
+; SI-NEXT: v_readlane_b32 s55, v32, 11
+; SI-NEXT: v_readlane_b32 s54, v32, 10
+; SI-NEXT: v_readlane_b32 s53, v32, 9
+; SI-NEXT: v_readlane_b32 s52, v32, 8
+; SI-NEXT: v_readlane_b32 s51, v32, 7
+; SI-NEXT: v_readlane_b32 s50, v32, 6
+; SI-NEXT: v_readlane_b32 s49, v32, 5
+; SI-NEXT: v_readlane_b32 s48, v32, 4
+; SI-NEXT: v_readlane_b32 s39, v32, 3
+; SI-NEXT: v_readlane_b32 s38, v32, 2
+; SI-NEXT: v_readlane_b32 s37, v32, 1
+; SI-NEXT: v_readlane_b32 s36, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -39073,6 +39462,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; VI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: v_writelane_b32 v32, s34, 0
; VI-NEXT: v_writelane_b32 v32, s35, 1
@@ -39092,143 +39482,172 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s30, 28
-; VI-NEXT: v_writelane_b32 v32, s31, 29
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v11
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v10
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v9
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s59, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v8
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s58, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 2
+; VI-NEXT: s_lshr_b32 s57, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 3
+; VI-NEXT: s_lshr_b32 s61, s72, 16
; VI-NEXT: v_readfirstlane_b32 s75, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 4
+; VI-NEXT: s_lshr_b32 s62, s75, 16
; VI-NEXT: v_readfirstlane_b32 s78, v4
-; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: v_writelane_b32 v33, s61, 5
+; VI-NEXT: s_lshr_b32 s63, s78, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 6
+; VI-NEXT: v_readfirstlane_b32 s34, v1
+; VI-NEXT: v_writelane_b32 v33, s63, 7
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; VI-NEXT: v_readfirstlane_b32 s30, v1
-; VI-NEXT: v_readfirstlane_b32 s68, v0
-; VI-NEXT: s_lshr_b32 s74, s29, 16
-; VI-NEXT: s_lshr_b32 s77, s28, 16
-; VI-NEXT: s_lshr_b32 s88, s27, 16
-; VI-NEXT: s_lshr_b32 s90, s26, 16
+; VI-NEXT: v_readfirstlane_b32 s51, v0
+; VI-NEXT: v_writelane_b32 v33, s34, 8
+; VI-NEXT: v_readfirstlane_b32 s89, v3
+; VI-NEXT: s_lshr_b32 s31, vcc_lo, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 9
+; VI-NEXT: s_lshr_b32 s30, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s31, 10
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s60, s28, 16
+; VI-NEXT: s_lshr_b32 s9, s27, 16
+; VI-NEXT: s_lshr_b32 s7, s26, 16
; VI-NEXT: s_lshr_b32 vcc_hi, s25, 16
-; VI-NEXT: s_lshr_b32 s31, s24, 16
-; VI-NEXT: s_lshr_b32 s35, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s85, s6, 16
-; VI-NEXT: s_lshr_b32 s9, s8, 16
-; VI-NEXT: s_lshr_b32 s11, s10, 16
-; VI-NEXT: s_lshr_b32 s13, s12, 16
-; VI-NEXT: s_lshr_b32 s15, s14, 16
-; VI-NEXT: s_lshr_b32 s73, s72, 16
-; VI-NEXT: s_lshr_b32 s76, s75, 16
-; VI-NEXT: s_lshr_b32 s79, s78, 16
-; VI-NEXT: s_lshr_b32 s91, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v12
-; VI-NEXT: s_cmp_lg_u32 s5, 0
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s45, s19, 16
+; VI-NEXT: s_lshr_b32 s11, s18, 16
+; VI-NEXT: s_lshr_b32 s46, s17, 16
+; VI-NEXT: s_lshr_b32 s13, s16, 16
+; VI-NEXT: s_lshr_b32 s35, s34, 16
+; VI-NEXT: s_lshr_b32 s50, s51, 16
+; VI-NEXT: v_readfirstlane_b32 s15, v12
+; VI-NEXT: v_writelane_b32 v33, s30, 11
+; VI-NEXT: s_cmp_lg_u32 s15, 0
+; VI-NEXT: v_writelane_b32 v33, s35, 12
+; VI-NEXT: v_writelane_b32 v33, s56, 13
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s7, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s7, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s7, s81, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s7, s80, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s7, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s7, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s7, s35, 16
-; VI-NEXT: s_or_b32 s43, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s7, s31, 16
-; VI-NEXT: s_or_b32 s44, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; VI-NEXT: s_or_b32 s45, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s7, s90, 16
-; VI-NEXT: s_or_b32 s46, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s7, s88, 16
-; VI-NEXT: s_or_b32 s47, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s7, s77, 16
-; VI-NEXT: s_or_b32 s48, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s7, s74, 16
-; VI-NEXT: s_or_b32 s49, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s7, s69, 16
-; VI-NEXT: s_or_b32 s50, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s7, s34, 16
-; VI-NEXT: s_or_b32 s51, s5, s7
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s73, s13, 16
+; VI-NEXT: s_and_b32 s74, 0xffff, s17
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s37, s74, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s38, s46, s11
+; VI-NEXT: s_and_b32 s46, 0xffff, s19
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s39, s46, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s45, s40
+; VI-NEXT: s_and_b32 s45, 0xffff, s21
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_or_b32 s41, s45, s41
+; VI-NEXT: s_and_b32 s45, 0xffff, s22
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s36, s47, s73
+; VI-NEXT: s_or_b32 s42, s45, s42
+; VI-NEXT: s_and_b32 s45, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s45, s43
+; VI-NEXT: s_and_b32 s45, 0xffff, s24
+; VI-NEXT: s_mov_b32 s77, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s45, s44
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s46, s46, s7
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s47, s47, s9
+; VI-NEXT: s_and_b32 s76, 0xffff, s28
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s48, s76, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s49, s60, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s60, s50, 16
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_or_b32 s50, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s60, s35, 16
+; VI-NEXT: s_or_b32 s51, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s60, s31, 16
+; VI-NEXT: s_or_b32 s52, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s60, s30, 16
+; VI-NEXT: s_or_b32 s53, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s60, s63, 16
+; VI-NEXT: s_or_b32 s54, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s60, s62, 16
+; VI-NEXT: s_or_b32 s55, s56, s60
+; VI-NEXT: s_and_b32 s56, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s61, 16
+; VI-NEXT: s_or_b32 s56, s56, s60
+; VI-NEXT: s_and_b32 s60, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s60, s57
+; VI-NEXT: s_and_b32 s60, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s60, s58
+; VI-NEXT: s_and_b32 s60, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s60, s59
+; VI-NEXT: s_and_b32 s60, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s60, s60, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s52, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s73, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s9, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
+; VI-NEXT: s_or_b32 s61, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s7, s4
+; VI-NEXT: s_mov_b32 s91, s7
+; VI-NEXT: s_mov_b32 s7, s9
+; VI-NEXT: s_mov_b32 s9, s60
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s46
+; VI-NEXT: s_mov_b32 s90, s11
+; VI-NEXT: s_mov_b32 s11, s45
+; VI-NEXT: s_mov_b32 s74, s40
+; VI-NEXT: s_mov_b32 s88, s41
+; VI-NEXT: s_mov_b32 s79, s42
+; VI-NEXT: s_mov_b32 s76, s50
+; VI-NEXT: s_mov_b32 s73, s43
+; VI-NEXT: s_mov_b32 s77, s44
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -39238,39 +39657,39 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v25, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s84
-; VI-NEXT: v_mov_b32_e32 v2, s83
+; VI-NEXT: v_mov_b32_e32 v0, s15
+; VI-NEXT: v_mov_b32_e32 v2, s13
; VI-NEXT: v_add_f16_sdwa v0, v0, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v25
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v25
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s82
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v25
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s81
+; VI-NEXT: v_mov_b32_e32 v3, s11
; VI-NEXT: v_add_f16_sdwa v3, v3, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v25
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s80
+; VI-NEXT: v_mov_b32_e32 v4, s74
; VI-NEXT: v_add_f16_sdwa v4, v4, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v25
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s71
+; VI-NEXT: v_mov_b32_e32 v5, s88
; VI-NEXT: v_add_f16_sdwa v5, v5, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v25
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s70
+; VI-NEXT: v_mov_b32_e32 v6, s79
; VI-NEXT: v_add_f16_sdwa v6, v6, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v25
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s35
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v25
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s31
+; VI-NEXT: v_mov_b32_e32 v8, s77
; VI-NEXT: v_add_f16_sdwa v8, v8, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v25
; VI-NEXT: v_or_b32_e32 v8, v9, v8
@@ -39278,67 +39697,81 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v9, v9, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v25
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s90
+; VI-NEXT: v_mov_b32_e32 v10, s91
; VI-NEXT: v_add_f16_sdwa v10, v10, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v25
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s88
+; VI-NEXT: v_mov_b32_e32 v11, s7
; VI-NEXT: v_add_f16_sdwa v11, v11, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v25
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s77
+; VI-NEXT: v_mov_b32_e32 v12, s9
; VI-NEXT: v_add_f16_sdwa v12, v12, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s74
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v25
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s69
+; VI-NEXT: v_mov_b32_e32 v14, s76
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v14, v14, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s68, v25
+; VI-NEXT: v_add_f16_e32 v15, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s34
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_add_f16_sdwa v15, v15, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s30, v25
+; VI-NEXT: v_add_f16_e32 v16, s4, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s7
+; VI-NEXT: v_mov_b32_e32 v16, s4
; VI-NEXT: v_add_f16_sdwa v16, v16, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v17, vcc_lo, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s91
+; VI-NEXT: v_mov_b32_e32 v17, s4
; VI-NEXT: v_add_f16_sdwa v17, v17, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v18, s89, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s79
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s78, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s76
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s75, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s73
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s72, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s15
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s14, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s13
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s12, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v22, v23, v22
-; VI-NEXT: v_mov_b32_e32 v23, s11
+; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s10, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v23, v24, v23
-; VI-NEXT: v_mov_b32_e32 v24, s9
+; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s8, v25
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v24, v26, v24
-; VI-NEXT: v_mov_b32_e32 v26, s85
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v25 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s6, v25
; VI-NEXT: v_or_b32_e32 v25, v25, v26
@@ -39377,18 +39810,8 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 28
-; VI-NEXT: v_readlane_b32 s31, v32, 29
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -39409,6 +39832,7 @@ define inreg <13 x double> @bitcast_v52f16_to_v13f64_scalar(<52 x half> inreg %a
; VI-NEXT: v_readlane_b32 s34, v32, 0
; VI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; VI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; VI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; VI-NEXT: s_mov_b64 exec, s[4:5]
; VI-NEXT: s_waitcnt vmcnt(0)
; VI-NEXT: s_setpc_b64 s[30:31]
@@ -41003,10 +41427,8 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v26, s51, 9
; SI-NEXT: v_writelane_b32 v26, s52, 10
; SI-NEXT: v_writelane_b32 v26, s53, 11
-; SI-NEXT: v_writelane_b32 v26, s54, 12
-; SI-NEXT: v_writelane_b32 v26, s55, 13
-; SI-NEXT: v_writelane_b32 v26, s30, 14
-; SI-NEXT: v_writelane_b32 v26, s31, 15
+; SI-NEXT: v_writelane_b32 v26, s30, 12
+; SI-NEXT: v_writelane_b32 v26, s31, 13
; SI-NEXT: s_lshr_b32 s40, s16, 16
; SI-NEXT: ; implicit-def: $vgpr27 : SGPR spill to VGPR lane
; SI-NEXT: s_lshr_b32 s14, s18, 16
@@ -41021,21 +41443,21 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s6, s26, 16
; SI-NEXT: v_writelane_b32 v27, s8, 4
; SI-NEXT: s_lshr_b32 s4, s28, 16
-; SI-NEXT: v_readfirstlane_b32 s48, v0
+; SI-NEXT: v_readfirstlane_b32 s92, v0
; SI-NEXT: v_writelane_b32 v27, s6, 5
-; SI-NEXT: v_readfirstlane_b32 s49, v2
-; SI-NEXT: s_lshr_b32 s46, s48, 16
+; SI-NEXT: v_readfirstlane_b32 s94, v2
+; SI-NEXT: s_lshr_b32 s46, s92, 16
; SI-NEXT: v_writelane_b32 v27, s4, 6
-; SI-NEXT: v_readfirstlane_b32 s50, v4
-; SI-NEXT: s_lshr_b32 s45, s49, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
+; SI-NEXT: s_lshr_b32 s45, s94, 16
; SI-NEXT: v_writelane_b32 v27, s46, 7
-; SI-NEXT: v_readfirstlane_b32 s51, v6
-; SI-NEXT: s_lshr_b32 s44, s50, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v6
+; SI-NEXT: s_lshr_b32 s44, vcc_lo, 16
; SI-NEXT: v_writelane_b32 v27, s45, 8
-; SI-NEXT: v_readfirstlane_b32 s52, v8
-; SI-NEXT: s_lshr_b32 s42, s51, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v8
+; SI-NEXT: s_lshr_b32 s42, s30, 16
; SI-NEXT: v_writelane_b32 v27, s44, 9
-; SI-NEXT: s_lshr_b32 s15, s52, 16
+; SI-NEXT: s_lshr_b32 s15, s34, 16
; SI-NEXT: v_writelane_b32 v27, s42, 10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v27, s15, 11
@@ -41045,10 +41467,11 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v27, s41, 13
; SI-NEXT: v_readfirstlane_b32 s11, v7
; SI-NEXT: v_writelane_b32 v27, s13, 14
-; SI-NEXT: v_readfirstlane_b32 s7, v11
-; SI-NEXT: v_readfirstlane_b32 s53, v10
; SI-NEXT: v_readfirstlane_b32 s9, v9
; SI-NEXT: v_writelane_b32 v27, s11, 15
+; SI-NEXT: v_readfirstlane_b32 s5, v11
+; SI-NEXT: v_readfirstlane_b32 s36, v10
+; SI-NEXT: v_writelane_b32 v27, s9, 16
; SI-NEXT: s_lshr_b32 s91, s29, 16
; SI-NEXT: s_lshr_b32 s89, s27, 16
; SI-NEXT: s_lshr_b32 s79, s25, 16
@@ -41056,18 +41479,18 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s75, s21, 16
; SI-NEXT: s_lshr_b32 s73, s19, 16
; SI-NEXT: s_lshr_b32 s63, s17, 16
-; SI-NEXT: s_lshr_b32 s37, s7, 16
-; SI-NEXT: s_lshr_b32 s5, s53, 16
+; SI-NEXT: s_lshr_b32 s37, s5, 16
+; SI-NEXT: s_lshr_b32 s7, s36, 16
; SI-NEXT: s_lshr_b32 s35, s9, 16
; SI-NEXT: s_lshr_b32 s31, s11, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s13, 16
; SI-NEXT: s_lshr_b32 s95, s41, 16
; SI-NEXT: s_lshr_b32 s93, s43, 16
; SI-NEXT: v_readfirstlane_b32 s47, v12
-; SI-NEXT: v_writelane_b32 v27, s9, 16
+; SI-NEXT: v_writelane_b32 v27, s5, 17
; SI-NEXT: s_cmp_lg_u32 s47, 0
-; SI-NEXT: v_writelane_b32 v27, s7, 17
-; SI-NEXT: v_writelane_b32 v27, s5, 18
+; SI-NEXT: v_writelane_b32 v27, s92, 18
+; SI-NEXT: v_writelane_b32 v27, s7, 19
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s47, s16, 0xffff
@@ -41106,19 +41529,19 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s73, s45, s47
; SI-NEXT: s_and_b32 s45, s21, 0xffff
; SI-NEXT: s_lshl_b32 s47, s75, 16
-; SI-NEXT: s_mov_b32 s92, s75
+; SI-NEXT: s_mov_b32 s38, s75
; SI-NEXT: s_or_b32 s75, s45, s47
; SI-NEXT: s_and_b32 s45, s23, 0xffff
; SI-NEXT: s_lshl_b32 s47, s77, 16
-; SI-NEXT: s_mov_b32 s94, s77
+; SI-NEXT: s_mov_b32 s39, s77
; SI-NEXT: s_or_b32 s77, s45, s47
; SI-NEXT: s_and_b32 s45, s25, 0xffff
; SI-NEXT: s_lshl_b32 s47, s79, 16
-; SI-NEXT: s_mov_b32 vcc_lo, s79
+; SI-NEXT: s_mov_b32 s48, s79
; SI-NEXT: s_or_b32 s79, s45, s47
; SI-NEXT: s_and_b32 s45, s27, 0xffff
; SI-NEXT: s_lshl_b32 s47, s89, 16
-; SI-NEXT: s_mov_b32 s30, s89
+; SI-NEXT: s_mov_b32 s49, s89
; SI-NEXT: s_or_b32 s89, s45, s47
; SI-NEXT: s_and_b32 s45, s29, 0xffff
; SI-NEXT: s_lshl_b32 s47, s91, 16
@@ -41132,82 +41555,96 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s41, vcc_hi, 16
; SI-NEXT: s_or_b32 s47, s13, s41
-; SI-NEXT: s_and_b32 s11, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s13, s31, 16
; SI-NEXT: s_mov_b32 s41, s63
; SI-NEXT: s_lshr_b64 s[62:63], s[62:63], 16
+; SI-NEXT: v_writelane_b32 v27, s62, 20
+; SI-NEXT: v_writelane_b32 v27, s63, 21
+; SI-NEXT: s_mov_b32 s63, s15
+; SI-NEXT: s_mov_b32 s15, s73
+; SI-NEXT: s_lshr_b64 s[50:51], s[72:73], 16
+; SI-NEXT: s_mov_b32 s73, s90
+; SI-NEXT: s_mov_b32 s62, s91
+; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
+; SI-NEXT: s_and_b32 s11, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s13, s31, 16
+; SI-NEXT: s_mov_b32 s91, s62
+; SI-NEXT: s_and_b32 s60, s92, 0xffff
+; SI-NEXT: s_mov_b32 s62, s93
+; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
; SI-NEXT: s_or_b32 s45, s11, s13
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, s35, 16
-; SI-NEXT: s_mov_b32 s63, s15
-; SI-NEXT: s_mov_b32 s15, s73
-; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
; SI-NEXT: s_mov_b32 s13, s75
; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
+; SI-NEXT: s_or_b32 s60, s60, s58
+; SI-NEXT: s_mov_b32 s93, s62
+; SI-NEXT: s_mov_b32 s62, s94
+; SI-NEXT: s_and_b32 s58, s94, 0xffff
+; SI-NEXT: s_mov_b32 s72, s95
+; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
; SI-NEXT: s_or_b32 s43, s9, s11
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s9, s37, 16
-; SI-NEXT: s_mov_b32 s73, s90
-; SI-NEXT: s_mov_b32 s75, s92
-; SI-NEXT: s_mov_b32 s11, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_mov_b32 s92, s91
-; SI-NEXT: s_lshr_b64 s[90:91], s[60:61], 16
-; SI-NEXT: s_or_b32 s55, s7, s9
-; SI-NEXT: s_mov_b32 s77, s94
-; SI-NEXT: s_mov_b32 s9, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
-; SI-NEXT: s_mov_b32 s91, s92
-; SI-NEXT: s_and_b32 s60, s48, 0xffff
-; SI-NEXT: s_mov_b32 s94, s93
-; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
-; SI-NEXT: s_mov_b32 s79, vcc_lo
+; SI-NEXT: s_lshl_b32 s52, s7, 16
+; SI-NEXT: s_mov_b32 s75, s38
; SI-NEXT: s_mov_b32 s7, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[88:89], 16
-; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s93, s94
-; SI-NEXT: s_and_b32 s58, s49, 0xffff
-; SI-NEXT: s_mov_b32 vcc_lo, s95
-; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
-; SI-NEXT: s_mov_b32 s89, s30
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s95, vcc_lo
-; SI-NEXT: s_and_b32 s56, s50, 0xffff
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s95, s72
+; SI-NEXT: s_mov_b32 s72, vcc_lo
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_mov_b32 s38, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[46:47], 16
+; SI-NEXT: s_or_b32 s53, s5, s9
+; SI-NEXT: s_mov_b32 s9, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[78:79], 16
+; SI-NEXT: s_mov_b32 s89, s49
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 vcc_hi, s30
-; SI-NEXT: s_and_b32 s46, s51, 0xffff
-; SI-NEXT: s_mov_b32 s34, s31
+; SI-NEXT: s_mov_b32 vcc_hi, s38
+; SI-NEXT: s_mov_b32 s49, s30
+; SI-NEXT: s_and_b32 s46, s30, 0xffff
+; SI-NEXT: s_mov_b32 s38, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[44:45], 16
-; SI-NEXT: s_lshl_b32 s54, s5, 16
+; SI-NEXT: s_mov_b32 s79, s48
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 s31, s34
-; SI-NEXT: s_and_b32 s44, s52, 0xffff
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s31, s38
+; SI-NEXT: s_mov_b32 s48, s34
+; SI-NEXT: s_and_b32 s44, s34, 0xffff
+; SI-NEXT: s_mov_b32 s38, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
+; SI-NEXT: s_mov_b32 s11, s77
+; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s35, s36
-; SI-NEXT: s_and_b32 s42, s53, 0xffff
+; SI-NEXT: s_mov_b32 s35, s38
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_and_b32 s42, s36, 0xffff
; SI-NEXT: s_mov_b32 s38, s37
-; SI-NEXT: s_lshr_b64 s[36:37], s[54:55], 16
+; SI-NEXT: s_lshr_b64 s[36:37], s[52:53], 16
+; SI-NEXT: s_mov_b32 s77, s39
; SI-NEXT: s_mov_b32 s5, s61
; SI-NEXT: s_mov_b32 s61, s59
; SI-NEXT: s_mov_b32 s59, s57
; SI-NEXT: s_mov_b32 s57, s47
; SI-NEXT: s_mov_b32 s47, s45
; SI-NEXT: s_mov_b32 s45, s43
-; SI-NEXT: s_or_b32 s42, s42, s54
-; SI-NEXT: s_mov_b32 s43, s55
+; SI-NEXT: s_or_b32 s42, s42, s52
+; SI-NEXT: s_mov_b32 s43, s53
; SI-NEXT: s_mov_b32 s37, s38
; SI-NEXT: s_mov_b64 s[38:39], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_mov_b32 s48, s34
+; SI-NEXT: s_mov_b32 s49, s30
+; SI-NEXT: s_mov_b32 s72, vcc_lo
+; SI-NEXT: s_mov_b32 s62, s94
; SI-NEXT: s_mov_b64 s[38:39], -1
+; SI-NEXT: v_writelane_b32 v27, s4, 20
; SI-NEXT: ; implicit-def: $sgpr40
-; SI-NEXT: ; implicit-def: $sgpr62
+; SI-NEXT: v_writelane_b32 v27, s5, 21
; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: ; implicit-def: $sgpr50
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr74
; SI-NEXT: ; implicit-def: $sgpr10
@@ -41234,11 +41671,13 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b64 s[38:39], s[38:39], exec
; SI-NEXT: s_cselect_b32 s38, 1, 0
; SI-NEXT: s_cmp_lg_u32 s38, 1
+; SI-NEXT: v_readlane_b32 s38, v27, 20
+; SI-NEXT: v_readlane_b32 s39, v27, 21
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s53, s53, 3
-; SI-NEXT: v_readlane_b32 s4, v27, 18
-; SI-NEXT: s_and_b32 s5, s53, 0xffff
+; SI-NEXT: s_add_i32 s4, s51, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v27, 19
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -41248,9 +41687,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s52, s52, 3
+; SI-NEXT: s_add_i32 s4, s48, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 11
-; SI-NEXT: s_and_b32 s5, s52, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -41260,9 +41699,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s45, s4, 0x30000
-; SI-NEXT: s_add_i32 s51, s51, 3
+; SI-NEXT: s_add_i32 s4, s49, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 10
-; SI-NEXT: s_and_b32 s5, s51, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -41272,9 +41711,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s47, s4, 0x30000
-; SI-NEXT: s_add_i32 s50, s50, 3
+; SI-NEXT: s_add_i32 s4, s72, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 9
-; SI-NEXT: s_and_b32 s5, s50, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -41284,9 +41723,9 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_add_i32 s49, s49, 3
+; SI-NEXT: s_add_i32 s4, s62, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 8
-; SI-NEXT: s_and_b32 s5, s49, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -41296,9 +41735,10 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_add_i32 s48, s48, 3
+; SI-NEXT: v_readlane_b32 s4, v27, 18
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v27, 7
-; SI-NEXT: s_and_b32 s5, s48, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -41371,7 +41811,6 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s40, s16, 0x30000
; SI-NEXT: s_and_b32 s16, s17, 0xffff
; SI-NEXT: s_lshl_b32 s17, s63, 16
-; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
@@ -41382,11 +41821,10 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s11, s11, 0x30000
; SI-NEXT: s_add_i32 s12, s12, 0x30000
; SI-NEXT: s_add_i32 s13, s13, 0x30000
+; SI-NEXT: s_or_b32 s16, s17, s16
; SI-NEXT: s_add_i32 s14, s14, 0x30000
; SI-NEXT: s_add_i32 s15, s15, 0x30000
; SI-NEXT: s_add_i32 s41, s16, 0x30000
-; SI-NEXT: s_lshr_b64 s[62:63], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[72:73], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[74:75], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[8:9], 16
@@ -41398,6 +41836,8 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[30:31], s[46:47], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[44:45], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[42:43], 16
+; SI-NEXT: s_lshr_b64 s[38:39], s[40:41], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[14:15], 16
; SI-NEXT: s_lshr_b32 s63, s41, 16
; SI-NEXT: s_lshr_b32 s73, s15, 16
; SI-NEXT: s_lshr_b32 s75, s13, 16
@@ -41413,13 +41853,13 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s37, s43, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s16, s40, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s62, 16
+; SI-NEXT: s_lshl_b32 s17, s38, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s41, 0xffff
; SI-NEXT: s_lshl_b32 s18, s63, 16
; SI-NEXT: s_or_b32 s17, s17, s18
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s18, s72, 16
+; SI-NEXT: s_lshl_b32 s18, s50, 16
; SI-NEXT: s_or_b32 s14, s14, s18
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s18, s73, 16
@@ -41490,7 +41930,7 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s29, s43, 0xffff
; SI-NEXT: s_lshl_b32 s40, s37, 16
; SI-NEXT: s_or_b32 s29, s29, s40
-; SI-NEXT: v_readlane_b32 s30, v26, 14
+; SI-NEXT: v_readlane_b32 s30, v26, 12
; SI-NEXT: v_mov_b32_e32 v0, s16
; SI-NEXT: v_mov_b32_e32 v1, s17
; SI-NEXT: v_mov_b32_e32 v2, s14
@@ -41517,9 +41957,7 @@ define inreg <52 x half> @bitcast_v52i16_to_v52f16_scalar(<52 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v23, s27
; SI-NEXT: v_mov_b32_e32 v24, s28
; SI-NEXT: v_mov_b32_e32 v25, s29
-; SI-NEXT: v_readlane_b32 s31, v26, 15
-; SI-NEXT: v_readlane_b32 s55, v26, 13
-; SI-NEXT: v_readlane_b32 s54, v26, 12
+; SI-NEXT: v_readlane_b32 s31, v26, 13
; SI-NEXT: v_readlane_b32 s53, v26, 11
; SI-NEXT: v_readlane_b32 s52, v26, 10
; SI-NEXT: v_readlane_b32 s51, v26, 9
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
index d3b2949c2a190..53ab47d7340c3 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
@@ -6595,8 +6595,9 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -6615,24 +6616,30 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -6640,27 +6647,20 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -6668,102 +6668,133 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB15_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB15_3
; SI-NEXT: .LBB15_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB15_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -6772,42 +6803,46 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -6840,9 +6875,9 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -6879,39 +6914,45 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB15_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -6940,17 +6981,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -6971,6 +7002,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -7000,170 +7032,184 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7172,52 +7218,52 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -7229,25 +7275,29 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -7257,63 +7307,71 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -7342,19 +7400,7 @@ define inreg <28 x i32> @bitcast_v56i16_to_v28i32_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -10897,8 +10943,9 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -10917,153 +10964,181 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -11072,9 +11147,9 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -11084,14 +11159,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -11101,7 +11176,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -11110,11 +11185,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -11125,7 +11200,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -11134,11 +11209,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -11149,7 +11224,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -11158,7 +11233,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -11181,11 +11256,13 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -11194,10 +11271,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -11205,12 +11283,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -11218,10 +11298,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -11229,12 +11310,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -11242,10 +11325,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -11253,12 +11337,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -11266,10 +11352,11 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -11277,12 +11364,14 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -11331,18 +11420,8 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -11363,6 +11442,7 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -11392,174 +11472,188 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -11569,117 +11663,133 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -11718,20 +11828,8 @@ define inreg <28 x i32> @bitcast_v56f16_to_v28i32_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -18147,8 +18245,9 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -18167,24 +18266,30 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -18192,27 +18297,20 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -18220,102 +18318,133 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB31_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB31_3
; SI-NEXT: .LBB31_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB31_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -18324,42 +18453,46 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -18392,9 +18525,9 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -18431,39 +18564,45 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB31_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -18492,17 +18631,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -18523,6 +18652,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -18552,170 +18682,184 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -18724,52 +18868,52 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -18781,25 +18925,29 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -18809,63 +18957,71 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -18894,19 +19050,7 @@ define inreg <28 x float> @bitcast_v56i16_to_v28f32_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -22810,8 +22954,9 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -22830,153 +22975,181 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -22985,9 +23158,9 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -22997,14 +23170,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -23014,7 +23187,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -23023,11 +23196,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -23038,7 +23211,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -23047,11 +23220,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -23062,7 +23235,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -23071,7 +23244,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -23094,11 +23267,13 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -23107,10 +23282,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -23118,12 +23294,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -23131,10 +23309,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -23142,12 +23321,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -23155,10 +23336,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -23166,12 +23348,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -23179,10 +23363,11 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -23190,12 +23375,14 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -23244,18 +23431,8 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -23276,6 +23453,7 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -23305,174 +23483,188 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -23482,117 +23674,133 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -23631,20 +23839,8 @@ define inreg <28 x float> @bitcast_v56f16_to_v28f32_scalar(<56 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -28279,8 +28475,9 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -28299,24 +28496,30 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -28324,27 +28527,20 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -28352,102 +28548,133 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB43_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB43_3
; SI-NEXT: .LBB43_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB43_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -28456,42 +28683,46 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -28524,9 +28755,9 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -28563,39 +28794,45 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB43_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -28624,17 +28861,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -28655,6 +28882,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -28684,170 +28912,184 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -28856,52 +29098,52 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -28913,25 +29155,29 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -28941,63 +29187,71 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -29026,19 +29280,7 @@ define inreg <14 x i64> @bitcast_v56i16_to_v14i64_scalar(<56 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -32628,8 +32870,9 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -32648,153 +32891,181 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -32803,9 +33074,9 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -32815,14 +33086,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -32832,7 +33103,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -32841,11 +33112,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -32856,7 +33127,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -32865,11 +33136,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -32880,7 +33151,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -32889,7 +33160,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -32912,11 +33183,13 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -32925,10 +33198,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -32936,12 +33210,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -32949,10 +33225,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -32960,12 +33237,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -32973,10 +33252,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -32984,12 +33264,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -32997,10 +33279,11 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -33008,12 +33291,14 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -33062,18 +33347,8 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -33094,6 +33369,7 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -33123,174 +33399,188 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -33300,117 +33590,133 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -33449,20 +33755,8 @@ define inreg <14 x i64> @bitcast_v56f16_to_v14i64_scalar(<56 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -37122,8 +37416,9 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v28, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v28, s34, 0
; SI-NEXT: v_writelane_b32 v28, s35, 1
; SI-NEXT: v_writelane_b32 v28, s36, 2
@@ -37142,24 +37437,30 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s82, 24
-; SI-NEXT: v_writelane_b32 v28, s83, 25
-; SI-NEXT: v_writelane_b32 v28, s84, 26
-; SI-NEXT: v_writelane_b32 v28, s85, 27
-; SI-NEXT: v_writelane_b32 v28, s30, 28
-; SI-NEXT: v_writelane_b32 v28, s31, 29
+; SI-NEXT: v_writelane_b32 v28, s30, 18
+; SI-NEXT: v_writelane_b32 v28, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v12
+; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v29, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v10
+; SI-NEXT: v_writelane_b32 v29, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v9
+; SI-NEXT: v_writelane_b32 v29, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v8
+; SI-NEXT: v_writelane_b32 v29, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s73, 16
+; SI-NEXT: v_writelane_b32 v29, s59, 4
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: v_writelane_b32 v29, s58, 5
+; SI-NEXT: s_lshr_b32 s6, s20, 16
; SI-NEXT: v_readfirstlane_b32 s75, v7
; SI-NEXT: v_readfirstlane_b32 s77, v6
; SI-NEXT: v_readfirstlane_b32 s88, v5
@@ -37167,27 +37468,20 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readfirstlane_b32 s94, v3
; SI-NEXT: v_readfirstlane_b32 vcc_hi, v2
; SI-NEXT: v_readfirstlane_b32 s34, v1
-; SI-NEXT: v_readfirstlane_b32 s81, v0
+; SI-NEXT: v_readfirstlane_b32 s66, v0
+; SI-NEXT: v_writelane_b32 v29, s41, 6
; SI-NEXT: s_lshr_b32 s78, s29, 16
; SI-NEXT: s_lshr_b32 s89, s28, 16
; SI-NEXT: s_lshr_b32 s92, s27, 16
; SI-NEXT: s_lshr_b32 s95, s26, 16
; SI-NEXT: s_lshr_b32 s30, s25, 16
; SI-NEXT: s_lshr_b32 s35, s24, 16
-; SI-NEXT: s_lshr_b32 s84, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s4, s21, 16
-; SI-NEXT: s_lshr_b32 s68, s20, 16
-; SI-NEXT: s_lshr_b32 s69, s19, 16
-; SI-NEXT: s_lshr_b32 s70, s18, 16
-; SI-NEXT: s_lshr_b32 s71, s17, 16
-; SI-NEXT: s_lshr_b32 s80, s16, 16
-; SI-NEXT: s_lshr_b32 s85, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
+; SI-NEXT: s_lshr_b32 s72, s23, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s19, 16
+; SI-NEXT: s_lshr_b32 s12, s18, 16
+; SI-NEXT: s_lshr_b32 s40, s17, 16
+; SI-NEXT: s_lshr_b32 s14, s16, 16
; SI-NEXT: s_lshr_b32 s74, s75, 16
; SI-NEXT: s_lshr_b32 s76, s77, 16
; SI-NEXT: s_lshr_b32 s79, s88, 16
@@ -37195,102 +37489,133 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshr_b32 s93, s94, 16
; SI-NEXT: s_lshr_b32 vcc_lo, vcc_hi, 16
; SI-NEXT: s_lshr_b32 s31, s34, 16
-; SI-NEXT: s_lshr_b32 s82, s81, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s64, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: v_writelane_b32 v29, s6, 7
+; SI-NEXT: s_cmp_lg_u32 s8, 0
+; SI-NEXT: v_writelane_b32 v29, s10, 8
+; SI-NEXT: v_writelane_b32 v29, s12, 9
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s71, 16
-; SI-NEXT: s_or_b32 s37, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s68, 16
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s42, s16, 0xffff
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s36, s42, s14
+; SI-NEXT: s_and_b32 s42, s17, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s37, s42, s40
+; SI-NEXT: s_and_b32 s40, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s38, s40, s12
+; SI-NEXT: s_and_b32 s12, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_or_b32 s39, s12, s10
+; SI-NEXT: s_and_b32 s10, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s6, s6, 16
+; SI-NEXT: s_or_b32 s40, s10, s6
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s41, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s67, 16
+; SI-NEXT: s_or_b32 s42, s6, s10
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s72, 16
+; SI-NEXT: s_or_b32 s43, s6, s10
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s35, 16
+; SI-NEXT: s_or_b32 s44, s6, s10
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s30, 16
+; SI-NEXT: s_or_b32 s45, s6, s10
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s95, 16
+; SI-NEXT: s_or_b32 s46, s6, s10
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s92, 16
+; SI-NEXT: s_or_b32 s47, s6, s10
+; SI-NEXT: s_and_b32 s6, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s89, 16
+; SI-NEXT: s_or_b32 s48, s6, s10
+; SI-NEXT: s_and_b32 s6, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s78, 16
+; SI-NEXT: s_or_b32 s49, s6, s10
+; SI-NEXT: s_and_b32 s6, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s64, 16
+; SI-NEXT: s_or_b32 s50, s6, s10
+; SI-NEXT: s_and_b32 s6, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s31, 16
+; SI-NEXT: s_or_b32 s51, s6, s10
+; SI-NEXT: s_and_b32 s6, vcc_hi, 0xffff
+; SI-NEXT: s_lshl_b32 s10, vcc_lo, 16
+; SI-NEXT: s_or_b32 s52, s6, s10
+; SI-NEXT: s_and_b32 s6, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s93, 16
+; SI-NEXT: s_or_b32 s53, s6, s10
+; SI-NEXT: s_and_b32 s6, s91, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s90, 16
+; SI-NEXT: s_or_b32 s54, s6, s10
+; SI-NEXT: s_and_b32 s6, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s79, 16
+; SI-NEXT: s_or_b32 s55, s6, s10
+; SI-NEXT: s_and_b32 s6, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s76, 16
+; SI-NEXT: s_or_b32 s56, s6, s10
+; SI-NEXT: s_and_b32 s6, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s74, 16
+; SI-NEXT: s_or_b32 s57, s6, s10
+; SI-NEXT: s_and_b32 s6, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s58, 16
+; SI-NEXT: s_or_b32 s58, s6, s10
+; SI-NEXT: s_and_b32 s6, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s59, 16
+; SI-NEXT: s_or_b32 s59, s6, s10
+; SI-NEXT: s_and_b32 s6, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s60, 16
+; SI-NEXT: s_or_b32 s60, s6, s10
+; SI-NEXT: s_and_b32 s6, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s10, s61, 16
+; SI-NEXT: s_or_b32 s61, s6, s10
+; SI-NEXT: s_and_b32 s6, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s6, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s41, s5, s4
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
-; SI-NEXT: s_or_b32 s42, s4, s5
-; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
-; SI-NEXT: s_or_b32 s43, s4, s5
-; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s44, s4, s5
-; SI-NEXT: s_and_b32 s4, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s45, s4, s5
-; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s46, s4, s5
-; SI-NEXT: s_and_b32 s4, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
-; SI-NEXT: s_or_b32 s47, s4, s5
-; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s48, s4, s5
-; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s49, s4, s5
-; SI-NEXT: s_and_b32 s4, s81, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s34, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s91, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s8, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s10, s64
+; SI-NEXT: s_mov_b32 s6, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s95, s93
+; SI-NEXT: s_mov_b32 s93, s92
+; SI-NEXT: s_mov_b32 s92, s90
+; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s89, s79
+; SI-NEXT: s_mov_b32 s79, s78
+; SI-NEXT: s_mov_b32 s78, s76
+; SI-NEXT: s_mov_b32 s76, s74
+; SI-NEXT: s_mov_b32 s74, s66
+; SI-NEXT: s_mov_b32 s12, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s12
+; SI-NEXT: s_mov_b32 s66, s74
+; SI-NEXT: s_mov_b32 s74, s76
+; SI-NEXT: s_mov_b32 s76, s78
+; SI-NEXT: s_mov_b32 s78, s79
+; SI-NEXT: s_mov_b32 s79, s89
+; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_mov_b32 s90, s92
+; SI-NEXT: s_mov_b32 s92, s93
+; SI-NEXT: s_mov_b32 s93, s95
+; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 vcc_lo, s6
+; SI-NEXT: s_mov_b32 s64, s10
; SI-NEXT: .LBB51_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -37299,42 +37624,46 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: s_add_i32 s18, s18, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 9
+; SI-NEXT: s_and_b32 s5, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s38, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: s_add_i32 s19, s19, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 8
+; SI-NEXT: s_and_b32 s5, s19, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s39, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: s_add_i32 s20, s20, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 7
+; SI-NEXT: s_and_b32 s5, s20, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s21, s21, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 6
+; SI-NEXT: s_and_b32 s5, s21, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s72, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
@@ -37367,9 +37696,9 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s50, s4, 0x30000
; SI-NEXT: s_add_i32 s4, s34, 3
@@ -37406,39 +37735,45 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, s75, 0xffff
; SI-NEXT: s_lshl_b32 s5, s74, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v28, 28
+; SI-NEXT: v_readlane_b32 s30, v28, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -37467,17 +37802,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v25, s61
; SI-NEXT: v_mov_b32_e32 v26, s62
; SI-NEXT: v_mov_b32_e32 v27, s63
-; SI-NEXT: v_readlane_b32 s31, v28, 29
-; SI-NEXT: v_readlane_b32 s85, v28, 27
-; SI-NEXT: v_readlane_b32 s84, v28, 26
-; SI-NEXT: v_readlane_b32 s83, v28, 25
-; SI-NEXT: v_readlane_b32 s82, v28, 24
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
+; SI-NEXT: v_readlane_b32 s31, v28, 19
; SI-NEXT: v_readlane_b32 s67, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
@@ -37498,6 +37823,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v28, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v28, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v29, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -37527,170 +37853,184 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v28, s65, 15
; VI-NEXT: v_writelane_b32 v28, s66, 16
; VI-NEXT: v_writelane_b32 v28, s67, 17
-; VI-NEXT: v_writelane_b32 v28, s68, 18
-; VI-NEXT: v_writelane_b32 v28, s69, 19
-; VI-NEXT: v_writelane_b32 v28, s70, 20
-; VI-NEXT: v_writelane_b32 v28, s71, 21
-; VI-NEXT: v_writelane_b32 v28, s80, 22
-; VI-NEXT: v_writelane_b32 v28, s81, 23
-; VI-NEXT: v_writelane_b32 v28, s82, 24
-; VI-NEXT: v_writelane_b32 v28, s83, 25
-; VI-NEXT: v_writelane_b32 v28, s84, 26
-; VI-NEXT: v_writelane_b32 v28, s85, 27
-; VI-NEXT: v_writelane_b32 v28, s86, 28
-; VI-NEXT: v_writelane_b32 v28, s87, 29
-; VI-NEXT: v_writelane_b32 v28, s30, 30
-; VI-NEXT: v_writelane_b32 v28, s31, 31
+; VI-NEXT: v_writelane_b32 v28, s30, 18
+; VI-NEXT: v_writelane_b32 v28, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v13
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v12
+; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v11
+; VI-NEXT: v_writelane_b32 v29, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v10
+; VI-NEXT: v_writelane_b32 v29, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v9
+; VI-NEXT: v_writelane_b32 v29, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v8
+; VI-NEXT: v_writelane_b32 v29, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v7
+; VI-NEXT: v_writelane_b32 v29, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v6
+; VI-NEXT: v_writelane_b32 v29, s58, 5
+; VI-NEXT: s_lshr_b32 s56, s77, 16
; VI-NEXT: v_readfirstlane_b32 s88, v5
+; VI-NEXT: v_writelane_b32 v29, s57, 6
+; VI-NEXT: s_lshr_b32 s62, s88, 16
; VI-NEXT: v_readfirstlane_b32 s91, v4
+; VI-NEXT: v_writelane_b32 v29, s56, 7
+; VI-NEXT: s_lshr_b32 s63, s91, 16
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v29, s62, 8
+; VI-NEXT: s_lshr_b32 s51, s50, 16
+; VI-NEXT: v_writelane_b32 v29, s63, 9
; VI-NEXT: v_readfirstlane_b32 s30, v3
; VI-NEXT: v_readfirstlane_b32 s35, v2
-; VI-NEXT: v_readfirstlane_b32 s87, v1
-; VI-NEXT: v_readfirstlane_b32 s69, v0
-; VI-NEXT: s_lshr_b32 s78, s29, 16
-; VI-NEXT: s_lshr_b32 s89, s28, 16
+; VI-NEXT: v_readfirstlane_b32 s66, v1
+; VI-NEXT: v_writelane_b32 v29, s51, 10
+; VI-NEXT: s_lshr_b32 s6, s29, 16
+; VI-NEXT: s_lshr_b32 s10, s28, 16
; VI-NEXT: s_lshr_b32 vcc_lo, s27, 16
; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s85, s25, 16
-; VI-NEXT: s_lshr_b32 s67, s24, 16
-; VI-NEXT: s_lshr_b32 s4, s23, 16
-; VI-NEXT: s_lshr_b32 s70, s22, 16
-; VI-NEXT: s_lshr_b32 s71, s21, 16
-; VI-NEXT: s_lshr_b32 s80, s20, 16
-; VI-NEXT: s_lshr_b32 s81, s19, 16
-; VI-NEXT: s_lshr_b32 s82, s18, 16
-; VI-NEXT: s_lshr_b32 s83, s17, 16
-; VI-NEXT: s_lshr_b32 s84, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s7, 16
-; VI-NEXT: s_lshr_b32 s10, s9, 16
-; VI-NEXT: s_lshr_b32 s12, s11, 16
-; VI-NEXT: s_lshr_b32 s60, s13, 16
-; VI-NEXT: s_lshr_b32 s14, s15, 16
-; VI-NEXT: s_lshr_b32 s58, s73, 16
-; VI-NEXT: s_lshr_b32 s74, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s79, s88, 16
-; VI-NEXT: s_lshr_b32 s90, s91, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s30, 16
+; VI-NEXT: s_lshr_b32 s64, s25, 16
+; VI-NEXT: s_lshr_b32 vcc_hi, s24, 16
+; VI-NEXT: s_lshr_b32 s43, s23, 16
+; VI-NEXT: s_lshr_b32 s12, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s8, s20, 16
+; VI-NEXT: s_lshr_b32 s40, s19, 16
+; VI-NEXT: s_lshr_b32 s14, s18, 16
+; VI-NEXT: s_lshr_b32 s42, s17, 16
+; VI-NEXT: s_lshr_b32 s44, s16, 16
+; VI-NEXT: s_lshr_b32 s79, s30, 16
; VI-NEXT: s_lshr_b32 s34, s35, 16
-; VI-NEXT: s_lshr_b32 s86, s87, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v29, s12, 0
-; VI-NEXT: v_writelane_b32 v29, s14, 1
+; VI-NEXT: s_lshr_b32 s65, s66, 16
+; VI-NEXT: v_readfirstlane_b32 s45, v14
+; VI-NEXT: v_writelane_b32 v29, s50, 11
+; VI-NEXT: s_cmp_lg_u32 s45, 0
+; VI-NEXT: v_writelane_b32 v29, s6, 12
+; VI-NEXT: v_writelane_b32 v29, s10, 13
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s6, s84, 16
-; VI-NEXT: s_or_b32 s36, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s6, s82, 16
-; VI-NEXT: s_or_b32 s38, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s6, s81, 16
-; VI-NEXT: s_and_b32 s8, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s83, 16
-; VI-NEXT: s_or_b32 s39, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s6, s80, 16
-; VI-NEXT: s_or_b32 s37, s8, s40
-; VI-NEXT: s_or_b32 s40, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s6, s71, 16
-; VI-NEXT: s_or_b32 s41, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s6, s70, 16
-; VI-NEXT: s_or_b32 s42, s5, s6
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_mov_b32 s6, s4
+; VI-NEXT: s_and_b32 s46, 0xffff, s17
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s46, s42
+; VI-NEXT: s_and_b32 s42, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s38, s42, s14
+; VI-NEXT: s_and_b32 s42, 0xffff, s19
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s39, s42, s40
+; VI-NEXT: s_and_b32 s40, 0xffff, s20
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s40, s40, s8
+; VI-NEXT: s_and_b32 s42, 0xffff, s21
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s45, 0xffff, s16
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s42, s41
+; VI-NEXT: s_and_b32 s42, 0xffff, s22
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_lshl_b32 s12, s12, 16
+; VI-NEXT: s_or_b32 s36, s45, s44
+; VI-NEXT: s_or_b32 s42, s42, s12
+; VI-NEXT: s_and_b32 s44, 0xffff, s23
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s43, s44, s43
+; VI-NEXT: s_and_b32 s44, 0xffff, s24
+; VI-NEXT: s_lshl_b32 s45, vcc_hi, 16
+; VI-NEXT: s_or_b32 s44, s44, s45
+; VI-NEXT: s_and_b32 s45, 0xffff, s25
+; VI-NEXT: s_lshl_b32 s46, s64, 16
+; VI-NEXT: s_or_b32 s45, s45, s46
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s31, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s90, vcc_lo, 16
+; VI-NEXT: s_or_b32 s47, s47, s90
+; VI-NEXT: s_and_b32 s90, 0xffff, s28
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s48, s90, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s49, s10, s6
+; VI-NEXT: s_and_b32 s6, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s10, s51, 16
+; VI-NEXT: s_or_b32 s50, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s66
+; VI-NEXT: s_lshl_b32 s10, s65, 16
+; VI-NEXT: s_or_b32 s51, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s10, s34, 16
+; VI-NEXT: s_or_b32 s52, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s30
+; VI-NEXT: s_lshl_b32 s10, s79, 16
+; VI-NEXT: s_or_b32 s53, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s10, s63, 16
+; VI-NEXT: s_or_b32 s54, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s88
+; VI-NEXT: s_lshl_b32 s10, s62, 16
+; VI-NEXT: s_or_b32 s55, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s10, s56, 16
+; VI-NEXT: s_or_b32 s56, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s10, s57, 16
+; VI-NEXT: s_or_b32 s57, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s10, s58, 16
+; VI-NEXT: s_or_b32 s58, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s10, s59, 16
+; VI-NEXT: s_or_b32 s59, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s10, s60, 16
+; VI-NEXT: s_or_b32 s60, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s10, s61, 16
+; VI-NEXT: s_or_b32 s61, s6, s10
+; VI-NEXT: s_and_b32 s6, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s6, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s43, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s5, s67, 16
-; VI-NEXT: s_or_b32 s44, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s5, s85, 16
-; VI-NEXT: s_or_b32 s45, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s87
-; VI-NEXT: s_lshl_b32 s5, s86, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s35
-; VI-NEXT: s_lshl_b32 s5, s34, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s88
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s72, s58
-; VI-NEXT: s_mov_b32 s14, s60
-; VI-NEXT: s_mov_b32 s12, s10
-; VI-NEXT: s_mov_b32 s10, s63
-; VI-NEXT: s_mov_b32 s8, s67
+; VI-NEXT: s_mov_b32 s76, s44
+; VI-NEXT: s_mov_b32 s89, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s40
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s41
+; VI-NEXT: s_mov_b32 s72, s12
+; VI-NEXT: s_mov_b32 s12, s43
+; VI-NEXT: s_mov_b32 s6, s64
+; VI-NEXT: s_mov_b32 s10, s65
+; VI-NEXT: s_mov_b32 s90, s79
+; VI-NEXT: s_mov_b32 s79, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s66
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s6, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s8
+; VI-NEXT: s_mov_b32 s66, vcc_lo
+; VI-NEXT: s_mov_b32 vcc_lo, s79
+; VI-NEXT: s_mov_b32 s79, s90
+; VI-NEXT: s_mov_b32 s65, s10
+; VI-NEXT: s_mov_b32 s64, s6
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -37699,52 +38039,52 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s83, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s6, s89, 16
+; VI-NEXT: s_or_b32 s5, s6, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s80, 16
+; VI-NEXT: s_lshl_b32 s5, s74, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s67, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s64, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -37756,25 +38096,29 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s27, 0xffff
; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: s_add_i32 s28, s28, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 13
+; VI-NEXT: s_and_b32 s5, s28, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 12
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v29, 11
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v29, 10
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_add_i32 s4, s87, 3
+; VI-NEXT: s_add_i32 s4, s66, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s65, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s51, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s35, 3
@@ -37784,63 +38128,71 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s30, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_hi, 16
+; VI-NEXT: s_lshl_b32 s5, s79, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s91, s91, 3
; VI-NEXT: s_add_i32 s53, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s91, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: s_add_i32 s91, s91, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 9
+; VI-NEXT: s_and_b32 s5, s91, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s88, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s88, s88, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 8
+; VI-NEXT: s_and_b32 s5, s88, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 1
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s14, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v29, 0
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v29, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v28, 30
+; VI-NEXT: v_readlane_b32 s30, v28, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -37869,19 +38221,7 @@ define inreg <14 x double> @bitcast_v56i16_to_v14f64_scalar(<56 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v25, s61
; VI-NEXT: v_mov_b32_e32 v26, s62
; VI-NEXT: v_mov_b32_e32 v27, s63
-; VI-NEXT: v_readlane_b32 s31, v28, 31
-; VI-NEXT: v_readlane_b32 s87, v28, 29
-; VI-NEXT: v_readlane_b32 s86, v28, 28
-; VI-NEXT: v_readlane_b32 s85, v28, 27
-; VI-NEXT: v_readlane_b32 s84, v28, 26
-; VI-NEXT: v_readlane_b32 s83, v28, 25
-; VI-NEXT: v_readlane_b32 s82, v28, 24
-; VI-NEXT: v_readlane_b32 s81, v28, 23
-; VI-NEXT: v_readlane_b32 s80, v28, 22
-; VI-NEXT: v_readlane_b32 s71, v28, 21
-; VI-NEXT: v_readlane_b32 s70, v28, 20
-; VI-NEXT: v_readlane_b32 s69, v28, 19
-; VI-NEXT: v_readlane_b32 s68, v28, 18
+; VI-NEXT: v_readlane_b32 s31, v28, 19
; VI-NEXT: v_readlane_b32 s67, v28, 17
; VI-NEXT: v_readlane_b32 s66, v28, 16
; VI-NEXT: v_readlane_b32 s65, v28, 15
@@ -41678,8 +42018,9 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -41698,153 +42039,181 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s30, 28
-; SI-NEXT: v_writelane_b32 v32, s31, 29
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
; SI-NEXT: v_readfirstlane_b32 s6, v13
+; SI-NEXT: s_lshr_b32 s4, s6, 16
; SI-NEXT: v_readfirstlane_b32 s8, v12
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v11
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s4, 0
+; SI-NEXT: s_lshr_b32 s61, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v10
+; SI-NEXT: v_writelane_b32 v33, s5, 1
+; SI-NEXT: s_lshr_b32 s60, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 2
+; SI-NEXT: s_lshr_b32 s59, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 3
+; SI-NEXT: s_lshr_b32 s58, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 4
+; SI-NEXT: s_lshr_b32 s57, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v6
+; SI-NEXT: v_writelane_b32 v33, s58, 5
+; SI-NEXT: s_lshr_b32 s56, s76, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 6
; SI-NEXT: v_readfirstlane_b32 s78, v5
+; SI-NEXT: v_readfirstlane_b32 s34, v0
+; SI-NEXT: v_writelane_b32 v33, s56, 7
+; SI-NEXT: s_lshr_b32 s62, s78, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 8
; SI-NEXT: v_readfirstlane_b32 s88, v4
+; SI-NEXT: s_lshr_b32 s35, s34, 16
+; SI-NEXT: v_writelane_b32 v33, s62, 9
+; SI-NEXT: s_lshr_b32 s63, s88, 16
; SI-NEXT: v_readfirstlane_b32 s90, v3
; SI-NEXT: v_readfirstlane_b32 s92, v2
; SI-NEXT: v_readfirstlane_b32 s95, v1
-; SI-NEXT: v_readfirstlane_b32 s4, v0
+; SI-NEXT: v_writelane_b32 v33, s35, 10
; SI-NEXT: s_lshr_b32 s94, s29, 16
; SI-NEXT: s_lshr_b32 vcc_lo, s28, 16
-; SI-NEXT: s_lshr_b32 s30, s27, 16
-; SI-NEXT: s_lshr_b32 s34, s26, 16
-; SI-NEXT: s_lshr_b32 s35, s25, 16
-; SI-NEXT: s_lshr_b32 s68, s24, 16
-; SI-NEXT: s_lshr_b32 s70, s23, 16
-; SI-NEXT: s_lshr_b32 s71, s22, 16
-; SI-NEXT: s_lshr_b32 s80, s21, 16
-; SI-NEXT: s_lshr_b32 s81, s20, 16
-; SI-NEXT: s_lshr_b32 s82, s19, 16
-; SI-NEXT: s_lshr_b32 s83, s18, 16
-; SI-NEXT: s_lshr_b32 s84, s17, 16
-; SI-NEXT: s_lshr_b32 s85, s16, 16
-; SI-NEXT: s_lshr_b32 s69, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s11, s26, 16
+; SI-NEXT: s_lshr_b32 s13, s25, 16
+; SI-NEXT: s_lshr_b32 s15, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s42, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s7, s20, 16
+; SI-NEXT: s_lshr_b32 s40, s19, 16
+; SI-NEXT: s_lshr_b32 s9, s18, 16
+; SI-NEXT: s_lshr_b32 s44, s17, 16
+; SI-NEXT: s_lshr_b32 s45, s16, 16
+; SI-NEXT: s_lshr_b32 s30, s90, 16
+; SI-NEXT: s_lshr_b32 s31, s92, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s95, 16
-; SI-NEXT: s_lshr_b32 s31, s4, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v14
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s46, v14
+; SI-NEXT: v_writelane_b32 v33, s63, 11
+; SI-NEXT: s_cmp_lg_u32 s46, 0
+; SI-NEXT: v_writelane_b32 v33, s30, 12
+; SI-NEXT: v_writelane_b32 v33, s31, 13
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s37, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s80, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s35, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s30, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_lo, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s94, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s31, 16
-; SI-NEXT: s_or_b32 s50, s4, s5
-; SI-NEXT: s_and_b32 s4, s95, 0xffff
-; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
-; SI-NEXT: s_or_b32 s51, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
+; SI-NEXT: s_and_b32 s46, s16, 0xffff
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s36, s46, s45
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s37, s45, s44
+; SI-NEXT: s_and_b32 s44, s18, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s38, s44, s9
+; SI-NEXT: s_and_b32 s44, s19, 0xffff
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s39, s44, s40
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s40, s40, s7
+; SI-NEXT: s_and_b32 s44, s21, 0xffff
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s44, s41
+; SI-NEXT: s_and_b32 s44, s22, 0xffff
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s42, s44, s42
+; SI-NEXT: s_and_b32 s44, s23, 0xffff
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s44, s43
+; SI-NEXT: s_and_b32 s44, s24, 0xffff
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s44, s44, s15
+; SI-NEXT: s_and_b32 s45, s25, 0xffff
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s45, s45, s13
+; SI-NEXT: s_and_b32 s46, s26, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s46, s46, s11
+; SI-NEXT: s_and_b32 s48, s27, 0xffff
+; SI-NEXT: s_mov_b32 s11, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s48, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_lo, 16
+; SI-NEXT: s_or_b32 s48, s48, s49
+; SI-NEXT: s_and_b32 s49, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s94, 16
+; SI-NEXT: s_and_b32 s34, s34, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s49, s49, s50
+; SI-NEXT: s_or_b32 s50, s34, s35
+; SI-NEXT: s_and_b32 s34, s95, 0xffff
+; SI-NEXT: s_lshl_b32 s35, vcc_hi, 16
+; SI-NEXT: s_or_b32 s51, s34, s35
+; SI-NEXT: s_and_b32 s34, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s52, s34, s31
+; SI-NEXT: s_and_b32 s31, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_or_b32 s53, s31, s30
+; SI-NEXT: s_and_b32 s30, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s54, s30, s63
+; SI-NEXT: s_and_b32 s63, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s55, s63, s62
+; SI-NEXT: s_and_b32 s62, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s56, s62, s56
+; SI-NEXT: s_and_b32 s62, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s62, s57
+; SI-NEXT: s_and_b32 s62, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s62, s58
+; SI-NEXT: s_and_b32 s62, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s62, s59
+; SI-NEXT: s_and_b32 s62, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s62, s60
+; SI-NEXT: s_and_b32 s62, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s62, s61
+; SI-NEXT: s_and_b32 s62, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s62, s62, s5
+; SI-NEXT: s_and_b32 s5, s6, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s63, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s93, s45
+; SI-NEXT: s_mov_b32 s91, s44
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s40
+; SI-NEXT: s_mov_b32 s79, s7
+; SI-NEXT: s_mov_b32 s7, s41
+; SI-NEXT: s_mov_b32 s77, s42
+; SI-NEXT: s_mov_b32 s75, s43
+; SI-NEXT: s_mov_b32 s73, s15
+; SI-NEXT: s_mov_b32 s15, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s47
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -41853,9 +42222,9 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s93
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s91
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_cvt_f16_f32_e32 v0, v0
@@ -41865,14 +42234,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s89
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
; SI-NEXT: v_lshlrev_b32_e32 v1, 16, v2
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s9
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
@@ -41882,7 +42251,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s79
; SI-NEXT: v_cvt_f32_f16_e32 v6, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
@@ -41891,11 +42260,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
; SI-NEXT: v_cvt_f16_f32_e32 v6, v6
; SI-NEXT: v_or_b32_e32 v3, v5, v3
-; SI-NEXT: v_cvt_f32_f16_e32 v5, s80
+; SI-NEXT: v_cvt_f32_f16_e32 v5, s7
; SI-NEXT: v_lshlrev_b32_e32 v4, 16, v4
; SI-NEXT: v_or_b32_e32 v4, v6, v4
; SI-NEXT: v_cvt_f32_f16_e32 v6, s21
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s77
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v6, 0x38000000, v6
@@ -41906,7 +42275,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_or_b32_e32 v5, v6, v5
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v7
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s75
; SI-NEXT: v_cvt_f32_f16_e32 v9, s23
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
@@ -41915,11 +42284,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
; SI-NEXT: v_cvt_f16_f32_e32 v9, v9
; SI-NEXT: v_or_b32_e32 v6, v8, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v8, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v8, s73
; SI-NEXT: v_lshlrev_b32_e32 v7, 16, v7
; SI-NEXT: v_or_b32_e32 v7, v9, v7
; SI-NEXT: v_cvt_f32_f16_e32 v9, s24
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s15
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_add_f32_e32 v9, 0x38000000, v9
@@ -41930,7 +42299,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_or_b32_e32 v8, v9, v8
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v10
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s34
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s13
; SI-NEXT: v_cvt_f32_f16_e32 v12, s26
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
@@ -41939,7 +42308,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v12, 0x38000000, v12
; SI-NEXT: v_cvt_f16_f32_e32 v12, v12
; SI-NEXT: v_or_b32_e32 v9, v11, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v11, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v11, s11
; SI-NEXT: v_lshlrev_b32_e32 v10, 16, v10
; SI-NEXT: v_or_b32_e32 v10, v12, v10
; SI-NEXT: v_cvt_f32_f16_e32 v12, s27
@@ -41962,11 +42331,13 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 10
; SI-NEXT: v_or_b32_e32 v12, v14, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v14, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v14, s4
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_or_b32_e32 v13, v15, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_cvt_f32_f16_e32 v16, vcc_hi
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
@@ -41975,10 +42346,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v14
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_cvt_f32_f16_e32 v17, s95
; SI-NEXT: v_or_b32_e32 v14, v15, v14
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v16
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_cvt_f32_f16_e32 v18, s92
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
@@ -41986,12 +42358,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v15, v17, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v16, v18, v16
; SI-NEXT: v_cvt_f32_f16_e32 v18, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_add_f32_e32 v18, 0x38000000, v18
@@ -41999,10 +42373,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_cvt_f32_f16_e32 v20, s88
; SI-NEXT: v_or_b32_e32 v17, v18, v17
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v19
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_cvt_f32_f16_e32 v21, s78
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
@@ -42010,12 +42385,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v18, v20, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v20, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v20, s4
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v19, v21, v19
; SI-NEXT: v_cvt_f32_f16_e32 v21, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_add_f32_e32 v21, 0x38000000, v21
@@ -42023,10 +42400,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v23, s74
; SI-NEXT: v_or_b32_e32 v20, v21, v20
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v22
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_cvt_f32_f16_e32 v24, s72
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
@@ -42034,12 +42412,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v21, v23, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v23, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v23, s4
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v22, v24, v22
; SI-NEXT: v_cvt_f32_f16_e32 v24, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_add_f32_e32 v24, 0x38000000, v24
@@ -42047,10 +42427,11 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v26, s12
; SI-NEXT: v_or_b32_e32 v23, v24, v23
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v25
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_cvt_f32_f16_e32 v27, s10
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -42058,12 +42439,14 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v27, 0x38000000, v27
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v24, v26, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v26, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v26, s4
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v25, v27, v25
; SI-NEXT: v_cvt_f32_f16_e32 v27, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s6
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
@@ -42112,18 +42495,8 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 28
-; SI-NEXT: v_readlane_b32 s31, v32, 29
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -42144,6 +42517,7 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -42173,174 +42547,188 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
; VI-NEXT: v_readfirstlane_b32 s6, v13
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v12
+; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
; VI-NEXT: v_readfirstlane_b32 s10, v11
+; VI-NEXT: v_writelane_b32 v33, s4, 0
+; VI-NEXT: s_lshr_b32 s61, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v10
+; VI-NEXT: v_writelane_b32 v33, s5, 1
+; VI-NEXT: s_lshr_b32 s60, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 2
+; VI-NEXT: s_lshr_b32 s59, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 3
+; VI-NEXT: s_lshr_b32 s58, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 4
+; VI-NEXT: s_lshr_b32 s57, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 5
+; VI-NEXT: s_lshr_b32 s62, s76, 16
; VI-NEXT: v_readfirstlane_b32 s79, v5
+; VI-NEXT: v_writelane_b32 v33, s57, 6
+; VI-NEXT: s_lshr_b32 s63, s79, 16
+; VI-NEXT: v_writelane_b32 v33, s62, 7
+; VI-NEXT: v_readfirstlane_b32 s31, v3
+; VI-NEXT: v_writelane_b32 v33, s63, 8
+; VI-NEXT: v_readfirstlane_b32 s35, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
; VI-NEXT: v_readfirstlane_b32 s91, v4
-; VI-NEXT: v_readfirstlane_b32 s4, v3
-; VI-NEXT: v_readfirstlane_b32 s34, v2
-; VI-NEXT: v_readfirstlane_b32 s69, v1
-; VI-NEXT: v_readfirstlane_b32 s80, v0
-; VI-NEXT: s_lshr_b32 s56, s29, 16
-; VI-NEXT: s_lshr_b32 s61, s28, 16
-; VI-NEXT: s_lshr_b32 s90, s27, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s26, 16
-; VI-NEXT: s_lshr_b32 s31, s25, 16
-; VI-NEXT: s_lshr_b32 s35, s24, 16
-; VI-NEXT: s_lshr_b32 s70, s23, 16
-; VI-NEXT: s_lshr_b32 s81, s22, 16
-; VI-NEXT: s_lshr_b32 s83, s21, 16
-; VI-NEXT: s_lshr_b32 s84, s20, 16
-; VI-NEXT: s_lshr_b32 s85, s19, 16
-; VI-NEXT: s_lshr_b32 s86, s18, 16
-; VI-NEXT: s_lshr_b32 s87, s17, 16
-; VI-NEXT: s_lshr_b32 s7, s16, 16
-; VI-NEXT: s_lshr_b32 s63, s6, 16
-; VI-NEXT: s_lshr_b32 s11, s8, 16
-; VI-NEXT: s_lshr_b32 s13, s10, 16
-; VI-NEXT: s_lshr_b32 s15, s12, 16
-; VI-NEXT: s_lshr_b32 s59, s14, 16
-; VI-NEXT: s_lshr_b32 s58, s72, 16
-; VI-NEXT: s_lshr_b32 s57, s74, 16
-; VI-NEXT: s_lshr_b32 s60, s76, 16
-; VI-NEXT: s_lshr_b32 s89, s79, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s91, 16
-; VI-NEXT: s_lshr_b32 s30, s4, 16
-; VI-NEXT: s_lshr_b32 s68, s34, 16
-; VI-NEXT: s_lshr_b32 s71, s69, 16
-; VI-NEXT: s_lshr_b32 s82, s80, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v14
-; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s15, 0
-; VI-NEXT: v_writelane_b32 v33, s59, 1
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s35, 10
+; VI-NEXT: s_lshr_b32 s30, s91, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s30, 12
+; VI-NEXT: s_lshr_b32 s34, s31, 16
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s52, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s34, 14
+; VI-NEXT: s_lshr_b32 s9, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
+; VI-NEXT: s_lshr_b32 s47, s27, 16
+; VI-NEXT: s_lshr_b32 s11, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s7, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s13, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s15, s19, 16
+; VI-NEXT: s_lshr_b32 s41, s18, 16
+; VI-NEXT: s_lshr_b32 s43, s17, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
+; VI-NEXT: s_lshr_b32 s53, s51, 16
+; VI-NEXT: s_lshr_b32 s54, s50, 16
+; VI-NEXT: v_readfirstlane_b32 s73, v14
+; VI-NEXT: v_writelane_b32 v33, s52, 15
+; VI-NEXT: s_cmp_lg_u32 s73, 0
+; VI-NEXT: v_writelane_b32 v33, s9, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 17
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s9, s7, 16
-; VI-NEXT: s_or_b32 s36, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s9, s86, 16
-; VI-NEXT: s_or_b32 s38, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s9, s85, 16
-; VI-NEXT: s_and_b32 s40, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s41, s87, 16
-; VI-NEXT: s_or_b32 s39, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s9, s84, 16
-; VI-NEXT: s_or_b32 s37, s40, s41
-; VI-NEXT: s_or_b32 s40, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s9, s83, 16
-; VI-NEXT: s_or_b32 s41, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s9, s81, 16
-; VI-NEXT: s_or_b32 s42, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s9, s70, 16
-; VI-NEXT: s_or_b32 s43, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s9, s35, 16
-; VI-NEXT: s_or_b32 s44, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s9, s31, 16
-; VI-NEXT: s_or_b32 s45, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s9, vcc_hi, 16
-; VI-NEXT: s_or_b32 s46, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s9, s90, 16
-; VI-NEXT: s_or_b32 s47, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s9, s61, 16
-; VI-NEXT: s_or_b32 s48, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s9, s56, 16
-; VI-NEXT: s_or_b32 s49, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s9, s82, 16
-; VI-NEXT: s_or_b32 s50, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s9, s71, 16
-; VI-NEXT: s_or_b32 s51, s5, s9
-; VI-NEXT: s_and_b32 s5, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s9, s68, 16
-; VI-NEXT: s_or_b32 s52, s5, s9
-; VI-NEXT: s_mov_b32 s9, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s4
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s91
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s89, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s60, 16
+; VI-NEXT: s_and_b32 s75, 0xffff, s17
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s37, s75, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s18
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s38, s43, s41
+; VI-NEXT: s_and_b32 s41, 0xffff, s19
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s39, s41, s15
+; VI-NEXT: s_and_b32 s41, 0xffff, s20
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s41, s40
+; VI-NEXT: s_and_b32 s41, 0xffff, s21
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s41, s41, s13
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s16
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s36, s73, s46
+; VI-NEXT: s_or_b32 s43, s43, s7
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s89, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_lshl_b32 s11, s11, 16
+; VI-NEXT: s_or_b32 s46, s46, s11
+; VI-NEXT: s_and_b32 s73, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s47, s73, s47
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s15, 16
-; VI-NEXT: s_mov_b32 s88, s60
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s13, 16
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s11, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_or_b32 s63, s4, s5
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s48, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s49, s56, s9
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s50, s54, 16
+; VI-NEXT: s_or_b32 s50, s56, s50
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s51, s53, 16
+; VI-NEXT: s_or_b32 s51, s56, s51
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s35, s52, 16
+; VI-NEXT: s_or_b32 s52, s56, s35
+; VI-NEXT: s_and_b32 s56, 0xffff, s31
+; VI-NEXT: s_lshl_b32 s31, s34, 16
+; VI-NEXT: s_or_b32 s53, s56, s31
+; VI-NEXT: s_and_b32 s56, 0xffff, s91
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_or_b32 s54, s56, s30
+; VI-NEXT: s_and_b32 s56, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s55, s56, s63
+; VI-NEXT: s_and_b32 s56, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s56, s56, s62
+; VI-NEXT: s_and_b32 s62, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s62, s57
+; VI-NEXT: s_and_b32 s62, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s62, s58
+; VI-NEXT: s_and_b32 s62, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s62, s59
+; VI-NEXT: s_and_b32 s62, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s62, s60
+; VI-NEXT: s_and_b32 s62, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s62, s61
+; VI-NEXT: s_and_b32 s62, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s62, s62, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s63, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s75, s61
-; VI-NEXT: s_mov_b32 s88, s60
+; VI-NEXT: s_mov_b32 s75, s11
+; VI-NEXT: s_mov_b32 s11, s47
; VI-NEXT: s_mov_b32 s73, s56
-; VI-NEXT: s_mov_b32 s78, s57
-; VI-NEXT: s_mov_b32 s77, s58
-; VI-NEXT: s_mov_b32 s15, s13
-; VI-NEXT: s_mov_b32 s13, s11
-; VI-NEXT: s_mov_b32 s11, s63
-; VI-NEXT: s_mov_b32 s9, s4
+; VI-NEXT: s_mov_b32 s88, s46
+; VI-NEXT: s_mov_b32 vcc_hi, s43
+; VI-NEXT: s_mov_b32 s78, s41
+; VI-NEXT: s_mov_b32 s77, s15
+; VI-NEXT: s_mov_b32 s15, s40
+; VI-NEXT: s_mov_b32 vcc_lo, s13
+; VI-NEXT: s_mov_b32 s9, s54
+; VI-NEXT: s_mov_b32 s13, s42
+; VI-NEXT: s_mov_b32 s90, s7
+; VI-NEXT: s_mov_b32 s7, s44
+; VI-NEXT: s_mov_b32 s89, s45
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -42350,117 +42738,133 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v27, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s7
-; VI-NEXT: v_mov_b32_e32 v2, s87
+; VI-NEXT: v_mov_b32_e32 v0, s88
+; VI-NEXT: v_mov_b32_e32 v2, vcc_hi
; VI-NEXT: v_add_f16_sdwa v0, v0, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v27
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v27
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s86
+; VI-NEXT: v_mov_b32_e32 v2, s78
; VI-NEXT: v_add_f16_sdwa v2, v2, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v27
; VI-NEXT: v_or_b32_e32 v2, v3, v2
-; VI-NEXT: v_mov_b32_e32 v3, s85
+; VI-NEXT: v_mov_b32_e32 v3, s77
; VI-NEXT: v_add_f16_sdwa v3, v3, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v27
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s84
+; VI-NEXT: v_mov_b32_e32 v4, s15
; VI-NEXT: v_add_f16_sdwa v4, v4, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v27
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s83
+; VI-NEXT: v_mov_b32_e32 v5, vcc_lo
; VI-NEXT: v_add_f16_sdwa v5, v5, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v27
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s81
+; VI-NEXT: v_mov_b32_e32 v6, s13
; VI-NEXT: v_add_f16_sdwa v6, v6, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v27
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s70
+; VI-NEXT: v_mov_b32_e32 v7, s90
; VI-NEXT: v_add_f16_sdwa v7, v7, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v27
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s35
+; VI-NEXT: v_mov_b32_e32 v8, s7
; VI-NEXT: v_add_f16_sdwa v8, v8, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v27
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s31
+; VI-NEXT: v_mov_b32_e32 v9, s89
; VI-NEXT: v_add_f16_sdwa v9, v9, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v27
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v10, s75
; VI-NEXT: v_add_f16_sdwa v10, v10, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v27
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, s90
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v27
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s75
+; VI-NEXT: v_mov_b32_e32 v12, s73
; VI-NEXT: v_add_f16_sdwa v12, v12, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s73
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v27
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s82
+; VI-NEXT: v_mov_b32_e32 v14, s9
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s80, v27
+; VI-NEXT: v_add_f16_e32 v15, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s71
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v15, v15, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s69, v27
+; VI-NEXT: v_add_f16_e32 v16, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s68
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s34, v27
+; VI-NEXT: v_add_f16_e32 v17, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s30
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_add_f16_sdwa v17, v17, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s9, v27
+; VI-NEXT: v_add_f16_e32 v18, s4, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, vcc_lo
+; VI-NEXT: v_mov_b32_e32 v18, s4
; VI-NEXT: v_add_f16_sdwa v18, v18, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v19, s91, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s89
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, s79, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s88
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s76, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s78
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s74, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s77
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s72, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s14, v27
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s12, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v24, v25, v24
-; VI-NEXT: v_mov_b32_e32 v25, s15
+; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s10, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v25, v26, v25
-; VI-NEXT: v_mov_b32_e32 v26, s13
+; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s8, v27
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v26, v28, v26
-; VI-NEXT: v_mov_b32_e32 v28, s11
+; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v27 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s6, v27
; VI-NEXT: v_or_b32_e32 v27, v27, v28
@@ -42499,20 +42903,8 @@ define inreg <14 x double> @bitcast_v56f16_to_v14f64_scalar(<56 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -44300,35 +44692,36 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v28, s64, 14
; SI-NEXT: v_writelane_b32 v28, s65, 15
; SI-NEXT: v_writelane_b32 v28, s66, 16
-; SI-NEXT: v_writelane_b32 v28, s67, 17
-; SI-NEXT: v_writelane_b32 v28, s68, 18
-; SI-NEXT: v_writelane_b32 v28, s69, 19
-; SI-NEXT: v_writelane_b32 v28, s70, 20
-; SI-NEXT: v_writelane_b32 v28, s71, 21
-; SI-NEXT: v_writelane_b32 v28, s80, 22
-; SI-NEXT: v_writelane_b32 v28, s81, 23
-; SI-NEXT: v_writelane_b32 v28, s30, 24
-; SI-NEXT: v_writelane_b32 v28, s31, 25
-; SI-NEXT: v_readfirstlane_b32 s53, v0
-; SI-NEXT: v_readfirstlane_b32 s55, v2
-; SI-NEXT: s_lshr_b32 s45, s53, 16
+; SI-NEXT: v_writelane_b32 v28, s68, 17
+; SI-NEXT: v_writelane_b32 v28, s69, 18
+; SI-NEXT: v_writelane_b32 v28, s70, 19
+; SI-NEXT: v_writelane_b32 v28, s71, 20
+; SI-NEXT: v_writelane_b32 v28, s80, 21
+; SI-NEXT: v_writelane_b32 v28, s81, 22
+; SI-NEXT: v_writelane_b32 v28, s82, 23
+; SI-NEXT: v_writelane_b32 v28, s83, 24
+; SI-NEXT: v_writelane_b32 v28, s30, 25
+; SI-NEXT: v_writelane_b32 v28, s31, 26
+; SI-NEXT: v_readfirstlane_b32 s94, v0
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: s_lshr_b32 s45, s94, 16
; SI-NEXT: ; implicit-def: $vgpr29 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s65, v4
-; SI-NEXT: s_lshr_b32 s44, s55, 16
+; SI-NEXT: v_readfirstlane_b32 s30, v4
+; SI-NEXT: s_lshr_b32 s44, vcc_lo, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v29, s45, 0
-; SI-NEXT: v_readfirstlane_b32 s66, v6
-; SI-NEXT: s_lshr_b32 s42, s65, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v6
+; SI-NEXT: s_lshr_b32 s42, s30, 16
; SI-NEXT: v_writelane_b32 v29, s44, 1
-; SI-NEXT: v_readfirstlane_b32 s68, v8
-; SI-NEXT: s_lshr_b32 s40, s66, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v8
+; SI-NEXT: s_lshr_b32 s40, s34, 16
; SI-NEXT: v_writelane_b32 v29, s42, 2
-; SI-NEXT: s_lshr_b32 s14, s68, 16
+; SI-NEXT: s_lshr_b32 s14, s36, 16
; SI-NEXT: v_writelane_b32 v29, s40, 3
-; SI-NEXT: v_readfirstlane_b32 s70, v10
+; SI-NEXT: v_readfirstlane_b32 s38, v10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v29, s14, 4
-; SI-NEXT: s_lshr_b32 s12, s70, 16
+; SI-NEXT: s_lshr_b32 s12, s38, 16
; SI-NEXT: v_writelane_b32 v29, s43, 5
; SI-NEXT: v_readfirstlane_b32 s41, v3
; SI-NEXT: v_writelane_b32 v29, s12, 6
@@ -44338,26 +44731,27 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v29, s15, 8
; SI-NEXT: v_readfirstlane_b32 s11, v9
; SI-NEXT: v_writelane_b32 v29, s13, 9
-; SI-NEXT: v_readfirstlane_b32 s7, v13
-; SI-NEXT: v_readfirstlane_b32 s80, v12
; SI-NEXT: v_readfirstlane_b32 s9, v11
; SI-NEXT: v_writelane_b32 v29, s11, 10
+; SI-NEXT: v_readfirstlane_b32 s7, v13
+; SI-NEXT: v_readfirstlane_b32 s50, v12
+; SI-NEXT: v_writelane_b32 v29, s9, 11
; SI-NEXT: s_lshr_b32 s93, s29, 16
-; SI-NEXT: s_lshr_b32 s81, s28, 16
+; SI-NEXT: s_lshr_b32 s66, s28, 16
; SI-NEXT: s_lshr_b32 s91, s27, 16
-; SI-NEXT: s_lshr_b32 s71, s26, 16
+; SI-NEXT: s_lshr_b32 s65, s26, 16
; SI-NEXT: s_lshr_b32 s89, s25, 16
-; SI-NEXT: s_lshr_b32 s69, s24, 16
+; SI-NEXT: s_lshr_b32 s64, s24, 16
; SI-NEXT: s_lshr_b32 s79, s23, 16
-; SI-NEXT: s_lshr_b32 s67, s22, 16
+; SI-NEXT: s_lshr_b32 s55, s22, 16
; SI-NEXT: s_lshr_b32 s77, s21, 16
-; SI-NEXT: s_lshr_b32 s64, s20, 16
+; SI-NEXT: s_lshr_b32 s54, s20, 16
; SI-NEXT: s_lshr_b32 s75, s19, 16
-; SI-NEXT: s_lshr_b32 s54, s18, 16
+; SI-NEXT: s_lshr_b32 s53, s18, 16
; SI-NEXT: s_lshr_b32 s73, s17, 16
; SI-NEXT: s_lshr_b32 s52, s16, 16
; SI-NEXT: s_lshr_b32 s49, s7, 16
-; SI-NEXT: s_lshr_b32 s5, s80, 16
+; SI-NEXT: s_lshr_b32 s5, s50, 16
; SI-NEXT: s_lshr_b32 s39, s9, 16
; SI-NEXT: s_lshr_b32 s37, s11, 16
; SI-NEXT: s_lshr_b32 s35, s13, 16
@@ -44365,10 +44759,10 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 vcc_hi, s41, 16
; SI-NEXT: s_lshr_b32 s95, s43, 16
; SI-NEXT: v_readfirstlane_b32 s4, v14
-; SI-NEXT: v_writelane_b32 v29, s9, 11
-; SI-NEXT: s_cmp_lg_u32 s4, 0
; SI-NEXT: v_writelane_b32 v29, s7, 12
-; SI-NEXT: v_writelane_b32 v29, s5, 13
+; SI-NEXT: s_cmp_lg_u32 s4, 0
+; SI-NEXT: v_writelane_b32 v29, s94, 13
+; SI-NEXT: v_writelane_b32 v29, s5, 14
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_lshl_b32 s46, s45, 16
@@ -44414,88 +44808,93 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s13, s37, 16
; SI-NEXT: s_or_b32 s10, s4, s72
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s74, s54, 16
+; SI-NEXT: s_lshl_b32 s74, s53, 16
; SI-NEXT: s_or_b32 s15, s11, s13
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s11, s39, 16
; SI-NEXT: s_or_b32 s8, s4, s74
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s76, s64, 16
+; SI-NEXT: s_lshl_b32 s76, s54, 16
; SI-NEXT: s_or_b32 s13, s9, s11
-; SI-NEXT: s_and_b32 s7, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s49, 16
-; SI-NEXT: s_or_b32 s6, s4, s76
-; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s62, s67, 16
-; SI-NEXT: s_mov_b32 s50, s49
-; SI-NEXT: s_or_b32 s49, s7, s9
; SI-NEXT: s_mov_b32 s11, s73
; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
-; SI-NEXT: s_mov_b32 s9, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[74:75], 16
+; SI-NEXT: s_or_b32 s6, s4, s76
+; SI-NEXT: s_and_b32 s4, s22, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s55, 16
+; SI-NEXT: v_writelane_b32 v29, s72, 15
; SI-NEXT: s_or_b32 s4, s4, s62
-; SI-NEXT: s_lshl_b32 s60, s69, 16
+; SI-NEXT: s_lshl_b32 s60, s64, 16
+; SI-NEXT: s_and_b32 s7, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s9, s49, 16
+; SI-NEXT: v_writelane_b32 v29, s73, 16
; SI-NEXT: s_mov_b32 s73, s78
+; SI-NEXT: s_mov_b32 s72, s79
+; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
+; SI-NEXT: s_and_b32 s62, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s65, 16
+; SI-NEXT: s_mov_b32 s51, s49
+; SI-NEXT: s_or_b32 s49, s7, s9
+; SI-NEXT: s_mov_b32 s9, s75
+; SI-NEXT: s_lshr_b64 s[80:81], s[74:75], 16
; SI-NEXT: s_mov_b32 s75, s88
; SI-NEXT: s_mov_b32 s7, s77
; SI-NEXT: s_lshr_b64 s[76:77], s[76:77], 16
-; SI-NEXT: s_mov_b32 s88, s79
-; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
-; SI-NEXT: s_and_b32 s62, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s71, 16
-; SI-NEXT: s_mov_b32 s77, s90
-; SI-NEXT: s_mov_b32 s79, s88
+; SI-NEXT: s_mov_b32 s79, s72
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s72, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[60:61], 16
; SI-NEXT: s_and_b32 s60, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s81, 16
-; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_lshl_b32 s56, s66, 16
+; SI-NEXT: s_mov_b32 s77, s90
+; SI-NEXT: s_mov_b32 s89, s72
; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s92, s91
+; SI-NEXT: s_mov_b32 s72, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s28, 0xffff
-; SI-NEXT: s_mov_b32 s91, s92
+; SI-NEXT: s_mov_b32 s91, s72
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s94, s93
+; SI-NEXT: s_mov_b32 s72, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[56:57], 16
-; SI-NEXT: s_and_b32 s56, s53, 0xffff
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
; SI-NEXT: s_lshl_b32 s44, s44, 16
-; SI-NEXT: s_mov_b32 s93, s94
+; SI-NEXT: s_mov_b32 s93, s72
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s72, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[46:47], 16
-; SI-NEXT: s_and_b32 s46, s55, 0xffff
+; SI-NEXT: s_and_b32 s46, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 s95, s72
+; SI-NEXT: s_mov_b32 s72, vcc_lo
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s74, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[44:45], 16
-; SI-NEXT: s_and_b32 s44, s65, 0xffff
+; SI-NEXT: s_and_b32 s44, s30, 0xffff
; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_mov_b32 vcc_hi, s30
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s34, s31
-; SI-NEXT: s_lshr_b64 s[30:31], s[42:43], 16
-; SI-NEXT: s_and_b32 s42, s66, 0xffff
+; SI-NEXT: s_lshr_b64 s[82:83], s[42:43], 16
+; SI-NEXT: s_and_b32 s42, s34, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_mov_b32 s31, s34
+; SI-NEXT: s_mov_b32 vcc_hi, s74
+; SI-NEXT: s_mov_b32 s74, s30
+; SI-NEXT: s_mov_b32 s69, s34
; SI-NEXT: s_or_b32 s42, s42, s40
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s30, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s68, 0xffff
+; SI-NEXT: s_and_b32 s40, s36, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_mov_b32 s35, s36
+; SI-NEXT: s_mov_b32 s35, s30
+; SI-NEXT: s_mov_b32 s68, s36
; SI-NEXT: s_or_b32 s40, s40, s14
-; SI-NEXT: s_mov_b32 s38, s37
+; SI-NEXT: s_mov_b32 s30, s37
; SI-NEXT: s_lshr_b64 s[36:37], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s70, 0xffff
+; SI-NEXT: s_and_b32 s14, s38, 0xffff
; SI-NEXT: s_lshl_b32 s48, s5, 16
-; SI-NEXT: s_mov_b32 s37, s38
+; SI-NEXT: s_mov_b32 s37, s30
+; SI-NEXT: s_mov_b32 s71, s38
; SI-NEXT: s_or_b32 s14, s14, s12
-; SI-NEXT: s_mov_b32 s51, s39
+; SI-NEXT: s_mov_b32 s30, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s80, 0xffff
+; SI-NEXT: s_and_b32 s12, s50, 0xffff
; SI-NEXT: s_mov_b32 s5, s63
; SI-NEXT: s_mov_b32 s63, s61
; SI-NEXT: s_mov_b32 s61, s59
@@ -44509,16 +44908,25 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s12, s12, s48
; SI-NEXT: s_mov_b32 s13, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[48:49], 16
-; SI-NEXT: s_mov_b32 s39, s51
-; SI-NEXT: s_mov_b32 s49, s50
+; SI-NEXT: s_mov_b32 s39, s30
+; SI-NEXT: s_mov_b32 s70, s50
+; SI-NEXT: s_mov_b32 s49, s51
; SI-NEXT: s_mov_b64 s[50:51], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr4
+; SI-NEXT: s_mov_b32 s70, s50
+; SI-NEXT: s_mov_b32 s71, s38
+; SI-NEXT: s_mov_b32 s68, s36
+; SI-NEXT: s_mov_b32 s69, s34
+; SI-NEXT: s_mov_b32 s74, s30
+; SI-NEXT: s_mov_b32 s72, vcc_lo
; SI-NEXT: s_mov_b64 s[50:51], -1
+; SI-NEXT: v_writelane_b32 v29, s4, 15
; SI-NEXT: ; implicit-def: $sgpr10
-; SI-NEXT: ; implicit-def: $sgpr72
+; SI-NEXT: v_writelane_b32 v29, s5, 16
; SI-NEXT: ; implicit-def: $sgpr8
-; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: ; implicit-def: $sgpr80
; SI-NEXT: ; implicit-def: $sgpr6
; SI-NEXT: ; implicit-def: $sgpr76
; SI-NEXT: ; implicit-def: $sgpr4
@@ -44534,7 +44942,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr46
; SI-NEXT: ; implicit-def: $vcc_lo
; SI-NEXT: ; implicit-def: $sgpr44
-; SI-NEXT: ; implicit-def: $sgpr30
+; SI-NEXT: ; implicit-def: $sgpr82
; SI-NEXT: ; implicit-def: $sgpr42
; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr40
@@ -44545,13 +44953,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr48
; SI-NEXT: .LBB57_3: ; %Flow
; SI-NEXT: s_and_b64 s[50:51], s[50:51], exec
-; SI-NEXT: s_cselect_b32 s50, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s50, 1
+; SI-NEXT: s_cselect_b32 s30, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s30, 1
+; SI-NEXT: v_readlane_b32 s50, v29, 15
+; SI-NEXT: v_readlane_b32 s51, v29, 16
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s80, s80, 3
-; SI-NEXT: v_readlane_b32 s4, v29, 13
-; SI-NEXT: s_and_b32 s5, s80, 0xffff
+; SI-NEXT: s_add_i32 s4, s70, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v29, 14
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s12, s4, 0x30000
@@ -44561,9 +44971,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s49, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s13, s4, 0x30000
-; SI-NEXT: s_add_i32 s70, s70, 3
+; SI-NEXT: s_add_i32 s4, s71, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 6
-; SI-NEXT: s_and_b32 s5, s70, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s14, s4, 0x30000
@@ -44573,9 +44983,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s39, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s15, s4, 0x30000
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s68, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 4
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
@@ -44585,9 +44995,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s41, s4, 0x30000
-; SI-NEXT: s_add_i32 s66, s66, 3
+; SI-NEXT: s_add_i32 s4, s69, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 3
-; SI-NEXT: s_and_b32 s5, s66, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -44597,9 +45007,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s65, s65, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 2
-; SI-NEXT: s_and_b32 s5, s65, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -44609,9 +45019,9 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s45, s4, 0x30000
-; SI-NEXT: s_add_i32 s55, s55, 3
+; SI-NEXT: s_add_i32 s4, s72, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 1
-; SI-NEXT: s_and_b32 s5, s55, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -44621,9 +45031,10 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, vcc_hi, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s47, s4, 0x30000
-; SI-NEXT: s_add_i32 s53, s53, 3
+; SI-NEXT: v_readlane_b32 s4, v29, 13
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v29, 0
-; SI-NEXT: s_and_b32 s5, s53, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -44635,7 +45046,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s66, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -44645,7 +45056,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s59, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s71, 16
+; SI-NEXT: s_lshl_b32 s5, s65, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -44655,7 +45066,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s61, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s62, s4, 0x30000
@@ -44665,7 +45076,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s67, 16
+; SI-NEXT: s_lshl_b32 s5, s55, 16
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s23, 0xffff
@@ -44673,7 +45084,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_or_b32 s5, s6, s5
; SI-NEXT: s_and_b32 s6, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s64, 16
+; SI-NEXT: s_lshl_b32 s7, s54, 16
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_or_b32 s6, s7, s6
; SI-NEXT: s_and_b32 s7, s21, 0xffff
@@ -44681,7 +45092,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_or_b32 s7, s8, s7
; SI-NEXT: s_and_b32 s8, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s9, s54, 16
+; SI-NEXT: s_lshl_b32 s9, s53, 16
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_or_b32 s8, s9, s8
; SI-NEXT: s_and_b32 s9, s19, 0xffff
@@ -44694,17 +45105,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s10, s11, s10
; SI-NEXT: s_and_b32 s11, s17, 0xffff
; SI-NEXT: s_lshl_b32 s16, s73, 16
-; SI-NEXT: s_or_b32 s11, s16, s11
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
; SI-NEXT: s_add_i32 s6, s6, 0x30000
; SI-NEXT: s_add_i32 s7, s7, 0x30000
+; SI-NEXT: s_or_b32 s11, s16, s11
; SI-NEXT: s_add_i32 s8, s8, 0x30000
; SI-NEXT: s_add_i32 s9, s9, 0x30000
; SI-NEXT: s_add_i32 s10, s10, 0x30000
; SI-NEXT: s_add_i32 s11, s11, 0x30000
-; SI-NEXT: s_lshr_b64 s[72:73], s[10:11], 16
-; SI-NEXT: s_lshr_b64 s[74:75], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[76:77], s[6:7], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[4:5], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[62:63], 16
@@ -44712,11 +45121,13 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 s[92:93], s[58:59], 16
; SI-NEXT: s_lshr_b64 s[94:95], s[56:57], 16
; SI-NEXT: s_lshr_b64 vcc, s[46:47], 16
-; SI-NEXT: s_lshr_b64 s[30:31], s[44:45], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[36:37], s[40:41], 16
; SI-NEXT: s_lshr_b64 s[38:39], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[12:13], 16
+; SI-NEXT: s_lshr_b64 s[50:51], s[10:11], 16
+; SI-NEXT: s_lshr_b64 s[80:81], s[8:9], 16
+; SI-NEXT: s_lshr_b64 s[82:83], s[44:45], 16
; SI-NEXT: s_lshr_b32 s73, s11, 16
; SI-NEXT: s_lshr_b32 s75, s9, 16
; SI-NEXT: s_lshr_b32 s77, s7, 16
@@ -44733,13 +45144,13 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s49, s13, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s10, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s72, 16
+; SI-NEXT: s_lshl_b32 s16, s50, 16
; SI-NEXT: s_or_b32 s10, s10, s16
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s16, s73, 16
; SI-NEXT: s_or_b32 s11, s11, s16
; SI-NEXT: s_and_b32 s8, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s74, 16
+; SI-NEXT: s_lshl_b32 s16, s80, 16
; SI-NEXT: s_or_b32 s8, s8, s16
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s16, s75, 16
@@ -44787,7 +45198,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s26, vcc_hi, 16
; SI-NEXT: s_or_b32 s25, s25, s26
; SI-NEXT: s_and_b32 s26, s44, 0xffff
-; SI-NEXT: s_lshl_b32 s27, s30, 16
+; SI-NEXT: s_lshl_b32 s27, s82, 16
; SI-NEXT: s_or_b32 s26, s26, s27
; SI-NEXT: s_and_b32 s27, s45, 0xffff
; SI-NEXT: s_lshl_b32 s28, s31, 16
@@ -44816,7 +45227,7 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s42, s49, 16
; SI-NEXT: s_or_b32 s13, s13, s42
-; SI-NEXT: v_readlane_b32 s30, v28, 24
+; SI-NEXT: v_readlane_b32 s30, v28, 25
; SI-NEXT: v_mov_b32_e32 v0, s10
; SI-NEXT: v_mov_b32_e32 v1, s11
; SI-NEXT: v_mov_b32_e32 v2, s8
@@ -44845,14 +45256,15 @@ define inreg <56 x half> @bitcast_v56i16_to_v56f16_scalar(<56 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v25, s15
; SI-NEXT: v_mov_b32_e32 v26, s12
; SI-NEXT: v_mov_b32_e32 v27, s13
-; SI-NEXT: v_readlane_b32 s31, v28, 25
-; SI-NEXT: v_readlane_b32 s81, v28, 23
-; SI-NEXT: v_readlane_b32 s80, v28, 22
-; SI-NEXT: v_readlane_b32 s71, v28, 21
-; SI-NEXT: v_readlane_b32 s70, v28, 20
-; SI-NEXT: v_readlane_b32 s69, v28, 19
-; SI-NEXT: v_readlane_b32 s68, v28, 18
-; SI-NEXT: v_readlane_b32 s67, v28, 17
+; SI-NEXT: v_readlane_b32 s31, v28, 26
+; SI-NEXT: v_readlane_b32 s83, v28, 24
+; SI-NEXT: v_readlane_b32 s82, v28, 23
+; SI-NEXT: v_readlane_b32 s81, v28, 22
+; SI-NEXT: v_readlane_b32 s80, v28, 21
+; SI-NEXT: v_readlane_b32 s71, v28, 20
+; SI-NEXT: v_readlane_b32 s70, v28, 19
+; SI-NEXT: v_readlane_b32 s69, v28, 18
+; SI-NEXT: v_readlane_b32 s68, v28, 17
; SI-NEXT: v_readlane_b32 s66, v28, 16
; SI-NEXT: v_readlane_b32 s65, v28, 15
; SI-NEXT: v_readlane_b32 s64, v28, 14
diff --git a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
index bf7a8696a9f7c..9d745069016fd 100644
--- a/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
+++ b/llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
@@ -7077,8 +7077,9 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -7097,169 +7098,195 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB15_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB15_3
; SI-NEXT: .LBB15_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB15_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7268,47 +7295,47 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -7328,25 +7355,28 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -7360,64 +7390,75 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB15_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -7448,21 +7489,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -7483,6 +7510,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -7512,180 +7540,198 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB15_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB15_3
; VI-NEXT: .LBB15_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB15_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -7694,52 +7740,52 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -7754,28 +7800,35 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -7786,70 +7839,75 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB15_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -7880,19 +7938,7 @@ define inreg <30 x i32> @bitcast_v60i16_to_v30i32_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -11749,8 +11795,9 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -11769,167 +11816,197 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB19_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB19_3
; SI-NEXT: .LBB19_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB19_3: ; %Flow
@@ -11938,9 +12015,9 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB19_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -11950,7 +12027,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -11960,20 +12037,20 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -11983,7 +12060,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -11992,11 +12069,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -12007,7 +12084,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -12016,11 +12093,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -12031,7 +12108,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -12042,29 +12119,34 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -12076,10 +12158,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -12087,12 +12170,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -12100,10 +12185,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -12111,12 +12197,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -12124,10 +12212,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -12135,12 +12224,14 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -12148,10 +12239,11 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -12159,13 +12251,16 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -12213,22 +12308,8 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB19_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -12249,6 +12330,7 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -12278,184 +12360,200 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB19_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB19_3
; VI-NEXT: .LBB19_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB19_3: ; %Flow
@@ -12465,15 +12563,15 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB19_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -12481,113 +12579,129 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -12626,20 +12740,8 @@ define inreg <30 x i32> @bitcast_v60f16_to_v30i32_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB19_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -19522,8 +19624,9 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -19542,169 +19645,195 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB31_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB31_3
; SI-NEXT: .LBB31_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB31_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -19713,47 +19842,47 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -19773,25 +19902,28 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -19805,64 +19937,75 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB31_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -19893,21 +20036,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -19928,6 +20057,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -19957,180 +20087,198 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB31_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB31_3
; VI-NEXT: .LBB31_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB31_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -20139,52 +20287,52 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -20199,28 +20347,35 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -20231,70 +20386,75 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB31_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -20325,19 +20485,7 @@ define inreg <30 x float> @bitcast_v60i16_to_v30f32_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -24574,8 +24722,9 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -24594,167 +24743,197 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB35_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB35_3
; SI-NEXT: .LBB35_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB35_3: ; %Flow
@@ -24763,9 +24942,9 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB35_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -24775,7 +24954,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -24785,20 +24964,20 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -24808,7 +24987,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -24817,11 +24996,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -24832,7 +25011,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -24841,11 +25020,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -24856,7 +25035,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -24867,29 +25046,34 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -24901,10 +25085,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -24912,12 +25097,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -24925,10 +25112,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -24936,12 +25124,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -24949,10 +25139,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -24960,12 +25151,14 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -24973,10 +25166,11 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -24984,13 +25178,16 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -25038,22 +25235,8 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB35_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -25074,6 +25257,7 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -25103,184 +25287,200 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB35_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB35_3
; VI-NEXT: .LBB35_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB35_3: ; %Flow
@@ -25290,15 +25490,15 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: s_cbranch_scc1 .LBB35_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -25306,113 +25506,129 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -25451,20 +25667,8 @@ define inreg <30 x float> @bitcast_v60f16_to_v30f32_scalar(<60 x half> inreg %a,
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB35_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -30478,8 +30682,9 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -30498,169 +30703,195 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB43_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB43_3
; SI-NEXT: .LBB43_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB43_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -30669,47 +30900,47 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -30729,25 +30960,28 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -30761,64 +30995,75 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB43_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -30849,21 +31094,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -30884,6 +31115,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -30913,180 +31145,198 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB43_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB43_3
; VI-NEXT: .LBB43_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB43_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -31095,52 +31345,52 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -31155,28 +31405,35 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -31187,70 +31444,75 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB43_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -31281,19 +31543,7 @@ define inreg <15 x i64> @bitcast_v60i16_to_v15i64_scalar(<60 x i16> inreg %a, i3
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -35206,8 +35456,9 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -35226,167 +35477,197 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB47_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB47_3
; SI-NEXT: .LBB47_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB47_3: ; %Flow
@@ -35395,9 +35676,9 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB47_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -35407,7 +35688,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -35417,20 +35698,20 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -35440,7 +35721,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -35449,11 +35730,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -35464,7 +35745,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -35473,11 +35754,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -35488,7 +35769,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -35499,29 +35780,34 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -35533,10 +35819,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -35544,12 +35831,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -35557,10 +35846,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -35568,12 +35858,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -35581,10 +35873,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -35592,12 +35885,14 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -35605,10 +35900,11 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -35616,13 +35912,16 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -35670,22 +35969,8 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB47_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -35706,6 +35991,7 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -35735,184 +36021,200 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB47_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB47_3
; VI-NEXT: .LBB47_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB47_3: ; %Flow
@@ -35922,15 +36224,15 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: s_cbranch_scc1 .LBB47_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -35938,113 +36240,129 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -36083,20 +36401,8 @@ define inreg <15 x i64> @bitcast_v60f16_to_v15i64_scalar(<60 x half> inreg %a, i
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB47_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -40074,8 +40380,9 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v30, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v30, s34, 0
; SI-NEXT: v_writelane_b32 v30, s35, 1
; SI-NEXT: v_writelane_b32 v30, s36, 2
@@ -40094,169 +40401,195 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_writelane_b32 v30, s65, 15
; SI-NEXT: v_writelane_b32 v30, s66, 16
; SI-NEXT: v_writelane_b32 v30, s67, 17
-; SI-NEXT: v_writelane_b32 v30, s68, 18
-; SI-NEXT: v_writelane_b32 v30, s69, 19
-; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
-; SI-NEXT: v_writelane_b32 v30, s87, 29
-; SI-NEXT: v_writelane_b32 v30, s96, 30
-; SI-NEXT: v_writelane_b32 v30, s97, 31
-; SI-NEXT: v_writelane_b32 v30, s30, 32
-; SI-NEXT: v_writelane_b32 v30, s31, 33
+; SI-NEXT: v_writelane_b32 v30, s30, 18
+; SI-NEXT: v_writelane_b32 v30, s31, 19
; SI-NEXT: v_readfirstlane_b32 s7, v15
+; SI-NEXT: s_lshr_b32 s4, s7, 16
; SI-NEXT: v_readfirstlane_b32 s9, v14
+; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s5, s9, 16
; SI-NEXT: v_readfirstlane_b32 s11, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v31, s4, 0
+; SI-NEXT: s_lshr_b32 s63, s11, 16
; SI-NEXT: v_readfirstlane_b32 s13, v12
+; SI-NEXT: v_writelane_b32 v31, s5, 1
+; SI-NEXT: s_lshr_b32 s62, s13, 16
; SI-NEXT: v_readfirstlane_b32 s15, v11
+; SI-NEXT: v_writelane_b32 v31, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s15, 16
; SI-NEXT: v_readfirstlane_b32 s73, v10
+; SI-NEXT: v_writelane_b32 v31, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s73, 16
; SI-NEXT: v_readfirstlane_b32 s75, v9
+; SI-NEXT: v_writelane_b32 v31, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s75, 16
; SI-NEXT: v_readfirstlane_b32 s77, v8
+; SI-NEXT: v_writelane_b32 v31, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s77, 16
; SI-NEXT: v_readfirstlane_b32 s79, v7
+; SI-NEXT: v_writelane_b32 v31, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s79, 16
; SI-NEXT: v_readfirstlane_b32 s89, v6
+; SI-NEXT: v_writelane_b32 v31, s58, 7
+; SI-NEXT: s_lshr_b32 s64, s89, 16
; SI-NEXT: v_readfirstlane_b32 s93, v5
+; SI-NEXT: v_writelane_b32 v31, s57, 8
+; SI-NEXT: s_lshr_b32 s55, s93, 16
+; SI-NEXT: v_readfirstlane_b32 s52, v1
+; SI-NEXT: v_writelane_b32 v31, s64, 9
; SI-NEXT: v_readfirstlane_b32 vcc_lo, v4
; SI-NEXT: v_readfirstlane_b32 s31, v3
-; SI-NEXT: v_readfirstlane_b32 s96, v2
-; SI-NEXT: v_readfirstlane_b32 s68, v1
-; SI-NEXT: v_readfirstlane_b32 s71, v0
-; SI-NEXT: s_lshr_b32 s90, s29, 16
-; SI-NEXT: s_lshr_b32 s92, s28, 16
+; SI-NEXT: v_readfirstlane_b32 s66, v2
+; SI-NEXT: s_lshr_b32 s51, s52, 16
+; SI-NEXT: v_readfirstlane_b32 s72, v0
+; SI-NEXT: v_writelane_b32 v31, s55, 10
+; SI-NEXT: s_lshr_b32 s12, s29, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
; SI-NEXT: s_lshr_b32 s94, s27, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s26, 16
; SI-NEXT: s_lshr_b32 s34, s25, 16
-; SI-NEXT: s_lshr_b32 s87, s24, 16
-; SI-NEXT: s_lshr_b32 s69, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
-; SI-NEXT: s_lshr_b32 s81, s21, 16
-; SI-NEXT: s_lshr_b32 s82, s20, 16
-; SI-NEXT: s_lshr_b32 s83, s19, 16
-; SI-NEXT: s_lshr_b32 s84, s18, 16
-; SI-NEXT: s_lshr_b32 s85, s17, 16
-; SI-NEXT: s_lshr_b32 s86, s16, 16
-; SI-NEXT: s_lshr_b32 s97, s7, 16
-; SI-NEXT: s_lshr_b32 s8, s9, 16
-; SI-NEXT: s_lshr_b32 s10, s11, 16
-; SI-NEXT: s_lshr_b32 s12, s13, 16
-; SI-NEXT: s_lshr_b32 s14, s15, 16
-; SI-NEXT: s_lshr_b32 s72, s73, 16
-; SI-NEXT: s_lshr_b32 s74, s75, 16
-; SI-NEXT: s_lshr_b32 s76, s77, 16
-; SI-NEXT: s_lshr_b32 s78, s79, 16
-; SI-NEXT: s_lshr_b32 s88, s89, 16
-; SI-NEXT: s_lshr_b32 s91, s93, 16
+; SI-NEXT: s_lshr_b32 s67, s24, 16
+; SI-NEXT: s_lshr_b32 s43, s23, 16
+; SI-NEXT: s_lshr_b32 s8, s22, 16
+; SI-NEXT: s_lshr_b32 s10, s21, 16
+; SI-NEXT: s_lshr_b32 s40, s20, 16
+; SI-NEXT: s_lshr_b32 s14, s19, 16
+; SI-NEXT: s_lshr_b32 s41, s18, 16
+; SI-NEXT: s_lshr_b32 s42, s17, 16
+; SI-NEXT: s_lshr_b32 s44, s16, 16
; SI-NEXT: s_lshr_b32 s95, vcc_lo, 16
; SI-NEXT: s_lshr_b32 s30, s31, 16
-; SI-NEXT: s_lshr_b32 s35, s96, 16
-; SI-NEXT: s_lshr_b32 s4, s68, 16
-; SI-NEXT: s_lshr_b32 s70, s71, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: s_lshr_b32 s35, s66, 16
+; SI-NEXT: s_lshr_b32 s50, s72, 16
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v31, s51, 11
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v31, s52, 12
+; SI-NEXT: v_writelane_b32 v31, s50, 13
; SI-NEXT: s_cbranch_scc0 .LBB51_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s86, 16
-; SI-NEXT: s_or_b32 s36, s5, s6
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s84, 16
-; SI-NEXT: s_or_b32 s38, s5, s6
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s83, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s85, 16
-; SI-NEXT: s_or_b32 s39, s5, s6
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s82, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s6
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s81, 16
-; SI-NEXT: s_or_b32 s41, s5, s6
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s80, 16
-; SI-NEXT: s_or_b32 s42, s5, s6
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s69, 16
-; SI-NEXT: s_or_b32 s43, s5, s6
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s87, 16
-; SI-NEXT: s_or_b32 s44, s5, s6
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s34, 16
-; SI-NEXT: s_or_b32 s45, s5, s6
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s6, vcc_hi, 16
-; SI-NEXT: s_or_b32 s46, s5, s6
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s94, 16
-; SI-NEXT: s_or_b32 s47, s5, s6
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s92, 16
-; SI-NEXT: s_or_b32 s48, s5, s6
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s90, 16
-; SI-NEXT: s_or_b32 s49, s5, s6
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s6, s70, 16
-; SI-NEXT: s_or_b32 s50, s5, s6
-; SI-NEXT: s_and_b32 s5, s68, 0xffff
-; SI-NEXT: s_mov_b32 s6, s4
+; SI-NEXT: s_and_b32 s6, s16, 0xffff
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s36, s6, s44
+; SI-NEXT: s_and_b32 s6, s18, 0xffff
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s38, s6, s41
+; SI-NEXT: s_and_b32 s6, s19, 0xffff
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_lshl_b32 s14, s14, 16
+; SI-NEXT: s_or_b32 s39, s6, s14
+; SI-NEXT: s_and_b32 s6, s20, 0xffff
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s40, s6, s40
+; SI-NEXT: s_and_b32 s6, s21, 0xffff
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_lshl_b32 s10, s10, 16
+; SI-NEXT: s_and_b32 s45, s17, 0xffff
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_or_b32 s41, s6, s10
+; SI-NEXT: s_and_b32 s6, s22, 0xffff
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_lshl_b32 s8, s8, 16
+; SI-NEXT: s_or_b32 s37, s45, s42
+; SI-NEXT: s_or_b32 s42, s6, s8
+; SI-NEXT: s_and_b32 s6, s23, 0xffff
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_lshl_b32 s43, s43, 16
+; SI-NEXT: s_or_b32 s43, s6, s43
+; SI-NEXT: s_and_b32 s6, s24, 0xffff
+; SI-NEXT: s_lshl_b32 s44, s67, 16
+; SI-NEXT: s_or_b32 s44, s6, s44
+; SI-NEXT: s_and_b32 s6, s25, 0xffff
+; SI-NEXT: s_lshl_b32 s45, s34, 16
+; SI-NEXT: s_or_b32 s45, s6, s45
+; SI-NEXT: s_and_b32 s6, s26, 0xffff
+; SI-NEXT: s_lshl_b32 s46, vcc_hi, 16
+; SI-NEXT: s_or_b32 s46, s6, s46
+; SI-NEXT: s_and_b32 s6, s27, 0xffff
+; SI-NEXT: s_lshl_b32 s47, s94, 16
+; SI-NEXT: s_or_b32 s47, s6, s47
+; SI-NEXT: s_and_b32 s74, s28, 0xffff
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s74, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_lshl_b32 s12, s12, 16
+; SI-NEXT: s_or_b32 s49, s56, s12
+; SI-NEXT: s_and_b32 s12, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s50, 16
+; SI-NEXT: s_or_b32 s50, s12, s56
+; SI-NEXT: s_and_b32 s12, s52, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s51, 16
+; SI-NEXT: s_or_b32 s51, s12, s56
+; SI-NEXT: s_and_b32 s12, s66, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s35, 16
+; SI-NEXT: s_or_b32 s52, s12, s56
+; SI-NEXT: s_and_b32 s12, s31, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s30, 16
+; SI-NEXT: s_or_b32 s53, s12, s56
+; SI-NEXT: s_and_b32 s12, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s95, 16
+; SI-NEXT: s_or_b32 s54, s12, s56
+; SI-NEXT: s_and_b32 s12, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s55, 16
+; SI-NEXT: s_or_b32 s55, s12, s56
+; SI-NEXT: s_and_b32 s12, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s56, s64, 16
+; SI-NEXT: s_or_b32 s56, s12, s56
+; SI-NEXT: s_and_b32 s12, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s12, s57
+; SI-NEXT: s_and_b32 s12, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s12, s58
+; SI-NEXT: s_and_b32 s12, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s12, s59
+; SI-NEXT: s_and_b32 s12, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s12, s60
+; SI-NEXT: s_and_b32 s12, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s12, s61
+; SI-NEXT: s_and_b32 s12, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s12, s62
+; SI-NEXT: s_and_b32 s12, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s12, s63
+; SI-NEXT: s_and_b32 s12, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s12, s5
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s51, s5, s4
-; SI-NEXT: s_and_b32 s4, s96, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s35, 16
-; SI-NEXT: s_or_b32 s52, s4, s5
-; SI-NEXT: s_and_b32 s4, s31, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s30, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s65, s4, s5
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_or_b32 s65, s5, s4
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB51_3
; SI-NEXT: .LBB51_2:
-; SI-NEXT: s_mov_b32 s6, s4
-; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s6, s56
+; SI-NEXT: s_mov_b32 s91, s12
+; SI-NEXT: s_mov_b32 s88, s44
+; SI-NEXT: s_mov_b32 s92, s42
+; SI-NEXT: s_mov_b32 s90, s41
+; SI-NEXT: s_mov_b32 s78, s14
+; SI-NEXT: s_mov_b32 s14, s40
+; SI-NEXT: s_mov_b32 s76, s10
+; SI-NEXT: s_mov_b32 s10, s8
+; SI-NEXT: s_mov_b32 s74, s72
+; SI-NEXT: s_mov_b32 s8, s43
+; SI-NEXT: s_mov_b32 s12, s66
+; SI-NEXT: s_mov_b32 s72, s67
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
+; SI-NEXT: s_mov_b64 s[4:5], -1
+; SI-NEXT: s_mov_b32 s67, s72
+; SI-NEXT: s_mov_b32 s66, s12
; SI-NEXT: .LBB51_3: ; %Flow
; SI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; SI-NEXT: s_cselect_b32 s4, 1, 0
@@ -40265,47 +40598,47 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: ; %bb.4: ; %cmp.true
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s88, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_add_i32 s36, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s85, 16
+; SI-NEXT: s_lshl_b32 s5, s92, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s37, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s90, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s38, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s83, 16
+; SI-NEXT: s_lshl_b32 s5, s78, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s39, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s14, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s40, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s81, 16
+; SI-NEXT: s_lshl_b32 s5, s76, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s41, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s10, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s42, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s69, 16
+; SI-NEXT: s_lshl_b32 s5, s8, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s43, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s87, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -40325,25 +40658,28 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s92, 16
+; SI-NEXT: s_lshl_b32 s5, s6, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s48, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s90, 16
+; SI-NEXT: s_lshl_b32 s5, s91, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s71, s71, 3
; SI-NEXT: s_add_i32 s49, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s71, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s68, s68, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 13
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s50, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s68, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s6, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: v_readlane_b32 s4, v31, 12
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 11
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s51, s4, 0x30000
-; SI-NEXT: s_add_i32 s4, s96, 3
+; SI-NEXT: s_add_i32 s4, s66, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
@@ -40357,64 +40693,75 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: s_and_b32 s4, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s5, s95, 16
; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s93, s93, 3
; SI-NEXT: s_add_i32 s54, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s93, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: s_add_i32 s93, s93, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 10
+; SI-NEXT: s_and_b32 s5, s93, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s55, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s89, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s88, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: s_add_i32 s89, s89, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 9
+; SI-NEXT: s_and_b32 s5, s89, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s56, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s79, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s78, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: s_add_i32 s79, s79, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 8
+; SI-NEXT: s_and_b32 s5, s79, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s57, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s77, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s76, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: s_add_i32 s77, s77, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 7
+; SI-NEXT: s_and_b32 s5, s77, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s58, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s75, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s74, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: s_add_i32 s75, s75, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 6
+; SI-NEXT: s_and_b32 s5, s75, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s59, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s73, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s72, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: s_add_i32 s73, s73, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 5
+; SI-NEXT: s_and_b32 s5, s73, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s60, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s15, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s14, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: s_add_i32 s15, s15, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 4
+; SI-NEXT: s_and_b32 s5, s15, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s61, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s13, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s12, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: s_add_i32 s13, s13, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 3
+; SI-NEXT: s_and_b32 s5, s13, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s62, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s11, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s10, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: s_add_i32 s11, s11, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 2
+; SI-NEXT: s_and_b32 s5, s11, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s63, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s9, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s8, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
-; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: s_add_i32 s9, s9, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 1
+; SI-NEXT: s_and_b32 s5, s9, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s64, s4, 0x30000
-; SI-NEXT: s_and_b32 s4, s7, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s97, 16
-; SI-NEXT: s_or_b32 s4, s5, s4
+; SI-NEXT: s_add_i32 s7, s7, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 0
+; SI-NEXT: s_and_b32 s5, s7, 0xffff
+; SI-NEXT: s_lshl_b32 s4, s4, 16
+; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s65, s4, 0x30000
; SI-NEXT: .LBB51_5: ; %end
-; SI-NEXT: v_readlane_b32 s30, v30, 32
+; SI-NEXT: v_readlane_b32 s30, v30, 18
; SI-NEXT: v_mov_b32_e32 v0, s36
; SI-NEXT: v_mov_b32_e32 v1, s37
; SI-NEXT: v_mov_b32_e32 v2, s38
@@ -40445,21 +40792,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_mov_b32_e32 v27, s63
; SI-NEXT: v_mov_b32_e32 v28, s64
; SI-NEXT: v_mov_b32_e32 v29, s65
-; SI-NEXT: v_readlane_b32 s31, v30, 33
-; SI-NEXT: v_readlane_b32 s97, v30, 31
-; SI-NEXT: v_readlane_b32 s96, v30, 30
-; SI-NEXT: v_readlane_b32 s87, v30, 29
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
-; SI-NEXT: v_readlane_b32 s70, v30, 20
-; SI-NEXT: v_readlane_b32 s69, v30, 19
-; SI-NEXT: v_readlane_b32 s68, v30, 18
+; SI-NEXT: v_readlane_b32 s31, v30, 19
; SI-NEXT: v_readlane_b32 s67, v30, 17
; SI-NEXT: v_readlane_b32 s66, v30, 16
; SI-NEXT: v_readlane_b32 s65, v30, 15
@@ -40480,6 +40813,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; SI-NEXT: v_readlane_b32 s34, v30, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v30, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v31, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -40509,180 +40843,198 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_writelane_b32 v30, s65, 15
; VI-NEXT: v_writelane_b32 v30, s66, 16
; VI-NEXT: v_writelane_b32 v30, s67, 17
-; VI-NEXT: v_writelane_b32 v30, s68, 18
-; VI-NEXT: v_writelane_b32 v30, s69, 19
-; VI-NEXT: v_writelane_b32 v30, s70, 20
-; VI-NEXT: v_writelane_b32 v30, s71, 21
-; VI-NEXT: v_writelane_b32 v30, s80, 22
-; VI-NEXT: v_writelane_b32 v30, s81, 23
-; VI-NEXT: v_writelane_b32 v30, s82, 24
-; VI-NEXT: v_writelane_b32 v30, s83, 25
-; VI-NEXT: v_writelane_b32 v30, s84, 26
-; VI-NEXT: v_writelane_b32 v30, s85, 27
-; VI-NEXT: v_writelane_b32 v30, s86, 28
-; VI-NEXT: v_writelane_b32 v30, s87, 29
-; VI-NEXT: v_writelane_b32 v30, s30, 30
-; VI-NEXT: v_writelane_b32 v30, s31, 31
+; VI-NEXT: v_writelane_b32 v30, s30, 18
+; VI-NEXT: v_writelane_b32 v30, s31, 19
; VI-NEXT: v_readfirstlane_b32 s7, v15
-; VI-NEXT: s_lshr_b32 s72, s7, 16
+; VI-NEXT: s_lshr_b32 s4, s7, 16
; VI-NEXT: v_readfirstlane_b32 s9, v14
; VI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; VI-NEXT: s_lshr_b32 s64, s9, 16
+; VI-NEXT: s_lshr_b32 s5, s9, 16
; VI-NEXT: v_readfirstlane_b32 s11, v13
-; VI-NEXT: v_writelane_b32 v31, s72, 0
+; VI-NEXT: v_writelane_b32 v31, s4, 0
; VI-NEXT: s_lshr_b32 s63, s11, 16
; VI-NEXT: v_readfirstlane_b32 s13, v12
-; VI-NEXT: v_writelane_b32 v31, s64, 1
+; VI-NEXT: v_writelane_b32 v31, s5, 1
; VI-NEXT: s_lshr_b32 s62, s13, 16
; VI-NEXT: v_readfirstlane_b32 s15, v11
+; VI-NEXT: v_writelane_b32 v31, s63, 2
+; VI-NEXT: s_lshr_b32 s61, s15, 16
; VI-NEXT: v_readfirstlane_b32 s73, v10
+; VI-NEXT: v_writelane_b32 v31, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s73, 16
; VI-NEXT: v_readfirstlane_b32 s75, v9
+; VI-NEXT: v_writelane_b32 v31, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s75, 16
; VI-NEXT: v_readfirstlane_b32 s77, v8
+; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s77, 16
; VI-NEXT: v_readfirstlane_b32 s79, v7
+; VI-NEXT: v_writelane_b32 v31, s59, 6
+; VI-NEXT: s_lshr_b32 s57, s79, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v31, s58, 7
+; VI-NEXT: s_lshr_b32 s64, s89, 16
; VI-NEXT: v_readfirstlane_b32 vcc_hi, v5
+; VI-NEXT: v_writelane_b32 v31, s57, 8
+; VI-NEXT: s_lshr_b32 s55, vcc_hi, 16
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v31, s64, 9
+; VI-NEXT: s_lshr_b32 s53, s52, 16
+; VI-NEXT: v_writelane_b32 v31, s55, 10
+; VI-NEXT: v_readfirstlane_b32 s54, v1
+; VI-NEXT: v_writelane_b32 v31, s53, 11
+; VI-NEXT: s_lshr_b32 s51, s54, 16
+; VI-NEXT: v_writelane_b32 v31, s52, 12
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v31, s51, 13
; VI-NEXT: v_readfirstlane_b32 s34, v4
; VI-NEXT: v_readfirstlane_b32 s67, v3
-; VI-NEXT: v_readfirstlane_b32 s69, v2
-; VI-NEXT: v_readfirstlane_b32 s80, v1
-; VI-NEXT: v_readfirstlane_b32 s83, v0
-; VI-NEXT: v_writelane_b32 v31, s63, 2
-; VI-NEXT: s_lshr_b32 s90, s29, 16
-; VI-NEXT: s_lshr_b32 vcc_lo, s28, 16
+; VI-NEXT: s_lshr_b32 s65, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s54, 14
+; VI-NEXT: s_lshr_b32 s10, s29, 16
+; VI-NEXT: s_lshr_b32 s56, s28, 16
; VI-NEXT: s_lshr_b32 s31, s27, 16
; VI-NEXT: s_lshr_b32 s35, s26, 16
-; VI-NEXT: s_lshr_b32 s4, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s84, s22, 16
-; VI-NEXT: s_lshr_b32 s85, s21, 16
-; VI-NEXT: s_lshr_b32 s86, s20, 16
-; VI-NEXT: s_lshr_b32 s87, s19, 16
-; VI-NEXT: s_lshr_b32 s6, s18, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s44, s24, 16
+; VI-NEXT: s_lshr_b32 s14, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s6, s19, 16
+; VI-NEXT: s_lshr_b32 s43, s18, 16
; VI-NEXT: s_lshr_b32 s8, s17, 16
-; VI-NEXT: s_lshr_b32 s10, s16, 16
-; VI-NEXT: s_lshr_b32 s61, s15, 16
-; VI-NEXT: s_lshr_b32 s60, s73, 16
-; VI-NEXT: s_lshr_b32 s59, s75, 16
-; VI-NEXT: s_lshr_b32 s76, s77, 16
-; VI-NEXT: s_lshr_b32 s78, s79, 16
-; VI-NEXT: s_lshr_b32 s88, s89, 16
-; VI-NEXT: s_lshr_b32 s91, vcc_hi, 16
+; VI-NEXT: s_lshr_b32 s46, s16, 16
; VI-NEXT: s_lshr_b32 s30, s34, 16
; VI-NEXT: s_lshr_b32 s66, s67, 16
-; VI-NEXT: s_lshr_b32 s68, s69, 16
-; VI-NEXT: s_lshr_b32 s71, s80, 16
-; VI-NEXT: s_lshr_b32 s82, s83, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v31, s62, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v31, s61, 4
-; VI-NEXT: v_writelane_b32 v31, s60, 5
+; VI-NEXT: v_readfirstlane_b32 s12, v16
+; VI-NEXT: v_writelane_b32 v31, s65, 15
+; VI-NEXT: s_cmp_lg_u32 s12, 0
+; VI-NEXT: v_writelane_b32 v31, s50, 16
+; VI-NEXT: v_writelane_b32 v31, s10, 17
; VI-NEXT: s_cbranch_scc0 .LBB51_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s12, s10, 16
-; VI-NEXT: s_or_b32 s36, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s12, s6, 16
-; VI-NEXT: s_or_b32 s38, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s12, s87, 16
-; VI-NEXT: s_and_b32 s14, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s40, s8, 16
-; VI-NEXT: s_or_b32 s39, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s12, s86, 16
-; VI-NEXT: s_or_b32 s37, s14, s40
-; VI-NEXT: s_or_b32 s40, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s12, s85, 16
-; VI-NEXT: s_or_b32 s41, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s12, s84, 16
-; VI-NEXT: s_or_b32 s42, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s12, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s12, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s12
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_mov_b32 s12, s4
+; VI-NEXT: s_and_b32 s47, 0xffff, s16
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_and_b32 s72, 0xffff, s17
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_lshl_b32 s8, s8, 16
+; VI-NEXT: s_or_b32 s36, s47, s46
+; VI-NEXT: s_or_b32 s37, s72, s8
+; VI-NEXT: s_and_b32 s46, 0xffff, s18
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s38, s46, s43
+; VI-NEXT: s_and_b32 s43, 0xffff, s19
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_lshl_b32 s6, s6, 16
+; VI-NEXT: s_or_b32 s39, s43, s6
+; VI-NEXT: s_and_b32 s43, 0xffff, s20
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s43, s40
+; VI-NEXT: s_and_b32 s43, 0xffff, s21
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_or_b32 s41, s43, s41
+; VI-NEXT: s_and_b32 s43, 0xffff, s22
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s42, s43, s42
+; VI-NEXT: s_and_b32 s43, 0xffff, s23
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_lshl_b32 s14, s14, 16
+; VI-NEXT: s_or_b32 s43, s43, s14
+; VI-NEXT: s_and_b32 s46, 0xffff, s24
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s44, s46, s44
+; VI-NEXT: s_and_b32 s46, 0xffff, s25
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s46, s45
+; VI-NEXT: s_and_b32 s46, 0xffff, s26
+; VI-NEXT: s_lshl_b32 s47, s35, 16
+; VI-NEXT: s_or_b32 s46, s46, s47
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_lshl_b32 s72, s31, 16
+; VI-NEXT: s_or_b32 s47, s47, s72
+; VI-NEXT: s_and_b32 s91, 0xffff, s28
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_lshl_b32 s56, s56, 16
+; VI-NEXT: s_or_b32 s48, s91, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s29
+; VI-NEXT: s_lshl_b32 s10, s10, 16
+; VI-NEXT: s_or_b32 s49, s56, s10
+; VI-NEXT: s_and_b32 s10, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s56, s65, 16
+; VI-NEXT: s_or_b32 s50, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s54
+; VI-NEXT: s_lshl_b32 s56, s51, 16
+; VI-NEXT: s_or_b32 s51, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s56, s53, 16
+; VI-NEXT: s_or_b32 s52, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s67
+; VI-NEXT: s_lshl_b32 s56, s66, 16
+; VI-NEXT: s_or_b32 s53, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s34
+; VI-NEXT: s_lshl_b32 s56, s30, 16
+; VI-NEXT: s_or_b32 s54, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, vcc_hi
+; VI-NEXT: s_lshl_b32 s56, s55, 16
+; VI-NEXT: s_or_b32 s55, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s56, s64, 16
+; VI-NEXT: s_or_b32 s56, s10, s56
+; VI-NEXT: s_and_b32 s10, 0xffff, s79
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s57, s10, s57
+; VI-NEXT: s_and_b32 s10, 0xffff, s77
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s10, s58
+; VI-NEXT: s_and_b32 s10, 0xffff, s75
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s10, s59
+; VI-NEXT: s_and_b32 s10, 0xffff, s73
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s10, s60
+; VI-NEXT: s_and_b32 s10, 0xffff, s15
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s10, s61
+; VI-NEXT: s_and_b32 s10, 0xffff, s13
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s10, s62
+; VI-NEXT: s_and_b32 s10, 0xffff, s11
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s10, s63
+; VI-NEXT: s_and_b32 s10, 0xffff, s9
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s10, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s7
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s45, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s5, s35, 16
-; VI-NEXT: s_or_b32 s46, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s5, s31, 16
-; VI-NEXT: s_or_b32 s47, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s28
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
-; VI-NEXT: s_or_b32 s48, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s49, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s83
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s50, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s80
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s51, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s69
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s52, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s67
-; VI-NEXT: s_lshl_b32 s5, s66, 16
-; VI-NEXT: s_or_b32 s53, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s34
-; VI-NEXT: s_lshl_b32 s5, s30, 16
-; VI-NEXT: s_or_b32 s54, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, vcc_hi
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s55, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s79
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s77
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s75
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s73
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s15
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s13
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s11
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s9
-; VI-NEXT: s_lshl_b32 s5, s64, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s7
-; VI-NEXT: s_lshl_b32 s5, s72, 16
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB51_3
; VI-NEXT: .LBB51_2:
-; VI-NEXT: s_mov_b32 s74, s59
-; VI-NEXT: s_mov_b32 s72, s66
-; VI-NEXT: s_mov_b32 s14, s67
+; VI-NEXT: s_mov_b32 s72, s56
+; VI-NEXT: s_mov_b32 s12, s46
+; VI-NEXT: s_mov_b32 s74, s8
+; VI-NEXT: s_mov_b32 s8, s43
+; VI-NEXT: s_mov_b32 vcc_lo, s6
+; VI-NEXT: s_mov_b32 s6, s40
+; VI-NEXT: s_mov_b32 s90, s41
+; VI-NEXT: s_mov_b32 s88, s42
+; VI-NEXT: s_mov_b32 s78, s14
+; VI-NEXT: s_mov_b32 s14, s44
+; VI-NEXT: s_mov_b32 s76, s45
+; VI-NEXT: s_mov_b32 s91, s66
+; VI-NEXT: s_mov_b32 s10, s67
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
-; VI-NEXT: s_mov_b32 s12, s4
; VI-NEXT: s_mov_b64 s[4:5], -1
-; VI-NEXT: s_mov_b32 s67, s14
-; VI-NEXT: s_mov_b32 s66, s72
+; VI-NEXT: s_mov_b32 s67, s10
+; VI-NEXT: s_mov_b32 s66, s91
; VI-NEXT: .LBB51_3: ; %Flow
; VI-NEXT: s_and_b64 s[4:5], s[4:5], exec
; VI-NEXT: s_cselect_b32 s4, 1, 0
@@ -40691,52 +41043,52 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: s_add_i32 s16, s16, 3
; VI-NEXT: s_and_b32 s4, s16, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s10, 16
+; VI-NEXT: s_lshl_b32 s5, s12, 16
; VI-NEXT: s_add_i32 s17, s17, 3
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_and_b32 s5, s17, 0xffff
-; VI-NEXT: s_lshl_b32 s8, s8, 16
-; VI-NEXT: s_or_b32 s5, s8, s5
+; VI-NEXT: s_lshl_b32 s10, s74, 16
+; VI-NEXT: s_or_b32 s5, s10, s5
; VI-NEXT: s_add_i32 s18, s18, 3
; VI-NEXT: s_add_i32 s36, s4, 0x30000
; VI-NEXT: s_add_i32 s37, s5, 0x30000
; VI-NEXT: s_and_b32 s4, s18, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s6, 16
+; VI-NEXT: s_lshl_b32 s5, s8, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s19, s19, 3
; VI-NEXT: s_add_i32 s38, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s19, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s87, 16
+; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s20, s20, 3
; VI-NEXT: s_add_i32 s39, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s20, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s86, 16
+; VI-NEXT: s_lshl_b32 s5, s6, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s21, s21, 3
; VI-NEXT: s_add_i32 s40, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s21, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s85, 16
+; VI-NEXT: s_lshl_b32 s5, s90, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s22, s22, 3
; VI-NEXT: s_add_i32 s41, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s22, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s84, 16
+; VI-NEXT: s_lshl_b32 s5, s88, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s23, s23, 3
; VI-NEXT: s_add_i32 s42, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s23, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s81, 16
+; VI-NEXT: s_lshl_b32 s5, s78, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s24, s24, 3
; VI-NEXT: s_add_i32 s43, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s24, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s70, 16
+; VI-NEXT: s_lshl_b32 s5, s14, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s25, s25, 3
; VI-NEXT: s_add_i32 s44, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s25, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s12, 16
+; VI-NEXT: s_lshl_b32 s5, s76, 16
; VI-NEXT: s_or_b32 s4, s5, s4
; VI-NEXT: s_add_i32 s26, s26, 3
; VI-NEXT: s_add_i32 s45, s4, 0x30000
@@ -40751,28 +41103,35 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_add_i32 s28, s28, 3
; VI-NEXT: s_add_i32 s47, s4, 0x30000
; VI-NEXT: s_and_b32 s4, s28, 0xffff
-; VI-NEXT: s_lshl_b32 s5, vcc_lo, 16
+; VI-NEXT: s_lshl_b32 s5, s72, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s29, s29, 3
; VI-NEXT: s_add_i32 s48, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s29, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s90, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s83, s83, 3
+; VI-NEXT: s_add_i32 s29, s29, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 17
+; VI-NEXT: s_and_b32 s5, s29, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s49, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s83, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s82, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s80, s80, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 16
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 15
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s50, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s80, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s71, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s69, s69, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 14
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 13
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s51, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s69, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s68, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: v_readlane_b32 s4, v31, 12
+; VI-NEXT: s_add_i32 s4, s4, 3
+; VI-NEXT: s_and_b32 s5, s4, 0xffff
+; VI-NEXT: v_readlane_b32 s4, v31, 11
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s52, s4, 0x30000
; VI-NEXT: s_add_i32 s4, s67, 3
; VI-NEXT: s_and_b32 s4, s4, 0xffff
@@ -40783,70 +41142,75 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: s_and_b32 s4, s4, 0xffff
; VI-NEXT: s_lshl_b32 s5, s30, 16
; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
; VI-NEXT: s_add_i32 s54, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, vcc_hi, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s91, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: s_add_i32 vcc_hi, vcc_hi, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 10
+; VI-NEXT: s_and_b32 s5, vcc_hi, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s55, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s89, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s88, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: s_add_i32 s89, s89, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 9
+; VI-NEXT: s_and_b32 s5, s89, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s56, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s79, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s78, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: s_add_i32 s79, s79, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 8
+; VI-NEXT: s_and_b32 s5, s79, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s57, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s77, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s76, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: s_add_i32 s77, s77, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 7
+; VI-NEXT: s_and_b32 s5, s77, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s58, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s75, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s74, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s73, s73, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 5
+; VI-NEXT: s_add_i32 s75, s75, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 6
+; VI-NEXT: s_and_b32 s5, s75, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s59, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s73, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s15, s15, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 4
+; VI-NEXT: s_add_i32 s73, s73, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 5
+; VI-NEXT: s_and_b32 s5, s73, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s60, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s15, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s13, s13, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 3
+; VI-NEXT: s_add_i32 s15, s15, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 4
+; VI-NEXT: s_and_b32 s5, s15, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s61, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s13, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s11, s11, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 2
+; VI-NEXT: s_add_i32 s13, s13, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 3
+; VI-NEXT: s_and_b32 s5, s13, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s62, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s11, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s9, s9, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 1
+; VI-NEXT: s_add_i32 s11, s11, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 2
+; VI-NEXT: s_and_b32 s5, s11, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s63, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s9, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
-; VI-NEXT: s_add_i32 s7, s7, 3
-; VI-NEXT: v_readlane_b32 s5, v31, 0
+; VI-NEXT: s_add_i32 s9, s9, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 1
+; VI-NEXT: s_and_b32 s5, s9, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s64, s4, 0x30000
-; VI-NEXT: s_and_b32 s4, s7, 0xffff
-; VI-NEXT: s_lshl_b32 s5, s5, 16
-; VI-NEXT: s_or_b32 s4, s5, s4
+; VI-NEXT: s_add_i32 s7, s7, 3
+; VI-NEXT: v_readlane_b32 s4, v31, 0
+; VI-NEXT: s_and_b32 s5, s7, 0xffff
+; VI-NEXT: s_lshl_b32 s4, s4, 16
+; VI-NEXT: s_or_b32 s4, s4, s5
; VI-NEXT: s_add_i32 s65, s4, 0x30000
; VI-NEXT: .LBB51_5: ; %end
-; VI-NEXT: v_readlane_b32 s30, v30, 30
+; VI-NEXT: v_readlane_b32 s30, v30, 18
; VI-NEXT: v_mov_b32_e32 v0, s36
; VI-NEXT: v_mov_b32_e32 v1, s37
; VI-NEXT: v_mov_b32_e32 v2, s38
@@ -40877,19 +41241,7 @@ define inreg <15 x double> @bitcast_v60i16_to_v15f64_scalar(<60 x i16> inreg %a,
; VI-NEXT: v_mov_b32_e32 v27, s63
; VI-NEXT: v_mov_b32_e32 v28, s64
; VI-NEXT: v_mov_b32_e32 v29, s65
-; VI-NEXT: v_readlane_b32 s31, v30, 31
-; VI-NEXT: v_readlane_b32 s87, v30, 29
-; VI-NEXT: v_readlane_b32 s86, v30, 28
-; VI-NEXT: v_readlane_b32 s85, v30, 27
-; VI-NEXT: v_readlane_b32 s84, v30, 26
-; VI-NEXT: v_readlane_b32 s83, v30, 25
-; VI-NEXT: v_readlane_b32 s82, v30, 24
-; VI-NEXT: v_readlane_b32 s81, v30, 23
-; VI-NEXT: v_readlane_b32 s80, v30, 22
-; VI-NEXT: v_readlane_b32 s71, v30, 21
-; VI-NEXT: v_readlane_b32 s70, v30, 20
-; VI-NEXT: v_readlane_b32 s69, v30, 19
-; VI-NEXT: v_readlane_b32 s68, v30, 18
+; VI-NEXT: v_readlane_b32 s31, v30, 19
; VI-NEXT: v_readlane_b32 s67, v30, 17
; VI-NEXT: v_readlane_b32 s66, v30, 16
; VI-NEXT: v_readlane_b32 s65, v30, 15
@@ -45012,8 +45364,9 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: s_waitcnt vmcnt(0) expcnt(0) lgkmcnt(0)
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_store_dword v32, off, s[0:3], s32 ; 4-byte Folded Spill
+; SI-NEXT: buffer_store_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Spill
; SI-NEXT: s_mov_b64 exec, s[4:5]
-; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: s_waitcnt expcnt(1)
; SI-NEXT: v_writelane_b32 v32, s34, 0
; SI-NEXT: v_writelane_b32 v32, s35, 1
; SI-NEXT: v_writelane_b32 v32, s36, 2
@@ -45032,167 +45385,197 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_writelane_b32 v32, s65, 15
; SI-NEXT: v_writelane_b32 v32, s66, 16
; SI-NEXT: v_writelane_b32 v32, s67, 17
-; SI-NEXT: v_writelane_b32 v32, s68, 18
-; SI-NEXT: v_writelane_b32 v32, s69, 19
-; SI-NEXT: v_writelane_b32 v32, s70, 20
-; SI-NEXT: v_writelane_b32 v32, s71, 21
-; SI-NEXT: v_writelane_b32 v32, s80, 22
-; SI-NEXT: v_writelane_b32 v32, s81, 23
-; SI-NEXT: v_writelane_b32 v32, s82, 24
-; SI-NEXT: v_writelane_b32 v32, s83, 25
-; SI-NEXT: v_writelane_b32 v32, s84, 26
-; SI-NEXT: v_writelane_b32 v32, s85, 27
-; SI-NEXT: v_writelane_b32 v32, s86, 28
-; SI-NEXT: v_writelane_b32 v32, s87, 29
-; SI-NEXT: v_writelane_b32 v32, s96, 30
-; SI-NEXT: v_writelane_b32 v32, s97, 31
-; SI-NEXT: v_writelane_b32 v32, s30, 32
-; SI-NEXT: v_writelane_b32 v32, s31, 33
-; SI-NEXT: v_readfirstlane_b32 s6, v15
+; SI-NEXT: v_writelane_b32 v32, s30, 18
+; SI-NEXT: v_writelane_b32 v32, s31, 19
+; SI-NEXT: v_readfirstlane_b32 s4, v15
+; SI-NEXT: s_lshr_b32 s5, s4, 16
; SI-NEXT: v_readfirstlane_b32 s8, v14
+; SI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; SI-NEXT: s_lshr_b32 s30, s8, 16
; SI-NEXT: v_readfirstlane_b32 s10, v13
+; SI-NEXT: s_waitcnt expcnt(0)
+; SI-NEXT: v_writelane_b32 v33, s5, 0
+; SI-NEXT: s_lshr_b32 s63, s10, 16
; SI-NEXT: v_readfirstlane_b32 s12, v12
+; SI-NEXT: v_writelane_b32 v33, s30, 1
+; SI-NEXT: s_lshr_b32 s62, s12, 16
; SI-NEXT: v_readfirstlane_b32 s14, v11
+; SI-NEXT: v_writelane_b32 v33, s63, 2
+; SI-NEXT: s_lshr_b32 s61, s14, 16
; SI-NEXT: v_readfirstlane_b32 s72, v10
+; SI-NEXT: v_writelane_b32 v33, s62, 3
+; SI-NEXT: s_lshr_b32 s60, s72, 16
; SI-NEXT: v_readfirstlane_b32 s74, v9
+; SI-NEXT: v_writelane_b32 v33, s61, 4
+; SI-NEXT: s_lshr_b32 s59, s74, 16
; SI-NEXT: v_readfirstlane_b32 s76, v8
+; SI-NEXT: v_writelane_b32 v33, s60, 5
+; SI-NEXT: s_lshr_b32 s58, s76, 16
; SI-NEXT: v_readfirstlane_b32 s78, v7
+; SI-NEXT: v_writelane_b32 v33, s59, 6
+; SI-NEXT: s_lshr_b32 s57, s78, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
+; SI-NEXT: v_writelane_b32 v33, s58, 7
+; SI-NEXT: s_lshr_b32 s52, vcc_lo, 16
+; SI-NEXT: v_writelane_b32 v33, s57, 8
; SI-NEXT: v_readfirstlane_b32 s88, v6
+; SI-NEXT: v_readfirstlane_b32 s51, v1
+; SI-NEXT: v_writelane_b32 v33, s52, 9
+; SI-NEXT: s_lshr_b32 s31, s88, 16
+; SI-NEXT: v_writelane_b32 v33, s51, 10
+; SI-NEXT: s_lshr_b32 s53, s51, 16
+; SI-NEXT: v_writelane_b32 v33, s31, 11
; SI-NEXT: v_readfirstlane_b32 s90, v5
+; SI-NEXT: v_readfirstlane_b32 s50, v0
+; SI-NEXT: v_writelane_b32 v33, s53, 12
+; SI-NEXT: s_lshr_b32 s34, s90, 16
+; SI-NEXT: v_writelane_b32 v33, s50, 13
; SI-NEXT: v_readfirstlane_b32 s92, v4
; SI-NEXT: v_readfirstlane_b32 s94, v3
-; SI-NEXT: v_readfirstlane_b32 vcc_lo, v2
-; SI-NEXT: v_readfirstlane_b32 s30, v1
-; SI-NEXT: v_readfirstlane_b32 s35, v0
+; SI-NEXT: s_lshr_b32 s54, s50, 16
+; SI-NEXT: v_writelane_b32 v33, s34, 14
; SI-NEXT: s_lshr_b32 vcc_hi, s29, 16
-; SI-NEXT: s_lshr_b32 s31, s28, 16
-; SI-NEXT: s_lshr_b32 s68, s27, 16
-; SI-NEXT: s_lshr_b32 s70, s26, 16
-; SI-NEXT: s_lshr_b32 s71, s25, 16
-; SI-NEXT: s_lshr_b32 s81, s24, 16
-; SI-NEXT: s_lshr_b32 s82, s23, 16
-; SI-NEXT: s_lshr_b32 s83, s22, 16
-; SI-NEXT: s_lshr_b32 s84, s21, 16
-; SI-NEXT: s_lshr_b32 s85, s20, 16
-; SI-NEXT: s_lshr_b32 s86, s19, 16
-; SI-NEXT: s_lshr_b32 s87, s18, 16
-; SI-NEXT: s_lshr_b32 s96, s17, 16
-; SI-NEXT: s_lshr_b32 s97, s16, 16
-; SI-NEXT: s_lshr_b32 s80, s6, 16
-; SI-NEXT: s_lshr_b32 s9, s8, 16
-; SI-NEXT: s_lshr_b32 s11, s10, 16
-; SI-NEXT: s_lshr_b32 s13, s12, 16
-; SI-NEXT: s_lshr_b32 s15, s14, 16
-; SI-NEXT: s_lshr_b32 s73, s72, 16
-; SI-NEXT: s_lshr_b32 s75, s74, 16
-; SI-NEXT: s_lshr_b32 s77, s76, 16
-; SI-NEXT: s_lshr_b32 s79, s78, 16
-; SI-NEXT: s_lshr_b32 s89, s88, 16
-; SI-NEXT: s_lshr_b32 s91, s90, 16
-; SI-NEXT: s_lshr_b32 s93, s92, 16
+; SI-NEXT: s_lshr_b32 s56, s28, 16
+; SI-NEXT: s_lshr_b32 s47, s27, 16
+; SI-NEXT: s_lshr_b32 s46, s26, 16
+; SI-NEXT: s_lshr_b32 s45, s25, 16
+; SI-NEXT: s_lshr_b32 s44, s24, 16
+; SI-NEXT: s_lshr_b32 s7, s23, 16
+; SI-NEXT: s_lshr_b32 s9, s22, 16
+; SI-NEXT: s_lshr_b32 s41, s21, 16
+; SI-NEXT: s_lshr_b32 s11, s20, 16
+; SI-NEXT: s_lshr_b32 s13, s19, 16
+; SI-NEXT: s_lshr_b32 s40, s18, 16
+; SI-NEXT: s_lshr_b32 s15, s17, 16
+; SI-NEXT: s_lshr_b32 s42, s16, 16
+; SI-NEXT: s_lshr_b32 s35, s92, 16
; SI-NEXT: s_lshr_b32 s95, s94, 16
-; SI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; SI-NEXT: s_lshr_b32 s34, s30, 16
-; SI-NEXT: s_lshr_b32 s69, s35, 16
-; SI-NEXT: v_readfirstlane_b32 s5, v16
-; SI-NEXT: s_cmp_lg_u32 s5, 0
+; SI-NEXT: v_readfirstlane_b32 s6, v16
+; SI-NEXT: v_writelane_b32 v33, s54, 15
+; SI-NEXT: s_cmp_lg_u32 s6, 0
+; SI-NEXT: v_writelane_b32 v33, s35, 16
+; SI-NEXT: v_writelane_b32 v33, s4, 17
; SI-NEXT: s_cbranch_scc0 .LBB55_2
; SI-NEXT: ; %bb.1: ; %cmp.false
-; SI-NEXT: s_and_b32 s5, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s97, 16
-; SI-NEXT: s_or_b32 s36, s5, s7
-; SI-NEXT: s_and_b32 s5, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s87, 16
-; SI-NEXT: s_or_b32 s38, s5, s7
-; SI-NEXT: s_and_b32 s5, s19, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s86, 16
-; SI-NEXT: s_and_b32 s40, s17, 0xffff
-; SI-NEXT: s_lshl_b32 s41, s96, 16
-; SI-NEXT: s_or_b32 s39, s5, s7
-; SI-NEXT: s_and_b32 s5, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s85, 16
-; SI-NEXT: s_or_b32 s37, s40, s41
-; SI-NEXT: s_or_b32 s40, s5, s7
-; SI-NEXT: s_and_b32 s5, s21, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s84, 16
-; SI-NEXT: s_or_b32 s41, s5, s7
-; SI-NEXT: s_and_b32 s5, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s83, 16
-; SI-NEXT: s_or_b32 s42, s5, s7
-; SI-NEXT: s_and_b32 s5, s23, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s82, 16
-; SI-NEXT: s_or_b32 s43, s5, s7
-; SI-NEXT: s_and_b32 s5, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s81, 16
-; SI-NEXT: s_or_b32 s44, s5, s7
-; SI-NEXT: s_and_b32 s5, s25, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s71, 16
-; SI-NEXT: s_or_b32 s45, s5, s7
-; SI-NEXT: s_and_b32 s5, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s70, 16
-; SI-NEXT: s_or_b32 s46, s5, s7
-; SI-NEXT: s_and_b32 s5, s27, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s68, 16
-; SI-NEXT: s_or_b32 s47, s5, s7
-; SI-NEXT: s_and_b32 s5, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s31, 16
-; SI-NEXT: s_or_b32 s48, s5, s7
-; SI-NEXT: s_and_b32 s5, s29, 0xffff
-; SI-NEXT: s_lshl_b32 s7, vcc_hi, 16
-; SI-NEXT: s_or_b32 s49, s5, s7
-; SI-NEXT: s_and_b32 s5, s35, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s69, 16
-; SI-NEXT: s_or_b32 s50, s5, s7
-; SI-NEXT: s_and_b32 s5, s30, 0xffff
-; SI-NEXT: s_lshl_b32 s7, s34, 16
-; SI-NEXT: s_or_b32 s51, s5, s7
-; SI-NEXT: s_and_b32 s5, vcc_lo, 0xffff
-; SI-NEXT: s_mov_b32 s7, s4
-; SI-NEXT: s_lshl_b32 s4, s4, 16
-; SI-NEXT: s_or_b32 s52, s5, s4
-; SI-NEXT: s_and_b32 s4, s94, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s95, 16
-; SI-NEXT: s_or_b32 s53, s4, s5
-; SI-NEXT: s_and_b32 s4, s92, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s93, 16
-; SI-NEXT: s_or_b32 s54, s4, s5
-; SI-NEXT: s_and_b32 s4, s90, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s91, 16
-; SI-NEXT: s_or_b32 s55, s4, s5
-; SI-NEXT: s_and_b32 s4, s88, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s89, 16
-; SI-NEXT: s_or_b32 s56, s4, s5
-; SI-NEXT: s_and_b32 s4, s78, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s79, 16
-; SI-NEXT: s_or_b32 s57, s4, s5
-; SI-NEXT: s_and_b32 s4, s76, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s77, 16
-; SI-NEXT: s_or_b32 s58, s4, s5
-; SI-NEXT: s_and_b32 s4, s74, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s75, 16
-; SI-NEXT: s_or_b32 s59, s4, s5
-; SI-NEXT: s_and_b32 s4, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s73, 16
-; SI-NEXT: s_or_b32 s60, s4, s5
-; SI-NEXT: s_and_b32 s4, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s15, 16
-; SI-NEXT: s_or_b32 s61, s4, s5
-; SI-NEXT: s_and_b32 s4, s12, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s13, 16
-; SI-NEXT: s_or_b32 s62, s4, s5
-; SI-NEXT: s_and_b32 s4, s10, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s11, 16
-; SI-NEXT: s_or_b32 s63, s4, s5
-; SI-NEXT: s_and_b32 s4, s8, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s9, 16
-; SI-NEXT: s_or_b32 s64, s4, s5
-; SI-NEXT: s_and_b32 s4, s6, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_and_b32 s43, s16, 0xffff
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_lshl_b32 s42, s42, 16
+; SI-NEXT: s_and_b32 s73, s17, 0xffff
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_lshl_b32 s15, s15, 16
+; SI-NEXT: s_or_b32 s36, s43, s42
+; SI-NEXT: s_or_b32 s37, s73, s15
+; SI-NEXT: s_and_b32 s42, s18, 0xffff
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_lshl_b32 s40, s40, 16
+; SI-NEXT: s_or_b32 s38, s42, s40
+; SI-NEXT: s_and_b32 s40, s19, 0xffff
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_lshl_b32 s13, s13, 16
+; SI-NEXT: s_or_b32 s39, s40, s13
+; SI-NEXT: s_and_b32 s40, s20, 0xffff
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_lshl_b32 s11, s11, 16
+; SI-NEXT: s_or_b32 s40, s40, s11
+; SI-NEXT: s_and_b32 s42, s21, 0xffff
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_lshl_b32 s41, s41, 16
+; SI-NEXT: s_or_b32 s41, s42, s41
+; SI-NEXT: s_and_b32 s42, s22, 0xffff
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_lshl_b32 s9, s9, 16
+; SI-NEXT: s_or_b32 s42, s42, s9
+; SI-NEXT: s_and_b32 s43, s23, 0xffff
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_lshl_b32 s7, s7, 16
+; SI-NEXT: s_or_b32 s43, s43, s7
+; SI-NEXT: s_and_b32 s73, s24, 0xffff
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_lshl_b32 s44, s44, 16
+; SI-NEXT: s_or_b32 s44, s73, s44
+; SI-NEXT: s_and_b32 s73, s25, 0xffff
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_lshl_b32 s45, s45, 16
+; SI-NEXT: s_or_b32 s45, s73, s45
+; SI-NEXT: s_and_b32 s73, s26, 0xffff
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_lshl_b32 s46, s46, 16
+; SI-NEXT: s_or_b32 s46, s73, s46
+; SI-NEXT: s_and_b32 s73, s27, 0xffff
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_lshl_b32 s47, s47, 16
+; SI-NEXT: s_or_b32 s47, s73, s47
+; SI-NEXT: s_and_b32 s48, s28, 0xffff
+; SI-NEXT: s_mov_b32 s73, s56
+; SI-NEXT: s_lshl_b32 s56, s56, 16
+; SI-NEXT: s_or_b32 s48, s48, s56
+; SI-NEXT: s_and_b32 s56, s29, 0xffff
+; SI-NEXT: s_lshl_b32 s49, vcc_hi, 16
+; SI-NEXT: s_or_b32 s49, s56, s49
+; SI-NEXT: s_and_b32 s56, s50, 0xffff
+; SI-NEXT: s_lshl_b32 s50, s54, 16
+; SI-NEXT: s_or_b32 s50, s56, s50
+; SI-NEXT: s_and_b32 s56, s51, 0xffff
+; SI-NEXT: s_lshl_b32 s51, s53, 16
+; SI-NEXT: s_or_b32 s51, s56, s51
+; SI-NEXT: s_and_b32 s56, vcc_lo, 0xffff
+; SI-NEXT: s_lshl_b32 s52, s52, 16
+; SI-NEXT: s_or_b32 s52, s56, s52
+; SI-NEXT: s_and_b32 s56, s94, 0xffff
+; SI-NEXT: s_lshl_b32 s53, s95, 16
+; SI-NEXT: s_or_b32 s53, s56, s53
+; SI-NEXT: s_and_b32 s56, s92, 0xffff
+; SI-NEXT: s_lshl_b32 s35, s35, 16
+; SI-NEXT: s_or_b32 s54, s56, s35
+; SI-NEXT: s_and_b32 s56, s90, 0xffff
+; SI-NEXT: s_lshl_b32 s34, s34, 16
+; SI-NEXT: s_or_b32 s55, s56, s34
+; SI-NEXT: s_and_b32 s56, s88, 0xffff
+; SI-NEXT: s_lshl_b32 s31, s31, 16
+; SI-NEXT: s_or_b32 s56, s56, s31
+; SI-NEXT: s_and_b32 s31, s78, 0xffff
+; SI-NEXT: s_lshl_b32 s57, s57, 16
+; SI-NEXT: s_or_b32 s57, s31, s57
+; SI-NEXT: s_and_b32 s31, s76, 0xffff
+; SI-NEXT: s_lshl_b32 s58, s58, 16
+; SI-NEXT: s_or_b32 s58, s31, s58
+; SI-NEXT: s_and_b32 s31, s74, 0xffff
+; SI-NEXT: s_lshl_b32 s59, s59, 16
+; SI-NEXT: s_or_b32 s59, s31, s59
+; SI-NEXT: s_and_b32 s31, s72, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s60, 16
+; SI-NEXT: s_or_b32 s60, s31, s60
+; SI-NEXT: s_and_b32 s31, s14, 0xffff
+; SI-NEXT: s_lshl_b32 s61, s61, 16
+; SI-NEXT: s_or_b32 s61, s31, s61
+; SI-NEXT: s_and_b32 s31, s12, 0xffff
+; SI-NEXT: s_lshl_b32 s62, s62, 16
+; SI-NEXT: s_or_b32 s62, s31, s62
+; SI-NEXT: s_and_b32 s31, s10, 0xffff
+; SI-NEXT: s_lshl_b32 s63, s63, 16
+; SI-NEXT: s_or_b32 s63, s31, s63
+; SI-NEXT: s_and_b32 s31, s8, 0xffff
+; SI-NEXT: s_lshl_b32 s30, s30, 16
+; SI-NEXT: s_and_b32 s4, s4, 0xffff
+; SI-NEXT: s_lshl_b32 s5, s5, 16
+; SI-NEXT: s_or_b32 s64, s31, s30
; SI-NEXT: s_or_b32 s65, s4, s5
; SI-NEXT: s_mov_b64 s[4:5], 0
; SI-NEXT: s_branch .LBB55_3
; SI-NEXT: .LBB55_2:
-; SI-NEXT: s_mov_b32 s7, s4
+; SI-NEXT: s_mov_b32 s6, s42
+; SI-NEXT: s_mov_b32 s93, s15
+; SI-NEXT: s_mov_b32 s15, s40
+; SI-NEXT: s_mov_b32 s91, s13
+; SI-NEXT: s_mov_b32 s13, s11
+; SI-NEXT: s_mov_b32 s11, s41
+; SI-NEXT: s_mov_b32 s89, s9
+; SI-NEXT: s_mov_b32 s9, s7
+; SI-NEXT: s_mov_b32 s7, s44
+; SI-NEXT: s_mov_b32 s79, s45
+; SI-NEXT: s_mov_b32 s77, s46
+; SI-NEXT: s_mov_b32 s75, s47
+; SI-NEXT: s_mov_b32 s73, s56
; SI-NEXT: s_mov_b64 s[4:5], -1
; SI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; SI-NEXT: .LBB55_3: ; %Flow
@@ -45201,9 +45584,9 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: s_cmp_lg_u32 s4, 1
; SI-NEXT: s_cbranch_scc1 .LBB55_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: v_cvt_f32_f16_e32 v0, s97
+; SI-NEXT: v_cvt_f32_f16_e32 v0, s6
; SI-NEXT: v_cvt_f32_f16_e32 v1, s16
-; SI-NEXT: v_cvt_f32_f16_e32 v2, s96
+; SI-NEXT: v_cvt_f32_f16_e32 v2, s93
; SI-NEXT: v_cvt_f32_f16_e32 v3, s17
; SI-NEXT: v_add_f32_e32 v0, 0x38000000, v0
; SI-NEXT: v_add_f32_e32 v1, 0x38000000, v1
@@ -45213,7 +45596,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_lshlrev_b32_e32 v0, 16, v0
; SI-NEXT: v_or_b32_e32 v0, v1, v0
-; SI-NEXT: v_cvt_f32_f16_e32 v1, s87
+; SI-NEXT: v_cvt_f32_f16_e32 v1, s15
; SI-NEXT: v_add_f32_e32 v3, 0x38000000, v3
; SI-NEXT: v_cvt_f16_f32_e32 v3, v3
; SI-NEXT: v_cvt_f32_f16_e32 v5, s19
@@ -45223,20 +45606,20 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v2, s18
; SI-NEXT: v_or_b32_e32 v1, v3, v1
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
-; SI-NEXT: v_cvt_f32_f16_e32 v4, s86
+; SI-NEXT: v_cvt_f32_f16_e32 v4, s91
; SI-NEXT: v_add_f32_e32 v2, 0x38000000, v2
; SI-NEXT: v_cvt_f16_f32_e32 v2, v2
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s85
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s13
; SI-NEXT: v_or_b32_e32 v2, v2, v3
; SI-NEXT: v_lshlrev_b32_e32 v3, 16, v4
; SI-NEXT: v_or_b32_e32 v3, v5, v3
; SI-NEXT: v_cvt_f32_f16_e32 v4, s20
; SI-NEXT: v_add_f32_e32 v5, 0x38000000, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s84
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s11
; SI-NEXT: v_cvt_f16_f32_e32 v5, v5
; SI-NEXT: v_add_f32_e32 v4, 0x38000000, v4
; SI-NEXT: v_cvt_f16_f32_e32 v4, v4
@@ -45246,7 +45629,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v7, s21
; SI-NEXT: v_or_b32_e32 v4, v4, v5
; SI-NEXT: v_lshlrev_b32_e32 v5, 16, v6
-; SI-NEXT: v_cvt_f32_f16_e32 v6, s83
+; SI-NEXT: v_cvt_f32_f16_e32 v6, s89
; SI-NEXT: v_cvt_f32_f16_e32 v8, s22
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
@@ -45255,11 +45638,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
; SI-NEXT: v_cvt_f16_f32_e32 v8, v8
; SI-NEXT: v_or_b32_e32 v5, v7, v5
-; SI-NEXT: v_cvt_f32_f16_e32 v7, s82
+; SI-NEXT: v_cvt_f32_f16_e32 v7, s9
; SI-NEXT: v_lshlrev_b32_e32 v6, 16, v6
; SI-NEXT: v_or_b32_e32 v6, v8, v6
; SI-NEXT: v_cvt_f32_f16_e32 v8, s23
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s81
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s7
; SI-NEXT: v_add_f32_e32 v7, 0x38000000, v7
; SI-NEXT: v_cvt_f16_f32_e32 v7, v7
; SI-NEXT: v_add_f32_e32 v8, 0x38000000, v8
@@ -45270,7 +45653,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v10, s24
; SI-NEXT: v_or_b32_e32 v7, v8, v7
; SI-NEXT: v_lshlrev_b32_e32 v8, 16, v9
-; SI-NEXT: v_cvt_f32_f16_e32 v9, s71
+; SI-NEXT: v_cvt_f32_f16_e32 v9, s79
; SI-NEXT: v_cvt_f32_f16_e32 v11, s25
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
@@ -45279,11 +45662,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
; SI-NEXT: v_cvt_f16_f32_e32 v11, v11
; SI-NEXT: v_or_b32_e32 v8, v10, v8
-; SI-NEXT: v_cvt_f32_f16_e32 v10, s70
+; SI-NEXT: v_cvt_f32_f16_e32 v10, s77
; SI-NEXT: v_lshlrev_b32_e32 v9, 16, v9
; SI-NEXT: v_or_b32_e32 v9, v11, v9
; SI-NEXT: v_cvt_f32_f16_e32 v11, s26
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s68
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s75
; SI-NEXT: v_add_f32_e32 v10, 0x38000000, v10
; SI-NEXT: v_cvt_f16_f32_e32 v10, v10
; SI-NEXT: v_add_f32_e32 v11, 0x38000000, v11
@@ -45294,7 +45677,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f32_f16_e32 v13, s27
; SI-NEXT: v_or_b32_e32 v10, v11, v10
; SI-NEXT: v_lshlrev_b32_e32 v11, 16, v12
-; SI-NEXT: v_cvt_f32_f16_e32 v12, s31
+; SI-NEXT: v_cvt_f32_f16_e32 v12, s73
; SI-NEXT: v_cvt_f32_f16_e32 v14, s28
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
@@ -45305,29 +45688,34 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_or_b32_e32 v11, v13, v11
; SI-NEXT: v_cvt_f32_f16_e32 v13, vcc_hi
; SI-NEXT: v_lshlrev_b32_e32 v12, 16, v12
+; SI-NEXT: v_readlane_b32 s4, v33, 15
; SI-NEXT: v_or_b32_e32 v12, v14, v12
; SI-NEXT: v_cvt_f32_f16_e32 v14, s29
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s69
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
; SI-NEXT: v_add_f32_e32 v13, 0x38000000, v13
; SI-NEXT: v_cvt_f16_f32_e32 v13, v13
; SI-NEXT: v_add_f32_e32 v14, 0x38000000, v14
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
; SI-NEXT: v_cvt_f16_f32_e32 v14, v14
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
+; SI-NEXT: v_readlane_b32 s4, v33, 13
; SI-NEXT: v_lshlrev_b32_e32 v13, 16, v13
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s35
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 12
; SI-NEXT: v_or_b32_e32 v13, v14, v13
; SI-NEXT: v_lshlrev_b32_e32 v14, 16, v15
-; SI-NEXT: v_cvt_f32_f16_e32 v15, s34
-; SI-NEXT: v_cvt_f32_f16_e32 v17, s30
+; SI-NEXT: v_cvt_f32_f16_e32 v15, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 10
+; SI-NEXT: v_cvt_f32_f16_e32 v17, s4
; SI-NEXT: v_add_f32_e32 v16, 0x38000000, v16
-; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_add_f32_e32 v15, 0x38000000, v15
+; SI-NEXT: v_cvt_f16_f32_e32 v16, v16
; SI-NEXT: v_cvt_f16_f32_e32 v15, v15
; SI-NEXT: v_add_f32_e32 v17, 0x38000000, v17
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
+; SI-NEXT: v_readlane_b32 s4, v33, 9
; SI-NEXT: v_or_b32_e32 v14, v16, v14
-; SI-NEXT: v_cvt_f32_f16_e32 v16, s7
+; SI-NEXT: v_cvt_f32_f16_e32 v16, s4
; SI-NEXT: v_lshlrev_b32_e32 v15, 16, v15
; SI-NEXT: v_or_b32_e32 v15, v17, v15
; SI-NEXT: v_cvt_f32_f16_e32 v17, vcc_lo
@@ -45339,10 +45727,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v17, v17
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_lshlrev_b32_e32 v16, 16, v16
+; SI-NEXT: v_readlane_b32 s4, v33, 16
; SI-NEXT: v_cvt_f32_f16_e32 v19, s94
; SI-NEXT: v_or_b32_e32 v16, v17, v16
; SI-NEXT: v_lshlrev_b32_e32 v17, 16, v18
-; SI-NEXT: v_cvt_f32_f16_e32 v18, s93
+; SI-NEXT: v_cvt_f32_f16_e32 v18, s4
; SI-NEXT: v_cvt_f32_f16_e32 v20, s92
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
@@ -45350,12 +45739,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v18, v18
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
+; SI-NEXT: v_readlane_b32 s4, v33, 14
; SI-NEXT: v_or_b32_e32 v17, v19, v17
-; SI-NEXT: v_cvt_f32_f16_e32 v19, s91
+; SI-NEXT: v_cvt_f32_f16_e32 v19, s4
; SI-NEXT: v_lshlrev_b32_e32 v18, 16, v18
+; SI-NEXT: v_readlane_b32 s4, v33, 11
; SI-NEXT: v_or_b32_e32 v18, v20, v18
; SI-NEXT: v_cvt_f32_f16_e32 v20, s90
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s89
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_add_f32_e32 v19, 0x38000000, v19
; SI-NEXT: v_cvt_f16_f32_e32 v19, v19
; SI-NEXT: v_add_f32_e32 v20, 0x38000000, v20
@@ -45363,10 +45754,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v20, v20
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_lshlrev_b32_e32 v19, 16, v19
+; SI-NEXT: v_readlane_b32 s4, v33, 8
; SI-NEXT: v_cvt_f32_f16_e32 v22, s88
; SI-NEXT: v_or_b32_e32 v19, v20, v19
; SI-NEXT: v_lshlrev_b32_e32 v20, 16, v21
-; SI-NEXT: v_cvt_f32_f16_e32 v21, s79
+; SI-NEXT: v_cvt_f32_f16_e32 v21, s4
; SI-NEXT: v_cvt_f32_f16_e32 v23, s78
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
@@ -45374,12 +45766,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v21, v21
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
+; SI-NEXT: v_readlane_b32 s4, v33, 7
; SI-NEXT: v_or_b32_e32 v20, v22, v20
-; SI-NEXT: v_cvt_f32_f16_e32 v22, s77
+; SI-NEXT: v_cvt_f32_f16_e32 v22, s4
; SI-NEXT: v_lshlrev_b32_e32 v21, 16, v21
+; SI-NEXT: v_readlane_b32 s4, v33, 6
; SI-NEXT: v_or_b32_e32 v21, v23, v21
; SI-NEXT: v_cvt_f32_f16_e32 v23, s76
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s75
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_add_f32_e32 v22, 0x38000000, v22
; SI-NEXT: v_cvt_f16_f32_e32 v22, v22
; SI-NEXT: v_add_f32_e32 v23, 0x38000000, v23
@@ -45387,10 +45781,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v23, v23
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_lshlrev_b32_e32 v22, 16, v22
+; SI-NEXT: v_readlane_b32 s4, v33, 5
; SI-NEXT: v_cvt_f32_f16_e32 v25, s74
; SI-NEXT: v_or_b32_e32 v22, v23, v22
; SI-NEXT: v_lshlrev_b32_e32 v23, 16, v24
-; SI-NEXT: v_cvt_f32_f16_e32 v24, s73
+; SI-NEXT: v_cvt_f32_f16_e32 v24, s4
; SI-NEXT: v_cvt_f32_f16_e32 v26, s72
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
@@ -45398,12 +45793,14 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v24, v24
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
+; SI-NEXT: v_readlane_b32 s4, v33, 4
; SI-NEXT: v_or_b32_e32 v23, v25, v23
-; SI-NEXT: v_cvt_f32_f16_e32 v25, s15
+; SI-NEXT: v_cvt_f32_f16_e32 v25, s4
; SI-NEXT: v_lshlrev_b32_e32 v24, 16, v24
+; SI-NEXT: v_readlane_b32 s4, v33, 3
; SI-NEXT: v_or_b32_e32 v24, v26, v24
; SI-NEXT: v_cvt_f32_f16_e32 v26, s14
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s13
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_add_f32_e32 v25, 0x38000000, v25
; SI-NEXT: v_cvt_f16_f32_e32 v25, v25
; SI-NEXT: v_add_f32_e32 v26, 0x38000000, v26
@@ -45411,10 +45808,11 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v26, v26
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_lshlrev_b32_e32 v25, 16, v25
+; SI-NEXT: v_readlane_b32 s4, v33, 2
; SI-NEXT: v_cvt_f32_f16_e32 v28, s12
; SI-NEXT: v_or_b32_e32 v25, v26, v25
; SI-NEXT: v_lshlrev_b32_e32 v26, 16, v27
-; SI-NEXT: v_cvt_f32_f16_e32 v27, s11
+; SI-NEXT: v_cvt_f32_f16_e32 v27, s4
; SI-NEXT: v_cvt_f32_f16_e32 v29, s10
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
@@ -45422,13 +45820,16 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_cvt_f16_f32_e32 v27, v27
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
; SI-NEXT: v_cvt_f16_f32_e32 v29, v29
+; SI-NEXT: v_readlane_b32 s4, v33, 1
; SI-NEXT: v_or_b32_e32 v26, v28, v26
-; SI-NEXT: v_cvt_f32_f16_e32 v28, s9
+; SI-NEXT: v_cvt_f32_f16_e32 v28, s4
; SI-NEXT: v_lshlrev_b32_e32 v27, 16, v27
+; SI-NEXT: v_readlane_b32 s4, v33, 0
; SI-NEXT: v_or_b32_e32 v27, v29, v27
; SI-NEXT: v_cvt_f32_f16_e32 v29, s8
-; SI-NEXT: v_cvt_f32_f16_e32 v30, s80
-; SI-NEXT: v_cvt_f32_f16_e32 v31, s6
+; SI-NEXT: v_cvt_f32_f16_e32 v30, s4
+; SI-NEXT: v_readlane_b32 s4, v33, 17
+; SI-NEXT: v_cvt_f32_f16_e32 v31, s4
; SI-NEXT: v_add_f32_e32 v28, 0x38000000, v28
; SI-NEXT: v_cvt_f16_f32_e32 v28, v28
; SI-NEXT: v_add_f32_e32 v29, 0x38000000, v29
@@ -45476,22 +45877,8 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_mov_b32_e32 v30, s66
; SI-NEXT: v_mov_b32_e32 v31, s67
; SI-NEXT: .LBB55_6: ; %end
-; SI-NEXT: v_readlane_b32 s30, v32, 32
-; SI-NEXT: v_readlane_b32 s31, v32, 33
-; SI-NEXT: v_readlane_b32 s97, v32, 31
-; SI-NEXT: v_readlane_b32 s96, v32, 30
-; SI-NEXT: v_readlane_b32 s87, v32, 29
-; SI-NEXT: v_readlane_b32 s86, v32, 28
-; SI-NEXT: v_readlane_b32 s85, v32, 27
-; SI-NEXT: v_readlane_b32 s84, v32, 26
-; SI-NEXT: v_readlane_b32 s83, v32, 25
-; SI-NEXT: v_readlane_b32 s82, v32, 24
-; SI-NEXT: v_readlane_b32 s81, v32, 23
-; SI-NEXT: v_readlane_b32 s80, v32, 22
-; SI-NEXT: v_readlane_b32 s71, v32, 21
-; SI-NEXT: v_readlane_b32 s70, v32, 20
-; SI-NEXT: v_readlane_b32 s69, v32, 19
-; SI-NEXT: v_readlane_b32 s68, v32, 18
+; SI-NEXT: v_readlane_b32 s30, v32, 18
+; SI-NEXT: v_readlane_b32 s31, v32, 19
; SI-NEXT: v_readlane_b32 s67, v32, 17
; SI-NEXT: v_readlane_b32 s66, v32, 16
; SI-NEXT: v_readlane_b32 s65, v32, 15
@@ -45512,6 +45899,7 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; SI-NEXT: v_readlane_b32 s34, v32, 0
; SI-NEXT: s_xor_saveexec_b64 s[4:5], -1
; SI-NEXT: buffer_load_dword v32, off, s[0:3], s32 ; 4-byte Folded Reload
+; SI-NEXT: buffer_load_dword v33, off, s[0:3], s32 offset:4 ; 4-byte Folded Reload
; SI-NEXT: s_mov_b64 exec, s[4:5]
; SI-NEXT: s_waitcnt vmcnt(0)
; SI-NEXT: s_setpc_b64 s[30:31]
@@ -45541,184 +45929,200 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_writelane_b32 v32, s65, 15
; VI-NEXT: v_writelane_b32 v32, s66, 16
; VI-NEXT: v_writelane_b32 v32, s67, 17
-; VI-NEXT: v_writelane_b32 v32, s68, 18
-; VI-NEXT: v_writelane_b32 v32, s69, 19
-; VI-NEXT: v_writelane_b32 v32, s70, 20
-; VI-NEXT: v_writelane_b32 v32, s71, 21
-; VI-NEXT: v_writelane_b32 v32, s80, 22
-; VI-NEXT: v_writelane_b32 v32, s81, 23
-; VI-NEXT: v_writelane_b32 v32, s82, 24
-; VI-NEXT: v_writelane_b32 v32, s83, 25
-; VI-NEXT: v_writelane_b32 v32, s84, 26
-; VI-NEXT: v_writelane_b32 v32, s85, 27
-; VI-NEXT: v_writelane_b32 v32, s86, 28
-; VI-NEXT: v_writelane_b32 v32, s87, 29
-; VI-NEXT: v_writelane_b32 v32, s30, 30
-; VI-NEXT: v_writelane_b32 v32, s31, 31
+; VI-NEXT: v_writelane_b32 v32, s30, 18
+; VI-NEXT: v_writelane_b32 v32, s31, 19
+; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: s_lshr_b32 s4, s6, 16
; VI-NEXT: v_readfirstlane_b32 s8, v14
-; VI-NEXT: s_lshr_b32 s75, s8, 16
-; VI-NEXT: v_readfirstlane_b32 s10, v13
; VI-NEXT: ; implicit-def: $vgpr33 : SGPR spill to VGPR lane
+; VI-NEXT: s_lshr_b32 s5, s8, 16
+; VI-NEXT: v_readfirstlane_b32 s10, v13
+; VI-NEXT: v_writelane_b32 v33, s4, 0
; VI-NEXT: s_lshr_b32 s63, s10, 16
; VI-NEXT: v_readfirstlane_b32 s12, v12
-; VI-NEXT: v_writelane_b32 v33, s75, 0
+; VI-NEXT: v_writelane_b32 v33, s5, 1
; VI-NEXT: s_lshr_b32 s62, s12, 16
; VI-NEXT: v_readfirstlane_b32 s14, v11
-; VI-NEXT: v_writelane_b32 v33, s63, 1
-; VI-NEXT: v_readfirstlane_b32 s6, v15
+; VI-NEXT: v_writelane_b32 v33, s63, 2
; VI-NEXT: s_lshr_b32 s61, s14, 16
; VI-NEXT: v_readfirstlane_b32 s72, v10
+; VI-NEXT: v_writelane_b32 v33, s62, 3
+; VI-NEXT: s_lshr_b32 s60, s72, 16
; VI-NEXT: v_readfirstlane_b32 s74, v9
+; VI-NEXT: v_writelane_b32 v33, s61, 4
+; VI-NEXT: s_lshr_b32 s59, s74, 16
; VI-NEXT: v_readfirstlane_b32 s76, v8
+; VI-NEXT: v_writelane_b32 v33, s60, 5
+; VI-NEXT: s_lshr_b32 s58, s76, 16
; VI-NEXT: v_readfirstlane_b32 s78, v7
+; VI-NEXT: v_writelane_b32 v33, s59, 6
+; VI-NEXT: s_lshr_b32 s30, s78, 16
; VI-NEXT: v_readfirstlane_b32 s89, v6
+; VI-NEXT: v_writelane_b32 v33, s58, 7
+; VI-NEXT: s_lshr_b32 s31, s89, 16
+; VI-NEXT: v_writelane_b32 v33, s30, 8
+; VI-NEXT: v_readfirstlane_b32 s52, v2
+; VI-NEXT: v_writelane_b32 v33, s31, 9
+; VI-NEXT: v_readfirstlane_b32 s35, v4
+; VI-NEXT: v_writelane_b32 v33, s52, 10
+; VI-NEXT: v_readfirstlane_b32 s55, v3
+; VI-NEXT: v_writelane_b32 v33, s35, 11
+; VI-NEXT: v_readfirstlane_b32 s50, v0
+; VI-NEXT: v_writelane_b32 v33, s55, 12
+; VI-NEXT: v_readfirstlane_b32 s51, v1
+; VI-NEXT: v_writelane_b32 v33, s50, 13
+; VI-NEXT: s_lshr_b32 s53, s55, 16
+; VI-NEXT: v_writelane_b32 v33, s51, 14
+; VI-NEXT: s_lshr_b32 s64, s52, 16
+; VI-NEXT: v_writelane_b32 v33, s53, 15
+; VI-NEXT: s_lshr_b32 s54, s35, 16
+; VI-NEXT: v_writelane_b32 v33, s64, 16
+; VI-NEXT: s_lshr_b32 s65, s51, 16
+; VI-NEXT: v_writelane_b32 v33, s54, 17
; VI-NEXT: v_readfirstlane_b32 vcc_lo, v5
-; VI-NEXT: v_readfirstlane_b32 s30, v4
-; VI-NEXT: v_readfirstlane_b32 s68, v3
-; VI-NEXT: v_readfirstlane_b32 s71, v2
-; VI-NEXT: v_readfirstlane_b32 s82, v1
-; VI-NEXT: v_readfirstlane_b32 s85, v0
-; VI-NEXT: v_writelane_b32 v33, s62, 2
-; VI-NEXT: s_lshr_b32 s73, s29, 16
-; VI-NEXT: s_lshr_b32 s56, s28, 16
-; VI-NEXT: s_lshr_b32 vcc_hi, s27, 16
-; VI-NEXT: s_lshr_b32 s31, s26, 16
-; VI-NEXT: s_lshr_b32 s35, s25, 16
-; VI-NEXT: s_lshr_b32 s70, s24, 16
-; VI-NEXT: s_lshr_b32 s81, s23, 16
-; VI-NEXT: s_lshr_b32 s83, s22, 16
-; VI-NEXT: s_lshr_b32 s86, s21, 16
-; VI-NEXT: s_lshr_b32 s7, s20, 16
-; VI-NEXT: s_lshr_b32 s9, s19, 16
-; VI-NEXT: s_lshr_b32 s11, s18, 16
-; VI-NEXT: s_lshr_b32 s13, s17, 16
-; VI-NEXT: s_lshr_b32 s15, s16, 16
-; VI-NEXT: s_lshr_b32 s65, s6, 16
-; VI-NEXT: s_lshr_b32 s60, s72, 16
-; VI-NEXT: s_lshr_b32 s59, s74, 16
-; VI-NEXT: s_lshr_b32 s58, s76, 16
-; VI-NEXT: s_lshr_b32 s57, s78, 16
-; VI-NEXT: s_lshr_b32 s79, s89, 16
-; VI-NEXT: s_lshr_b32 s4, vcc_lo, 16
-; VI-NEXT: s_lshr_b32 s34, s30, 16
-; VI-NEXT: s_lshr_b32 s69, s68, 16
-; VI-NEXT: s_lshr_b32 s80, s71, 16
-; VI-NEXT: s_lshr_b32 s84, s82, 16
-; VI-NEXT: s_lshr_b32 s87, s85, 16
-; VI-NEXT: v_readfirstlane_b32 s5, v16
-; VI-NEXT: v_writelane_b32 v33, s61, 3
-; VI-NEXT: s_cmp_lg_u32 s5, 0
-; VI-NEXT: v_writelane_b32 v33, s60, 4
-; VI-NEXT: v_writelane_b32 v33, s59, 5
+; VI-NEXT: s_lshr_b32 s66, s50, 16
+; VI-NEXT: v_writelane_b32 v33, s65, 18
+; VI-NEXT: s_lshr_b32 s56, s29, 16
+; VI-NEXT: s_lshr_b32 s57, s28, 16
+; VI-NEXT: s_lshr_b32 s7, s27, 16
+; VI-NEXT: s_lshr_b32 s46, s26, 16
+; VI-NEXT: s_lshr_b32 s45, s25, 16
+; VI-NEXT: s_lshr_b32 s13, s24, 16
+; VI-NEXT: s_lshr_b32 s15, s23, 16
+; VI-NEXT: s_lshr_b32 s42, s22, 16
+; VI-NEXT: s_lshr_b32 s41, s21, 16
+; VI-NEXT: s_lshr_b32 s40, s20, 16
+; VI-NEXT: s_lshr_b32 s43, s19, 16
+; VI-NEXT: s_lshr_b32 s9, s18, 16
+; VI-NEXT: s_lshr_b32 s44, s17, 16
+; VI-NEXT: s_lshr_b32 s47, s16, 16
+; VI-NEXT: s_lshr_b32 s34, vcc_lo, 16
+; VI-NEXT: v_readfirstlane_b32 s11, v16
+; VI-NEXT: v_writelane_b32 v33, s66, 19
+; VI-NEXT: s_cmp_lg_u32 s11, 0
+; VI-NEXT: v_writelane_b32 v33, s34, 20
+; VI-NEXT: v_writelane_b32 v33, s56, 21
; VI-NEXT: s_cbranch_scc0 .LBB55_2
; VI-NEXT: ; %bb.1: ; %cmp.false
-; VI-NEXT: s_and_b32 s5, 0xffff, s16
-; VI-NEXT: s_lshl_b32 s40, s15, 16
-; VI-NEXT: s_or_b32 s36, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s18
-; VI-NEXT: s_lshl_b32 s40, s11, 16
-; VI-NEXT: s_or_b32 s38, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s19
-; VI-NEXT: s_lshl_b32 s40, s9, 16
-; VI-NEXT: s_and_b32 s41, 0xffff, s17
-; VI-NEXT: s_lshl_b32 s42, s13, 16
-; VI-NEXT: s_or_b32 s39, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s20
-; VI-NEXT: s_lshl_b32 s40, s7, 16
-; VI-NEXT: s_or_b32 s37, s41, s42
-; VI-NEXT: s_or_b32 s40, s5, s40
-; VI-NEXT: s_and_b32 s5, 0xffff, s21
-; VI-NEXT: s_lshl_b32 s41, s86, 16
-; VI-NEXT: s_or_b32 s41, s5, s41
-; VI-NEXT: s_and_b32 s5, 0xffff, s22
-; VI-NEXT: s_lshl_b32 s42, s83, 16
-; VI-NEXT: s_or_b32 s42, s5, s42
-; VI-NEXT: s_and_b32 s5, 0xffff, s23
-; VI-NEXT: s_lshl_b32 s43, s81, 16
-; VI-NEXT: s_or_b32 s43, s5, s43
-; VI-NEXT: s_and_b32 s5, 0xffff, s24
-; VI-NEXT: s_lshl_b32 s44, s70, 16
-; VI-NEXT: s_or_b32 s44, s5, s44
-; VI-NEXT: s_and_b32 s5, 0xffff, s25
-; VI-NEXT: s_lshl_b32 s45, s35, 16
-; VI-NEXT: s_or_b32 s45, s5, s45
-; VI-NEXT: s_and_b32 s5, 0xffff, s26
-; VI-NEXT: s_lshl_b32 s46, s31, 16
-; VI-NEXT: s_or_b32 s46, s5, s46
-; VI-NEXT: s_and_b32 s5, 0xffff, s27
-; VI-NEXT: s_lshl_b32 s47, vcc_hi, 16
-; VI-NEXT: s_or_b32 s47, s5, s47
-; VI-NEXT: s_and_b32 s5, 0xffff, s28
-; VI-NEXT: s_mov_b32 s91, s56
+; VI-NEXT: s_and_b32 s11, 0xffff, s16
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_lshl_b32 s47, s47, 16
+; VI-NEXT: s_or_b32 s36, s11, s47
+; VI-NEXT: s_and_b32 s11, 0xffff, s18
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_lshl_b32 s9, s9, 16
+; VI-NEXT: s_or_b32 s38, s11, s9
+; VI-NEXT: s_and_b32 s11, 0xffff, s19
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_lshl_b32 s43, s43, 16
+; VI-NEXT: s_or_b32 s39, s11, s43
+; VI-NEXT: s_and_b32 s11, 0xffff, s20
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_lshl_b32 s40, s40, 16
+; VI-NEXT: s_or_b32 s40, s11, s40
+; VI-NEXT: s_and_b32 s11, 0xffff, s21
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_lshl_b32 s41, s41, 16
+; VI-NEXT: s_and_b32 s73, 0xffff, s17
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_lshl_b32 s44, s44, 16
+; VI-NEXT: s_or_b32 s41, s11, s41
+; VI-NEXT: s_and_b32 s11, 0xffff, s22
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_lshl_b32 s42, s42, 16
+; VI-NEXT: s_or_b32 s37, s73, s44
+; VI-NEXT: s_or_b32 s42, s11, s42
+; VI-NEXT: s_and_b32 s11, 0xffff, s23
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_lshl_b32 s15, s15, 16
+; VI-NEXT: s_or_b32 s43, s11, s15
+; VI-NEXT: s_and_b32 s11, 0xffff, s24
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_lshl_b32 s13, s13, 16
+; VI-NEXT: s_or_b32 s44, s11, s13
+; VI-NEXT: s_and_b32 s11, 0xffff, s25
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_lshl_b32 s45, s45, 16
+; VI-NEXT: s_or_b32 s45, s11, s45
+; VI-NEXT: s_and_b32 s11, 0xffff, s26
+; VI-NEXT: s_mov_b32 vcc_hi, s46
+; VI-NEXT: s_lshl_b32 s46, s46, 16
+; VI-NEXT: s_or_b32 s46, s11, s46
+; VI-NEXT: s_and_b32 s47, 0xffff, s27
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_lshl_b32 s7, s7, 16
+; VI-NEXT: s_or_b32 s47, s47, s7
+; VI-NEXT: s_and_b32 s48, 0xffff, s28
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_lshl_b32 s57, s57, 16
+; VI-NEXT: s_or_b32 s48, s48, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s29
; VI-NEXT: s_lshl_b32 s56, s56, 16
-; VI-NEXT: s_or_b32 s48, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s29
-; VI-NEXT: s_lshl_b32 s56, s73, 16
-; VI-NEXT: s_or_b32 s49, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s85
-; VI-NEXT: s_lshl_b32 s56, s87, 16
-; VI-NEXT: s_or_b32 s50, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s82
-; VI-NEXT: s_lshl_b32 s56, s84, 16
-; VI-NEXT: s_or_b32 s51, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s71
-; VI-NEXT: s_lshl_b32 s56, s80, 16
-; VI-NEXT: s_or_b32 s52, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s68
-; VI-NEXT: s_lshl_b32 s56, s69, 16
-; VI-NEXT: s_or_b32 s53, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, s30
-; VI-NEXT: s_lshl_b32 s56, s34, 16
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_or_b32 s54, s5, s56
-; VI-NEXT: s_and_b32 s5, 0xffff, vcc_lo
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_or_b32 s49, s57, s56
+; VI-NEXT: s_and_b32 s56, 0xffff, s50
+; VI-NEXT: s_lshl_b32 s57, s66, 16
+; VI-NEXT: s_or_b32 s50, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s51
+; VI-NEXT: s_lshl_b32 s57, s65, 16
+; VI-NEXT: s_or_b32 s51, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s52
+; VI-NEXT: s_lshl_b32 s57, s64, 16
+; VI-NEXT: s_or_b32 s52, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s55
+; VI-NEXT: s_lshl_b32 s57, s53, 16
+; VI-NEXT: s_or_b32 s53, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s35
+; VI-NEXT: s_lshl_b32 s57, s54, 16
+; VI-NEXT: s_or_b32 s54, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, vcc_lo
+; VI-NEXT: s_lshl_b32 s57, s34, 16
+; VI-NEXT: s_or_b32 s55, s56, s57
+; VI-NEXT: s_and_b32 s56, 0xffff, s89
+; VI-NEXT: s_lshl_b32 s57, s31, 16
+; VI-NEXT: s_or_b32 s56, s56, s57
+; VI-NEXT: s_and_b32 s57, 0xffff, s78
+; VI-NEXT: s_lshl_b32 s30, s30, 16
+; VI-NEXT: s_or_b32 s57, s57, s30
+; VI-NEXT: s_and_b32 s30, 0xffff, s76
+; VI-NEXT: s_lshl_b32 s58, s58, 16
+; VI-NEXT: s_or_b32 s58, s30, s58
+; VI-NEXT: s_and_b32 s30, 0xffff, s74
+; VI-NEXT: s_lshl_b32 s59, s59, 16
+; VI-NEXT: s_or_b32 s59, s30, s59
+; VI-NEXT: s_and_b32 s30, 0xffff, s72
+; VI-NEXT: s_lshl_b32 s60, s60, 16
+; VI-NEXT: s_or_b32 s60, s30, s60
+; VI-NEXT: s_and_b32 s30, 0xffff, s14
+; VI-NEXT: s_lshl_b32 s61, s61, 16
+; VI-NEXT: s_or_b32 s61, s30, s61
+; VI-NEXT: s_and_b32 s30, 0xffff, s12
+; VI-NEXT: s_lshl_b32 s62, s62, 16
+; VI-NEXT: s_or_b32 s62, s30, s62
+; VI-NEXT: s_and_b32 s30, 0xffff, s10
+; VI-NEXT: s_lshl_b32 s63, s63, 16
+; VI-NEXT: s_or_b32 s63, s30, s63
+; VI-NEXT: s_and_b32 s30, 0xffff, s8
+; VI-NEXT: s_lshl_b32 s5, s5, 16
+; VI-NEXT: s_or_b32 s64, s30, s5
+; VI-NEXT: s_and_b32 s5, 0xffff, s6
; VI-NEXT: s_lshl_b32 s4, s4, 16
-; VI-NEXT: s_or_b32 s55, s5, s4
-; VI-NEXT: s_and_b32 s4, 0xffff, s89
-; VI-NEXT: s_lshl_b32 s5, s79, 16
-; VI-NEXT: s_or_b32 s56, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s78
-; VI-NEXT: s_lshl_b32 s5, s57, 16
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_or_b32 s57, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s76
-; VI-NEXT: s_lshl_b32 s5, s58, 16
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_or_b32 s58, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s74
-; VI-NEXT: s_lshl_b32 s5, s59, 16
-; VI-NEXT: s_or_b32 s59, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s72
-; VI-NEXT: s_lshl_b32 s5, s60, 16
-; VI-NEXT: s_or_b32 s60, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s14
-; VI-NEXT: s_lshl_b32 s5, s61, 16
-; VI-NEXT: s_or_b32 s61, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s12
-; VI-NEXT: s_lshl_b32 s5, s62, 16
-; VI-NEXT: s_or_b32 s62, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s10
-; VI-NEXT: s_lshl_b32 s5, s63, 16
-; VI-NEXT: s_or_b32 s63, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s8
-; VI-NEXT: s_lshl_b32 s5, s75, 16
-; VI-NEXT: s_or_b32 s64, s4, s5
-; VI-NEXT: s_and_b32 s4, 0xffff, s6
-; VI-NEXT: s_lshl_b32 s5, s65, 16
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_or_b32 s65, s4, s5
+; VI-NEXT: s_or_b32 s65, s5, s4
; VI-NEXT: s_mov_b64 s[4:5], 0
; VI-NEXT: s_branch .LBB55_3
; VI-NEXT: .LBB55_2:
-; VI-NEXT: s_mov_b32 s77, s79
-; VI-NEXT: s_mov_b32 s91, s56
-; VI-NEXT: s_mov_b32 s90, s73
-; VI-NEXT: s_mov_b32 s79, s57
-; VI-NEXT: s_mov_b32 s88, s58
-; VI-NEXT: s_mov_b32 s75, s65
-; VI-NEXT: s_mov_b32 s73, s4
+; VI-NEXT: s_mov_b32 s11, s7
+; VI-NEXT: s_mov_b32 s7, s57
+; VI-NEXT: s_mov_b32 s91, s47
+; VI-NEXT: s_mov_b32 s88, s44
+; VI-NEXT: s_mov_b32 s90, s9
+; VI-NEXT: s_mov_b32 s9, s43
+; VI-NEXT: s_mov_b32 s77, s40
+; VI-NEXT: s_mov_b32 s79, s41
+; VI-NEXT: s_mov_b32 s75, s42
+; VI-NEXT: s_mov_b32 s73, s15
+; VI-NEXT: s_mov_b32 s15, s13
+; VI-NEXT: s_mov_b32 s13, s45
+; VI-NEXT: s_mov_b32 vcc_hi, s46
; VI-NEXT: s_mov_b64 s[4:5], -1
; VI-NEXT: ; implicit-def: $sgpr36_sgpr37_sgpr38_sgpr39_sgpr40_sgpr41_sgpr42_sgpr43_sgpr44_sgpr45_sgpr46_sgpr47_sgpr48_sgpr49_sgpr50_sgpr51_sgpr52_sgpr53_sgpr54_sgpr55_sgpr56_sgpr57_sgpr58_sgpr59_sgpr60_sgpr61_sgpr62_sgpr63_sgpr64_sgpr65_sgpr66_sgpr67
; VI-NEXT: .LBB55_3: ; %Flow
@@ -45728,15 +46132,15 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: s_cbranch_scc1 .LBB55_5
; VI-NEXT: ; %bb.4: ; %cmp.true
; VI-NEXT: v_mov_b32_e32 v29, 0x200
-; VI-NEXT: v_mov_b32_e32 v0, s15
-; VI-NEXT: v_mov_b32_e32 v2, s13
+; VI-NEXT: v_mov_b32_e32 v0, s91
+; VI-NEXT: v_mov_b32_e32 v2, s88
; VI-NEXT: v_add_f16_sdwa v0, v0, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v1, s16, v29
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s17, v29
; VI-NEXT: v_or_b32_e32 v0, v1, v0
; VI-NEXT: v_or_b32_e32 v1, v3, v2
-; VI-NEXT: v_mov_b32_e32 v2, s11
+; VI-NEXT: v_mov_b32_e32 v2, s90
; VI-NEXT: v_add_f16_sdwa v2, v2, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v3, s18, v29
; VI-NEXT: v_or_b32_e32 v2, v3, v2
@@ -45744,113 +46148,129 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_add_f16_sdwa v3, v3, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v4, s19, v29
; VI-NEXT: v_or_b32_e32 v3, v4, v3
-; VI-NEXT: v_mov_b32_e32 v4, s7
+; VI-NEXT: v_mov_b32_e32 v4, s77
; VI-NEXT: v_add_f16_sdwa v4, v4, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v5, s20, v29
; VI-NEXT: v_or_b32_e32 v4, v5, v4
-; VI-NEXT: v_mov_b32_e32 v5, s86
+; VI-NEXT: v_mov_b32_e32 v5, s79
; VI-NEXT: v_add_f16_sdwa v5, v5, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v6, s21, v29
; VI-NEXT: v_or_b32_e32 v5, v6, v5
-; VI-NEXT: v_mov_b32_e32 v6, s83
+; VI-NEXT: v_mov_b32_e32 v6, s75
; VI-NEXT: v_add_f16_sdwa v6, v6, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v7, s22, v29
; VI-NEXT: v_or_b32_e32 v6, v7, v6
-; VI-NEXT: v_mov_b32_e32 v7, s81
+; VI-NEXT: v_mov_b32_e32 v7, s73
; VI-NEXT: v_add_f16_sdwa v7, v7, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v8, s23, v29
; VI-NEXT: v_or_b32_e32 v7, v8, v7
-; VI-NEXT: v_mov_b32_e32 v8, s70
+; VI-NEXT: v_mov_b32_e32 v8, s15
; VI-NEXT: v_add_f16_sdwa v8, v8, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v9, s24, v29
; VI-NEXT: v_or_b32_e32 v8, v9, v8
-; VI-NEXT: v_mov_b32_e32 v9, s35
+; VI-NEXT: v_mov_b32_e32 v9, s13
; VI-NEXT: v_add_f16_sdwa v9, v9, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v10, s25, v29
; VI-NEXT: v_or_b32_e32 v9, v10, v9
-; VI-NEXT: v_mov_b32_e32 v10, s31
+; VI-NEXT: v_mov_b32_e32 v10, vcc_hi
; VI-NEXT: v_add_f16_sdwa v10, v10, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v11, s26, v29
; VI-NEXT: v_or_b32_e32 v10, v11, v10
-; VI-NEXT: v_mov_b32_e32 v11, vcc_hi
+; VI-NEXT: v_mov_b32_e32 v11, s11
; VI-NEXT: v_add_f16_sdwa v11, v11, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v12, s27, v29
; VI-NEXT: v_or_b32_e32 v11, v12, v11
-; VI-NEXT: v_mov_b32_e32 v12, s91
+; VI-NEXT: v_mov_b32_e32 v12, s7
; VI-NEXT: v_add_f16_sdwa v12, v12, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v13, s28, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 21
; VI-NEXT: v_or_b32_e32 v12, v13, v12
-; VI-NEXT: v_mov_b32_e32 v13, s90
+; VI-NEXT: v_mov_b32_e32 v13, s4
; VI-NEXT: v_add_f16_sdwa v13, v13, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v14, s29, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 19
; VI-NEXT: v_or_b32_e32 v13, v14, v13
-; VI-NEXT: v_mov_b32_e32 v14, s87
+; VI-NEXT: v_mov_b32_e32 v14, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 13
; VI-NEXT: v_add_f16_sdwa v14, v14, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v15, s85, v29
+; VI-NEXT: v_add_f16_e32 v15, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 18
; VI-NEXT: v_or_b32_e32 v14, v15, v14
-; VI-NEXT: v_mov_b32_e32 v15, s84
+; VI-NEXT: v_mov_b32_e32 v15, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 14
; VI-NEXT: v_add_f16_sdwa v15, v15, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v16, s82, v29
+; VI-NEXT: v_add_f16_e32 v16, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 16
; VI-NEXT: v_or_b32_e32 v15, v16, v15
-; VI-NEXT: v_mov_b32_e32 v16, s80
+; VI-NEXT: v_mov_b32_e32 v16, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 10
; VI-NEXT: v_add_f16_sdwa v16, v16, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v17, s71, v29
+; VI-NEXT: v_add_f16_e32 v17, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 15
; VI-NEXT: v_or_b32_e32 v16, v17, v16
-; VI-NEXT: v_mov_b32_e32 v17, s69
+; VI-NEXT: v_mov_b32_e32 v17, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 12
; VI-NEXT: v_add_f16_sdwa v17, v17, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v18, s68, v29
+; VI-NEXT: v_add_f16_e32 v18, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 17
; VI-NEXT: v_or_b32_e32 v17, v18, v17
-; VI-NEXT: v_mov_b32_e32 v18, s34
+; VI-NEXT: v_mov_b32_e32 v18, s4
+; VI-NEXT: v_readlane_b32 s4, v33, 11
; VI-NEXT: v_add_f16_sdwa v18, v18, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
-; VI-NEXT: v_add_f16_e32 v19, s30, v29
+; VI-NEXT: v_add_f16_e32 v19, s4, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 20
; VI-NEXT: v_or_b32_e32 v18, v19, v18
-; VI-NEXT: v_mov_b32_e32 v19, s73
+; VI-NEXT: v_mov_b32_e32 v19, s4
; VI-NEXT: v_add_f16_sdwa v19, v19, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v20, vcc_lo, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 9
; VI-NEXT: v_or_b32_e32 v19, v20, v19
-; VI-NEXT: v_mov_b32_e32 v20, s77
+; VI-NEXT: v_mov_b32_e32 v20, s4
; VI-NEXT: v_add_f16_sdwa v20, v20, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v21, s89, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 8
; VI-NEXT: v_or_b32_e32 v20, v21, v20
-; VI-NEXT: v_mov_b32_e32 v21, s79
+; VI-NEXT: v_mov_b32_e32 v21, s4
; VI-NEXT: v_add_f16_sdwa v21, v21, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v22, s78, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 7
; VI-NEXT: v_or_b32_e32 v21, v22, v21
-; VI-NEXT: v_mov_b32_e32 v22, s88
+; VI-NEXT: v_mov_b32_e32 v22, s4
; VI-NEXT: v_add_f16_sdwa v22, v22, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v23, s76, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 5
+; VI-NEXT: v_readlane_b32 s4, v33, 6
; VI-NEXT: v_or_b32_e32 v22, v23, v22
; VI-NEXT: v_mov_b32_e32 v23, s4
; VI-NEXT: v_add_f16_sdwa v23, v23, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v24, s74, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 4
+; VI-NEXT: v_readlane_b32 s4, v33, 5
; VI-NEXT: v_or_b32_e32 v23, v24, v23
; VI-NEXT: v_mov_b32_e32 v24, s4
; VI-NEXT: v_add_f16_sdwa v24, v24, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v25, s72, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 3
+; VI-NEXT: v_readlane_b32 s4, v33, 4
; VI-NEXT: v_or_b32_e32 v24, v25, v24
; VI-NEXT: v_mov_b32_e32 v25, s4
; VI-NEXT: v_add_f16_sdwa v25, v25, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v26, s14, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 2
+; VI-NEXT: v_readlane_b32 s4, v33, 3
; VI-NEXT: v_or_b32_e32 v25, v26, v25
; VI-NEXT: v_mov_b32_e32 v26, s4
; VI-NEXT: v_add_f16_sdwa v26, v26, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v27, s12, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 1
+; VI-NEXT: v_readlane_b32 s4, v33, 2
; VI-NEXT: v_or_b32_e32 v26, v27, v26
; VI-NEXT: v_mov_b32_e32 v27, s4
; VI-NEXT: v_add_f16_sdwa v27, v27, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v28, s10, v29
-; VI-NEXT: v_readlane_b32 s4, v33, 0
+; VI-NEXT: v_readlane_b32 s4, v33, 1
; VI-NEXT: v_or_b32_e32 v27, v28, v27
; VI-NEXT: v_mov_b32_e32 v28, s4
; VI-NEXT: v_add_f16_sdwa v28, v28, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v30, s8, v29
+; VI-NEXT: v_readlane_b32 s4, v33, 0
; VI-NEXT: v_or_b32_e32 v28, v30, v28
-; VI-NEXT: v_mov_b32_e32 v30, s75
+; VI-NEXT: v_mov_b32_e32 v30, s4
; VI-NEXT: v_add_f16_sdwa v30, v30, v29 dst_sel:WORD_1 dst_unused:UNUSED_PAD src0_sel:DWORD src1_sel:DWORD
; VI-NEXT: v_add_f16_e32 v29, s6, v29
; VI-NEXT: v_or_b32_e32 v29, v29, v30
@@ -45889,20 +46309,8 @@ define inreg <15 x double> @bitcast_v60f16_to_v15f64_scalar(<60 x half> inreg %a
; VI-NEXT: v_mov_b32_e32 v30, s66
; VI-NEXT: v_mov_b32_e32 v31, s67
; VI-NEXT: .LBB55_6: ; %end
-; VI-NEXT: v_readlane_b32 s30, v32, 30
-; VI-NEXT: v_readlane_b32 s31, v32, 31
-; VI-NEXT: v_readlane_b32 s87, v32, 29
-; VI-NEXT: v_readlane_b32 s86, v32, 28
-; VI-NEXT: v_readlane_b32 s85, v32, 27
-; VI-NEXT: v_readlane_b32 s84, v32, 26
-; VI-NEXT: v_readlane_b32 s83, v32, 25
-; VI-NEXT: v_readlane_b32 s82, v32, 24
-; VI-NEXT: v_readlane_b32 s81, v32, 23
-; VI-NEXT: v_readlane_b32 s80, v32, 22
-; VI-NEXT: v_readlane_b32 s71, v32, 21
-; VI-NEXT: v_readlane_b32 s70, v32, 20
-; VI-NEXT: v_readlane_b32 s69, v32, 19
-; VI-NEXT: v_readlane_b32 s68, v32, 18
+; VI-NEXT: v_readlane_b32 s30, v32, 18
+; VI-NEXT: v_readlane_b32 s31, v32, 19
; VI-NEXT: v_readlane_b32 s67, v32, 17
; VI-NEXT: v_readlane_b32 s66, v32, 16
; VI-NEXT: v_readlane_b32 s65, v32, 15
@@ -47778,39 +48186,39 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v30, s68, 18
; SI-NEXT: v_writelane_b32 v30, s69, 19
; SI-NEXT: v_writelane_b32 v30, s70, 20
-; SI-NEXT: v_writelane_b32 v30, s71, 21
-; SI-NEXT: v_writelane_b32 v30, s80, 22
-; SI-NEXT: v_writelane_b32 v30, s81, 23
-; SI-NEXT: v_writelane_b32 v30, s82, 24
-; SI-NEXT: v_writelane_b32 v30, s83, 25
-; SI-NEXT: v_writelane_b32 v30, s84, 26
-; SI-NEXT: v_writelane_b32 v30, s85, 27
-; SI-NEXT: v_writelane_b32 v30, s86, 28
+; SI-NEXT: v_writelane_b32 v30, s80, 21
+; SI-NEXT: v_writelane_b32 v30, s81, 22
+; SI-NEXT: v_writelane_b32 v30, s82, 23
+; SI-NEXT: v_writelane_b32 v30, s83, 24
+; SI-NEXT: v_writelane_b32 v30, s84, 25
+; SI-NEXT: v_writelane_b32 v30, s85, 26
+; SI-NEXT: v_writelane_b32 v30, s86, 27
+; SI-NEXT: v_writelane_b32 v30, s87, 28
; SI-NEXT: v_writelane_b32 v30, s30, 29
; SI-NEXT: v_writelane_b32 v30, s31, 30
-; SI-NEXT: v_readfirstlane_b32 s64, v0
-; SI-NEXT: v_readfirstlane_b32 s65, v2
-; SI-NEXT: s_lshr_b32 s42, s64, 16
+; SI-NEXT: v_readfirstlane_b32 vcc_lo, v0
+; SI-NEXT: v_readfirstlane_b32 s30, v2
+; SI-NEXT: s_lshr_b32 s42, vcc_lo, 16
; SI-NEXT: ; implicit-def: $vgpr31 : SGPR spill to VGPR lane
-; SI-NEXT: v_readfirstlane_b32 s67, v4
-; SI-NEXT: s_lshr_b32 s40, s65, 16
+; SI-NEXT: v_readfirstlane_b32 s34, v4
+; SI-NEXT: s_lshr_b32 s40, s30, 16
; SI-NEXT: s_waitcnt expcnt(0)
; SI-NEXT: v_writelane_b32 v31, s42, 0
-; SI-NEXT: v_readfirstlane_b32 s69, v6
-; SI-NEXT: s_lshr_b32 s14, s67, 16
+; SI-NEXT: v_readfirstlane_b32 s36, v6
+; SI-NEXT: s_lshr_b32 s14, s34, 16
; SI-NEXT: v_writelane_b32 v31, s40, 1
-; SI-NEXT: v_readfirstlane_b32 s71, v8
-; SI-NEXT: s_lshr_b32 s12, s69, 16
+; SI-NEXT: v_readfirstlane_b32 s38, v8
+; SI-NEXT: s_lshr_b32 s12, s36, 16
; SI-NEXT: v_writelane_b32 v31, s14, 2
-; SI-NEXT: s_lshr_b32 s10, s71, 16
+; SI-NEXT: s_lshr_b32 s10, s38, 16
; SI-NEXT: v_writelane_b32 v31, s12, 3
-; SI-NEXT: v_readfirstlane_b32 s81, v10
+; SI-NEXT: v_readfirstlane_b32 s48, v10
; SI-NEXT: v_readfirstlane_b32 s43, v1
; SI-NEXT: v_writelane_b32 v31, s10, 4
-; SI-NEXT: v_readfirstlane_b32 s83, v12
-; SI-NEXT: s_lshr_b32 s8, s81, 16
+; SI-NEXT: v_readfirstlane_b32 s50, v12
+; SI-NEXT: s_lshr_b32 s8, s48, 16
; SI-NEXT: v_writelane_b32 v31, s43, 5
-; SI-NEXT: s_lshr_b32 s6, s83, 16
+; SI-NEXT: s_lshr_b32 s6, s50, 16
; SI-NEXT: v_writelane_b32 v31, s8, 6
; SI-NEXT: v_readfirstlane_b32 s41, v3
; SI-NEXT: v_writelane_b32 v31, s6, 7
@@ -47822,26 +48230,28 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_writelane_b32 v31, s13, 10
; SI-NEXT: v_readfirstlane_b32 s9, v11
; SI-NEXT: v_writelane_b32 v31, s11, 11
-; SI-NEXT: v_readfirstlane_b32 s74, v15
-; SI-NEXT: v_readfirstlane_b32 s85, v14
; SI-NEXT: v_readfirstlane_b32 s7, v13
; SI-NEXT: v_writelane_b32 v31, s9, 12
+; SI-NEXT: v_writelane_b32 v31, s7, 13
+; SI-NEXT: v_readfirstlane_b32 s74, v15
+; SI-NEXT: v_readfirstlane_b32 s54, v14
+; SI-NEXT: v_writelane_b32 v31, vcc_lo, 14
; SI-NEXT: s_lshr_b32 s95, s29, 16
-; SI-NEXT: s_lshr_b32 s86, s28, 16
+; SI-NEXT: s_lshr_b32 s70, s28, 16
; SI-NEXT: s_lshr_b32 s93, s27, 16
-; SI-NEXT: s_lshr_b32 s84, s26, 16
+; SI-NEXT: s_lshr_b32 s69, s26, 16
; SI-NEXT: s_lshr_b32 s91, s25, 16
-; SI-NEXT: s_lshr_b32 s82, s24, 16
+; SI-NEXT: s_lshr_b32 s68, s24, 16
; SI-NEXT: s_lshr_b32 s89, s23, 16
-; SI-NEXT: s_lshr_b32 s80, s22, 16
+; SI-NEXT: s_lshr_b32 s67, s22, 16
; SI-NEXT: s_lshr_b32 s79, s21, 16
-; SI-NEXT: s_lshr_b32 s70, s20, 16
+; SI-NEXT: s_lshr_b32 s66, s20, 16
; SI-NEXT: s_lshr_b32 s77, s19, 16
-; SI-NEXT: s_lshr_b32 s68, s18, 16
+; SI-NEXT: s_lshr_b32 s65, s18, 16
; SI-NEXT: s_lshr_b32 s75, s17, 16
-; SI-NEXT: s_lshr_b32 s66, s16, 16
+; SI-NEXT: s_lshr_b32 s64, s16, 16
; SI-NEXT: s_lshr_b32 s53, s74, 16
-; SI-NEXT: s_lshr_b32 s5, s85, 16
+; SI-NEXT: s_lshr_b32 s5, s54, 16
; SI-NEXT: s_lshr_b32 s51, s7, 16
; SI-NEXT: s_lshr_b32 s49, s9, 16
; SI-NEXT: s_lshr_b32 s39, s11, 16
@@ -47850,10 +48260,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s31, s41, 16
; SI-NEXT: s_lshr_b32 vcc_hi, s43, 16
; SI-NEXT: v_readfirstlane_b32 s4, v16
-; SI-NEXT: v_writelane_b32 v31, s7, 13
+; SI-NEXT: v_writelane_b32 v31, s74, 15
; SI-NEXT: s_cmp_lg_u32 s4, 0
-; SI-NEXT: v_writelane_b32 v31, s74, 14
-; SI-NEXT: v_writelane_b32 v31, s5, 15
+; SI-NEXT: v_writelane_b32 v31, s5, 16
+; SI-NEXT: v_writelane_b32 v31, s30, 17
; SI-NEXT: s_cbranch_scc0 .LBB57_2
; SI-NEXT: ; %bb.1: ; %cmp.false
; SI-NEXT: s_and_b32 s45, s17, 0xffff
@@ -47885,109 +48295,110 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s41, s41, s76
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s76, s35, 16
+; SI-NEXT: s_and_b32 s4, s16, 0xffff
+; SI-NEXT: s_lshl_b32 s72, s64, 16
; SI-NEXT: s_or_b32 s15, s15, s76
; SI-NEXT: s_and_b32 s13, s13, 0xffff
; SI-NEXT: s_lshl_b32 s76, s37, 16
+; SI-NEXT: s_or_b32 s4, s4, s72
; SI-NEXT: s_or_b32 s13, s13, s76
; SI-NEXT: s_and_b32 s11, s11, 0xffff
; SI-NEXT: s_lshl_b32 s76, s39, 16
+; SI-NEXT: s_lshl_b32 s52, s5, 16
+; SI-NEXT: s_mov_b32 s5, s73
+; SI-NEXT: s_lshr_b64 s[72:73], s[72:73], 16
; SI-NEXT: s_or_b32 s11, s11, s76
; SI-NEXT: s_and_b32 s9, s9, 0xffff
; SI-NEXT: s_lshl_b32 s76, s49, 16
+; SI-NEXT: v_writelane_b32 v31, s72, 18
+; SI-NEXT: s_lshl_b32 s62, s65, 16
; SI-NEXT: s_or_b32 s9, s9, s76
; SI-NEXT: s_and_b32 s7, s7, 0xffff
; SI-NEXT: s_lshl_b32 s76, s51, 16
-; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s72, s66, 16
+; SI-NEXT: v_writelane_b32 v31, s73, 19
+; SI-NEXT: s_and_b32 s72, s18, 0xffff
+; SI-NEXT: s_lshl_b32 s60, s66, 16
; SI-NEXT: s_or_b32 s7, s7, s76
; SI-NEXT: s_and_b32 s74, s74, 0xffff
; SI-NEXT: s_lshl_b32 s76, s53, 16
-; SI-NEXT: s_or_b32 s4, s4, s72
-; SI-NEXT: s_lshl_b32 s62, s68, 16
-; SI-NEXT: s_mov_b32 s54, s53
-; SI-NEXT: s_or_b32 s53, s74, s76
-; SI-NEXT: s_mov_b32 s76, s75
-; SI-NEXT: s_lshr_b64 s[74:75], s[72:73], 16
-; SI-NEXT: s_and_b32 s72, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s60, s70, 16
-; SI-NEXT: s_mov_b32 s75, s76
; SI-NEXT: s_or_b32 s72, s72, s62
-; SI-NEXT: s_mov_b32 s78, s77
-; SI-NEXT: s_lshr_b64 s[76:77], s[62:63], 16
+; SI-NEXT: s_lshr_b64 s[84:85], s[62:63], 16
; SI-NEXT: s_and_b32 s62, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s58, s80, 16
-; SI-NEXT: s_mov_b32 s77, s78
+; SI-NEXT: s_lshl_b32 s58, s67, 16
+; SI-NEXT: s_mov_b32 s55, s53
+; SI-NEXT: s_or_b32 s53, s74, s76
; SI-NEXT: s_or_b32 s62, s62, s60
-; SI-NEXT: s_mov_b32 s88, s79
+; SI-NEXT: s_mov_b32 s74, s79
; SI-NEXT: s_lshr_b64 s[78:79], s[60:61], 16
; SI-NEXT: s_and_b32 s60, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s56, s82, 16
-; SI-NEXT: s_mov_b32 s79, s88
+; SI-NEXT: s_lshl_b32 s56, s68, 16
+; SI-NEXT: s_mov_b32 s79, s74
; SI-NEXT: s_or_b32 s60, s60, s58
-; SI-NEXT: s_mov_b32 s90, s89
+; SI-NEXT: s_mov_b32 s74, s89
; SI-NEXT: s_lshr_b64 s[88:89], s[58:59], 16
; SI-NEXT: s_and_b32 s58, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s46, s84, 16
-; SI-NEXT: s_mov_b32 s89, s90
+; SI-NEXT: s_lshl_b32 s46, s69, 16
+; SI-NEXT: s_mov_b32 s89, s74
; SI-NEXT: s_or_b32 s58, s58, s56
-; SI-NEXT: s_mov_b32 s92, s91
+; SI-NEXT: s_mov_b32 s74, s91
; SI-NEXT: s_lshr_b64 s[90:91], s[56:57], 16
; SI-NEXT: s_and_b32 s56, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s86, 16
-; SI-NEXT: s_mov_b32 s91, s92
+; SI-NEXT: s_lshl_b32 s44, s70, 16
+; SI-NEXT: s_mov_b32 s91, s74
; SI-NEXT: s_or_b32 s56, s56, s46
-; SI-NEXT: s_mov_b32 s94, s93
+; SI-NEXT: s_mov_b32 s74, s93
; SI-NEXT: s_lshr_b64 s[92:93], s[46:47], 16
; SI-NEXT: s_and_b32 s46, s28, 0xffff
; SI-NEXT: s_lshl_b32 s42, s42, 16
-; SI-NEXT: s_mov_b32 s93, s94
+; SI-NEXT: s_mov_b32 s93, s74
; SI-NEXT: s_or_b32 s46, s46, s44
-; SI-NEXT: s_mov_b32 vcc_lo, s95
+; SI-NEXT: s_mov_b32 s74, s95
; SI-NEXT: s_lshr_b64 s[94:95], s[44:45], 16
-; SI-NEXT: s_and_b32 s44, s64, 0xffff
+; SI-NEXT: s_and_b32 s44, vcc_lo, 0xffff
; SI-NEXT: s_lshl_b32 s40, s40, 16
-; SI-NEXT: s_mov_b32 s95, vcc_lo
+; SI-NEXT: s_mov_b32 s95, s74
; SI-NEXT: s_or_b32 s44, s44, s42
-; SI-NEXT: s_mov_b32 s30, vcc_hi
+; SI-NEXT: s_mov_b32 s74, vcc_hi
; SI-NEXT: s_lshr_b64 vcc, s[42:43], 16
-; SI-NEXT: s_and_b32 s42, s65, 0xffff
+; SI-NEXT: s_and_b32 s42, s30, 0xffff
; SI-NEXT: s_lshl_b32 s14, s14, 16
-; SI-NEXT: s_mov_b32 vcc_hi, s30
+; SI-NEXT: s_mov_b32 vcc_hi, s74
; SI-NEXT: s_or_b32 s42, s42, s40
-; SI-NEXT: s_mov_b32 s34, s31
+; SI-NEXT: s_mov_b32 s74, s31
; SI-NEXT: s_lshr_b64 s[30:31], s[40:41], 16
-; SI-NEXT: s_and_b32 s40, s67, 0xffff
+; SI-NEXT: s_and_b32 s40, s34, 0xffff
; SI-NEXT: s_lshl_b32 s12, s12, 16
-; SI-NEXT: s_mov_b32 s31, s34
+; SI-NEXT: s_mov_b32 s31, s74
+; SI-NEXT: s_mov_b32 s74, s34
; SI-NEXT: s_or_b32 s40, s40, s14
-; SI-NEXT: s_mov_b32 s36, s35
+; SI-NEXT: s_mov_b32 s76, s35
; SI-NEXT: s_lshr_b64 s[34:35], s[14:15], 16
-; SI-NEXT: s_and_b32 s14, s69, 0xffff
+; SI-NEXT: s_and_b32 s14, s36, 0xffff
; SI-NEXT: s_lshl_b32 s10, s10, 16
-; SI-NEXT: s_mov_b32 s35, s36
; SI-NEXT: s_or_b32 s14, s14, s12
-; SI-NEXT: s_mov_b32 s38, s37
-; SI-NEXT: s_lshr_b64 s[36:37], s[12:13], 16
-; SI-NEXT: s_and_b32 s12, s71, 0xffff
+; SI-NEXT: s_lshr_b64 s[86:87], s[12:13], 16
+; SI-NEXT: s_and_b32 s12, s38, 0xffff
; SI-NEXT: s_lshl_b32 s8, s8, 16
-; SI-NEXT: s_mov_b32 s37, s38
+; SI-NEXT: s_mov_b32 s35, s76
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: s_mov_b32 s81, s38
; SI-NEXT: s_or_b32 s12, s12, s10
-; SI-NEXT: s_mov_b32 s48, s39
+; SI-NEXT: s_mov_b32 s36, s39
; SI-NEXT: s_lshr_b64 s[38:39], s[10:11], 16
-; SI-NEXT: s_and_b32 s10, s81, 0xffff
+; SI-NEXT: s_and_b32 s10, s48, 0xffff
; SI-NEXT: s_lshl_b32 s6, s6, 16
-; SI-NEXT: s_mov_b32 s39, s48
+; SI-NEXT: s_mov_b32 s39, s36
+; SI-NEXT: s_mov_b32 s80, s48
; SI-NEXT: s_or_b32 s10, s10, s8
-; SI-NEXT: s_mov_b32 s50, s49
+; SI-NEXT: s_mov_b32 s36, s49
; SI-NEXT: s_lshr_b64 s[48:49], s[8:9], 16
-; SI-NEXT: s_and_b32 s8, s83, 0xffff
-; SI-NEXT: s_lshl_b32 s52, s5, 16
-; SI-NEXT: s_mov_b32 s49, s50
+; SI-NEXT: s_and_b32 s8, s50, 0xffff
+; SI-NEXT: s_mov_b32 s49, s36
+; SI-NEXT: s_mov_b32 s83, s50
; SI-NEXT: s_or_b32 s8, s8, s6
-; SI-NEXT: s_mov_b32 s55, s51
+; SI-NEXT: s_mov_b32 s36, s51
; SI-NEXT: s_lshr_b64 s[50:51], s[6:7], 16
-; SI-NEXT: s_and_b32 s6, s85, 0xffff
-; SI-NEXT: s_mov_b32 s5, s73
+; SI-NEXT: s_and_b32 s6, s54, 0xffff
; SI-NEXT: s_mov_b32 s73, s63
; SI-NEXT: s_mov_b32 s63, s61
; SI-NEXT: s_mov_b32 s61, s59
@@ -48004,16 +48415,25 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s6, s6, s52
; SI-NEXT: s_mov_b32 s7, s53
; SI-NEXT: s_lshr_b64 s[52:53], s[52:53], 16
-; SI-NEXT: s_mov_b32 s51, s55
-; SI-NEXT: s_mov_b32 s53, s54
+; SI-NEXT: s_mov_b32 s51, s36
+; SI-NEXT: s_mov_b32 s82, s54
+; SI-NEXT: s_mov_b32 s53, s55
; SI-NEXT: s_mov_b64 s[54:55], 0
; SI-NEXT: s_branch .LBB57_3
; SI-NEXT: .LBB57_2:
+; SI-NEXT: ; implicit-def: $sgpr6
+; SI-NEXT: s_mov_b32 s82, s54
+; SI-NEXT: s_mov_b32 s83, s50
+; SI-NEXT: s_mov_b32 s80, s48
+; SI-NEXT: s_mov_b32 s81, s38
+; SI-NEXT: s_mov_b32 s76, s36
+; SI-NEXT: s_mov_b32 s74, s34
; SI-NEXT: s_mov_b64 s[54:55], -1
+; SI-NEXT: v_writelane_b32 v31, s6, 18
; SI-NEXT: ; implicit-def: $sgpr4
-; SI-NEXT: ; implicit-def: $sgpr74
+; SI-NEXT: v_writelane_b32 v31, s7, 19
; SI-NEXT: ; implicit-def: $sgpr72
-; SI-NEXT: ; implicit-def: $sgpr76
+; SI-NEXT: ; implicit-def: $sgpr84
; SI-NEXT: ; implicit-def: $sgpr62
; SI-NEXT: ; implicit-def: $sgpr78
; SI-NEXT: ; implicit-def: $sgpr60
@@ -48031,7 +48451,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr40
; SI-NEXT: ; implicit-def: $sgpr34
; SI-NEXT: ; implicit-def: $sgpr14
-; SI-NEXT: ; implicit-def: $sgpr36
+; SI-NEXT: ; implicit-def: $sgpr86
; SI-NEXT: ; implicit-def: $sgpr12
; SI-NEXT: ; implicit-def: $sgpr38
; SI-NEXT: ; implicit-def: $sgpr10
@@ -48042,25 +48462,27 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: ; implicit-def: $sgpr52
; SI-NEXT: .LBB57_3: ; %Flow
; SI-NEXT: s_and_b64 s[54:55], s[54:55], exec
-; SI-NEXT: s_cselect_b32 s54, 1, 0
-; SI-NEXT: s_cmp_lg_u32 s54, 1
+; SI-NEXT: s_cselect_b32 s36, 1, 0
+; SI-NEXT: s_cmp_lg_u32 s36, 1
+; SI-NEXT: v_readlane_b32 s54, v31, 18
+; SI-NEXT: v_readlane_b32 s55, v31, 19
; SI-NEXT: s_cbranch_scc1 .LBB57_5
; SI-NEXT: ; %bb.4: ; %cmp.true
-; SI-NEXT: s_add_i32 s85, s85, 3
-; SI-NEXT: v_readlane_b32 s4, v31, 15
-; SI-NEXT: s_and_b32 s5, s85, 0xffff
+; SI-NEXT: s_add_i32 s4, s82, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
+; SI-NEXT: v_readlane_b32 s4, v31, 16
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s6, s4, 0x30000
-; SI-NEXT: v_readlane_b32 s4, v31, 14
+; SI-NEXT: v_readlane_b32 s4, v31, 15
; SI-NEXT: s_add_i32 s4, s4, 3
; SI-NEXT: s_and_b32 s4, s4, 0xffff
; SI-NEXT: s_lshl_b32 s5, s53, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s7, s4, 0x30000
-; SI-NEXT: s_add_i32 s83, s83, 3
+; SI-NEXT: s_add_i32 s4, s83, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 7
-; SI-NEXT: s_and_b32 s5, s83, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s8, s4, 0x30000
@@ -48070,9 +48492,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s51, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s9, s4, 0x30000
-; SI-NEXT: s_add_i32 s81, s81, 3
+; SI-NEXT: s_add_i32 s4, s80, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 6
-; SI-NEXT: s_and_b32 s5, s81, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s10, s4, 0x30000
@@ -48082,9 +48504,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s49, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s11, s4, 0x30000
-; SI-NEXT: s_add_i32 s71, s71, 3
+; SI-NEXT: s_add_i32 s4, s81, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 4
-; SI-NEXT: s_and_b32 s5, s71, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s12, s4, 0x30000
@@ -48094,9 +48516,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s39, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s13, s4, 0x30000
-; SI-NEXT: s_add_i32 s69, s69, 3
+; SI-NEXT: s_add_i32 s4, s76, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 3
-; SI-NEXT: s_and_b32 s5, s69, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s14, s4, 0x30000
@@ -48106,9 +48528,9 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s37, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s15, s4, 0x30000
-; SI-NEXT: s_add_i32 s67, s67, 3
+; SI-NEXT: s_add_i32 s4, s74, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 2
-; SI-NEXT: s_and_b32 s5, s67, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s40, s4, 0x30000
@@ -48118,9 +48540,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s35, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s41, s4, 0x30000
-; SI-NEXT: s_add_i32 s65, s65, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 17
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 1
-; SI-NEXT: s_and_b32 s5, s65, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s42, s4, 0x30000
@@ -48130,9 +48553,10 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s5, s31, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s43, s4, 0x30000
-; SI-NEXT: s_add_i32 s64, s64, 3
+; SI-NEXT: v_readlane_b32 s4, v31, 14
+; SI-NEXT: s_add_i32 s4, s4, 3
+; SI-NEXT: s_and_b32 s5, s4, 0xffff
; SI-NEXT: v_readlane_b32 s4, v31, 0
-; SI-NEXT: s_and_b32 s5, s64, 0xffff
; SI-NEXT: s_lshl_b32 s4, s4, 16
; SI-NEXT: s_or_b32 s4, s4, s5
; SI-NEXT: s_add_i32 s44, s4, 0x30000
@@ -48144,7 +48568,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s28, s28, 3
; SI-NEXT: s_add_i32 s45, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s28, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s86, 16
+; SI-NEXT: s_lshl_b32 s5, s70, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s29, s29, 3
; SI-NEXT: s_add_i32 s46, s4, 0x30000
@@ -48154,7 +48578,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s26, s26, 3
; SI-NEXT: s_add_i32 s47, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s26, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s84, 16
+; SI-NEXT: s_lshl_b32 s5, s69, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s27, s27, 3
; SI-NEXT: s_add_i32 s56, s4, 0x30000
@@ -48164,7 +48588,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s24, s24, 3
; SI-NEXT: s_add_i32 s57, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s24, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s82, 16
+; SI-NEXT: s_lshl_b32 s5, s68, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s25, s25, 3
; SI-NEXT: s_add_i32 s58, s4, 0x30000
@@ -48174,7 +48598,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s22, s22, 3
; SI-NEXT: s_add_i32 s59, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s22, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s80, 16
+; SI-NEXT: s_lshl_b32 s5, s67, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s23, s23, 3
; SI-NEXT: s_add_i32 s60, s4, 0x30000
@@ -48184,7 +48608,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s20, s20, 3
; SI-NEXT: s_add_i32 s61, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s20, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s70, 16
+; SI-NEXT: s_lshl_b32 s5, s66, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s21, s21, 3
; SI-NEXT: s_add_i32 s62, s4, 0x30000
@@ -48194,7 +48618,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s18, s18, 3
; SI-NEXT: s_add_i32 s63, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s18, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s68, 16
+; SI-NEXT: s_lshl_b32 s5, s65, 16
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_add_i32 s19, s19, 3
; SI-NEXT: s_add_i32 s72, s4, 0x30000
@@ -48204,7 +48628,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_add_i32 s16, s16, 3
; SI-NEXT: s_add_i32 s73, s4, 0x30000
; SI-NEXT: s_and_b32 s4, s16, 0xffff
-; SI-NEXT: s_lshl_b32 s5, s66, 16
+; SI-NEXT: s_lshl_b32 s5, s64, 16
; SI-NEXT: s_add_i32 s17, s17, 3
; SI-NEXT: s_or_b32 s4, s5, s4
; SI-NEXT: s_and_b32 s5, s17, 0xffff
@@ -48212,8 +48636,6 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_or_b32 s5, s16, s5
; SI-NEXT: s_add_i32 s4, s4, 0x30000
; SI-NEXT: s_add_i32 s5, s5, 0x30000
-; SI-NEXT: s_lshr_b64 s[74:75], s[4:5], 16
-; SI-NEXT: s_lshr_b64 s[76:77], s[72:73], 16
; SI-NEXT: s_lshr_b64 s[78:79], s[62:63], 16
; SI-NEXT: s_lshr_b64 s[88:89], s[60:61], 16
; SI-NEXT: s_lshr_b64 s[90:91], s[58:59], 16
@@ -48222,11 +48644,13 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b64 vcc, s[44:45], 16
; SI-NEXT: s_lshr_b64 s[30:31], s[42:43], 16
; SI-NEXT: s_lshr_b64 s[34:35], s[40:41], 16
-; SI-NEXT: s_lshr_b64 s[36:37], s[14:15], 16
; SI-NEXT: s_lshr_b64 s[38:39], s[12:13], 16
; SI-NEXT: s_lshr_b64 s[48:49], s[10:11], 16
; SI-NEXT: s_lshr_b64 s[50:51], s[8:9], 16
; SI-NEXT: s_lshr_b64 s[52:53], s[6:7], 16
+; SI-NEXT: s_lshr_b64 s[54:55], s[4:5], 16
+; SI-NEXT: s_lshr_b64 s[84:85], s[72:73], 16
+; SI-NEXT: s_lshr_b64 s[86:87], s[14:15], 16
; SI-NEXT: s_lshr_b32 s75, s5, 16
; SI-NEXT: s_lshr_b32 s77, s73, 16
; SI-NEXT: s_lshr_b32 s79, s63, 16
@@ -48244,13 +48668,13 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshr_b32 s53, s7, 16
; SI-NEXT: .LBB57_5: ; %end
; SI-NEXT: s_and_b32 s4, s4, 0xffff
-; SI-NEXT: s_lshl_b32 s16, s74, 16
+; SI-NEXT: s_lshl_b32 s16, s54, 16
; SI-NEXT: s_or_b32 s4, s4, s16
; SI-NEXT: s_and_b32 s5, s5, 0xffff
; SI-NEXT: s_lshl_b32 s16, s75, 16
; SI-NEXT: s_or_b32 s5, s5, s16
; SI-NEXT: s_and_b32 s16, s72, 0xffff
-; SI-NEXT: s_lshl_b32 s17, s76, 16
+; SI-NEXT: s_lshl_b32 s17, s84, 16
; SI-NEXT: s_or_b32 s16, s16, s17
; SI-NEXT: s_and_b32 s17, s73, 0xffff
; SI-NEXT: s_lshl_b32 s18, s77, 16
@@ -48304,7 +48728,7 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: s_lshl_b32 s44, s35, 16
; SI-NEXT: s_or_b32 s41, s41, s44
; SI-NEXT: s_and_b32 s14, s14, 0xffff
-; SI-NEXT: s_lshl_b32 s44, s36, 16
+; SI-NEXT: s_lshl_b32 s44, s86, 16
; SI-NEXT: s_or_b32 s14, s14, s44
; SI-NEXT: s_and_b32 s15, s15, 0xffff
; SI-NEXT: s_lshl_b32 s44, s37, 16
@@ -48365,14 +48789,14 @@ define inreg <60 x half> @bitcast_v60i16_to_v60f16_scalar(<60 x i16> inreg %a, i
; SI-NEXT: v_mov_b32_e32 v28, s6
; SI-NEXT: v_mov_b32_e32 v29, s7
; SI-NEXT: v_readlane_b32 s31, v30, 30
-; SI-NEXT: v_readlane_b32 s86, v30, 28
-; SI-NEXT: v_readlane_b32 s85, v30, 27
-; SI-NEXT: v_readlane_b32 s84, v30, 26
-; SI-NEXT: v_readlane_b32 s83, v30, 25
-; SI-NEXT: v_readlane_b32 s82, v30, 24
-; SI-NEXT: v_readlane_b32 s81, v30, 23
-; SI-NEXT: v_readlane_b32 s80, v30, 22
-; SI-NEXT: v_readlane_b32 s71, v30, 21
+; SI-NEXT: v_readlane_b32 s87, v30, 28
+; SI-NEXT: v_readlane_b32 s86, v30, 27
+; SI-NEXT: v_readlane_b32 s85, v30, 26
+; SI-NEXT: v_readlane_b32 s84, v30, 25
+; SI-NEXT: v_readlane_b32 s83, v30, 24
+; SI-NEXT: v_readlane_b32 s82, v30, 23
+; SI-NEXT: v_readlane_b32 s81, v30, 22
+; SI-NEXT: v_readlane_b32 s80, v30, 21
; SI-NEXT: v_readlane_b32 s70, v30, 20
; SI-NEXT: v_readlane_b32 s69, v30, 19
; SI-NEXT: v_readlane_b32 s68, v30, 18
More information about the llvm-commits
mailing list